Details

Autor(en) / Beteiligte

• Shaha, Ricko
• Talukder, Dipon
• Iqbal, MD. Asif
• Haque, Md. Mokammel

Titel

TOS: A Relative Metric Approach for Model Selection in Machine Learning Solutions

Ist Teil von

• 2021 IEEE International Conference on Robotics, Automation, Artificial-Intelligence and Internet-of-Things (RAAICON), 2021, p.26-31

Ort / Verlag

IEEE

Erscheinungsjahr

2021

Link zum Volltext

Quelle

IEEE Electronic Library (IEL)

Beschreibungen/Notizen

• One of the major tasks for a machine learning practitioner is to determine a classifier that would perform well not only on the test data but also on the unseen data from the real world in the future. Metrics to evaluate machine learning algorithms play a significant role to adopt one model from a wide range of machine learning algorithms to achieve optimal performance. Popular metrics like accuracy, precision, recall, F1 measure are greatly affected by the proportion of the majority class, sometimes provide less distinctive and discriminating value and also neglect the impact of true-negative in the outcome result. Moreover, real-world scenarios are skewed. Depending on the environment, problems can be either Type-I or Type-II error case sensitive and a particular error case has to be minimized whether the other one increases. To address this, we proposed a novel relative metric, Trade Off Score(TOS) with informativeness property, good distinctive property considering the type of sensitivity the environment holds and also immune to an imbalanced dataset. TOS provides a score based on the trade-off between accuracy and error rate according to case sensitivity. The proposed metric also quantifies model efficiency by taking into account all the model performances thus forming a relative-metric. We also evaluated our metrics from three different perspectives to justify the performance along with other metrics.