Faculty of Informatics and Statistics, Department of Information and Knowledge Engineering (DIKE)

Date and time: September 14 2006 (13:30 – 14:00). Non–standard date or time!

Room: 403 NB


Evaluating Misclassifications in Imbalanced Data


  • William Elazmeh, Ottawa-Carleton Institute for Computer Science, University of Ottawa, Canada

Evaluating classifier performance with ROC curves is popular in the machine learning community. To date, the only method to assess confidence of ROC curves is to construct ROC bands. In the case of severe class imbalance with few instances of the minority class, ROC bands become unreliable. We propose a generic framework for classifier evaluation to identify a segment of an ROC curve in which misclassifications are balanced. Confidence is measured by Tango's 95%–confidence interval for the difference in misclassification in both classes. We test our method with severe class imbalance in a two-class problem. Our evaluation favors classifiers with low numbers of misclassifications in both classes. Our results show that the proposed evaluation method is more confident than ROC bands.

Downloads: slides 1 

Summer School on Multimedia Semantics 2006


  • Jan Nemrava, KIZI, VŠE Praha

Downloads: slides 1 

Powered by Resource Description Framework (RDF)