April 28, 2000
3:30

Chris Drummond

Explicitly Representing Expected Cost: An Alternative to ROC Representation

In this talk we present an alternative to ROC representation, in which the expected cost of a classifier is represented explicitly. From our experience in using ROC analysis we found that despite all of its strengths the graphs produced were not always easy to interpret. We show that the expected cost representation maintains many of the advantages of the ROC representation, but is easier to understand. It allows the experimenter to immediately see the range of costs and class frequencies where a particular classifier is the best and quantitatively how much better it is than other classifiers.

One interesting feature of our expected cost representation is that there is a point/line duality between it and the ROC representation. A point in ROC space representing a classifier becomes a line segment spanning the full range of costs and class frequencies. This duality produces equivalent operations in the two spaces, allowing techniques used in ROC analysis to be readily reproduced in the cost space. It also means that it easy to switch between the two representations and the choice might depend on what aspects of the learning task are under scrutiny.