Maximizing classifier yield for a given accuracy
conference paper
We propose a novel and intuitive way to quantify the utility of a classifier in cases where automatic classification is deployed as partial replacement of human effort, but accuracy requirements exceed the capabilities of the classifier at hand. in our approach, a binary classifier is combined with a meta-classifier mapping all decisions of the first classifier that do not meet a pre-specified confidence level to a third category: for manual inspection. this ternary classifier can now be evaluated in terms of its yield, where yield is defined as the proportion of observations that can be classified automatically with a pre-specified minimum accuracy.
Topics
TNO Identifier
471516
Source title
20th Belgian-Dutch Conference on Artificial Intelligence, (Belgian/Netherlands Artificial Intelligence Conference) BNAIC 2008, 30-31 October 2008, Enschede, Netherlands
Pages
121-128