Multiclass AUC Scores
Multiclass AUC measures.
AUNU: AUC of each class against the rest, using the uniform class
distribution. Computes the AUC treating a c-dimensional classifier
as c two-dimensional 1-vs-rest classifiers, where classes are assumed to have
uniform distribution, in order to have a measure which is independent
of class distribution change (Fawcett 2001).
AUNP: AUC of each class against the rest, using the a-priori class
distribution. Computes the AUC treating a c-dimensional classifier as c
two-dimensional 1-vs-rest classifiers, taking into account the prior probability of
each class (Fawcett 2001).
AU1U: AUC of each class against each other, using the uniform class
distribution. Computes something like the AUC of c(c - 1) binary classifiers
(all possible pairwise combinations). See Hand (2001) for details.
AU1P: AUC of each class against each other, using the a-priori class
distribution. Computes something like AUC of c(c - 1) binary classifiers
while considering the a-priori distribution of the classes as suggested
in Ferri (2009). Note we deviate from the definition in
Ferri (2009) by a factor of c.
The person implementing this function and writing this very
documentation right now cautions against using this measure because it is
an imperfect generalization of AU1U.
mauc_aunu(truth, prob, na_value = NaN, ...) mauc_aunp(truth, prob, na_value = NaN, ...) mauc_au1u(truth, prob, na_value = NaN, ...) mauc_au1p(truth, prob, na_value = NaN, ...)
truth |
( |
prob |
( |
na_value |
( |
... |
( |
Performance value as numeric(1).
Type: "classif"
Range: [0, 1]
Minimize: FALSE
Required prediction: prob
Fawcett, Tom (2001). “Using rule sets to maximize ROC performance.” In Proceedings 2001 IEEE international conference on data mining, 131–138. IEEE.
Ferri, César, Hernández-Orallo, José, Modroiu, R (2009). “An experimental comparison of performance measures for classification.” Pattern Recognition Letters, 30(1), 27–38. doi: 10.1016/j.patrec.2008.08.010.
Hand, J D, Till, J R (2001). “A simple generalisation of the area under the ROC curve for multiple class classification problems.” Machine learning, 45(2), 171–186.
set.seed(1)
lvls = c("a", "b", "c")
truth = factor(sample(lvls, 10, replace = TRUE), levels = lvls)
prob = matrix(runif(3 * 10), ncol = 3)
colnames(prob) = levels(truth)
mauc_aunu(truth, prob)Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.