The theory of type II ROC analysis

Abstract

Type II ROC analysis is concerned with the ability of observers to distinguish between their own correct and incorrect decisions. The Type I task considered here is to say which of two possible events has occurred during an observation interval; the Type II task is to decide whether the response made in the Type I task was correct. Each task can be modelled by a different pair of overlapping probability functions of an evidence variable. Equations are derived which give the probability functions for the Type II task in terms of the Type I functions. Because likelihood ratio may or may not be used as decision axis in each task, four kinds of Type II probability function can be obtained. From the general equations for Type II probability functions the following results relating the Type I and Type II tasks were found: 1. A different pair of Type II probability functions is yielded by each criterion used in the Type I task. 2. Under certain conditions, all the members of a family of Type II ROC curves lie between the Type I ROC curve and its reflection in the chance line. 3. It is not necessary to know the Type I functions underlying an experimentally obtained Type I ROC curve in order to predict Type II ROC curves for the associated Type II task. In principle, any pair of probability functions which yield a Type I curve of the same shape as the obtained curve can be used. 4. If X is used as the Type I decision axis, and the likelihood ratio of the resulting Type II probability function is used as the Type II decision axis, then the maximum probability of a correct decision achievable in the Type II task is equal to the maximum probability of a correct decision achievable in the Type I task in which likelihood ratio is used as the decision axis.