Type I and type II ROC analysis of change in human decision axis

Abstract

A major aim of this research was to investigate two assumptions often made in the psychological literature. First, that human observers, in a binary choice situation, use a decision axis at least monotonic with likelihood ratio. Second, that their ability to discriminate among stimulus alternatives is equivalent to their ability to discriminate among response alternatives. A single-interval, forced-choice experiment was run. The observers had to decide which of two possible events had occurred, and rate their certainty that their decision was correct. The results indicate that, when an observer is given insufficient information to decide with certainty, he uses a decision axis similar to stimulus magnitude. When, however, he is given knowledge of results (KR) about his performance, he may change his decision axis to one similar to likelihood ratio. The implications of these results were discussed with respect to the common practice of providing KR without treating it as a major independent variable. The theory of Type II ROC curve analysis was developed and used in the assessment of the observers' abilities to evaluate their own decisions. It was shown, both theoretically and empirically, that an observer's ability to discriminate between his own correct and incorrect decisions is not equivalent to his ability to discriminate between stimulus events. The more often an observer was informed by KR that he was making correct stimulus-contingent decisions, the better he was able to evaluate his own performance. Reasons for this improvement were discussed in terms of a reduction in criterion instability for the response-contingent decisions.