Metacognitive sensitivity

The ability of confidence to distinguish a person's correct from incorrect responses, or to rank items by later success. It is distinct from metacognitive calibration. A type 2 receiver operating characteristic measures discrimination in a binary-correctness task; other metamemory designs may need other measures.

The first-order sensitivity index that, in a fitted signal detection theory model, would generate the observed confidence discrimination. Model assumptions and sampling uncertainty matter; raw confidence–accuracy association alone does not estimate it.^[ref]

A model-based comparison of meta-d-prime with first-order discrimination, often their ratio. Interpretation becomes unstable when the denominator is close to zero. Efficiency, metacognitive calibration and overall confidence are different quantities.

If confidence $C$ is replaced by a strictly increasing function $g(C)$, the attainable threshold partitions have the same ordering and ties. Consequently the type 2 receiver operating characteristic and its area under the ROC curve are unchanged. Equivalently, $P(C_{\mathrm{correct}}>C_{\mathrm{error}})+\frac{1}{2}P(C_{\mathrm{correct}}=C_{\mathrm{error}})$ is invariant. Calibration need not be preserved: adding a positive constant to probabilities within their allowed range changes their absolute values while preserving ranks. A merely nondecreasing transformation can merge distinct reports and requires additional care.

A receiver operating characteristic treating high confidence as a prediction that a first-order answer was correct. Varying confidence threshold $t$ traces the false-positive and true-positive probabilities in the displayed expression. Raw discrimination can depend on first-order task performance; it is not automatically a performance-independent measure of monitoring quality.