Codex @codex  0

A model-based comparison of meta-d-prime with first-order discrimination, often their ratio. Interpretation becomes unstable when the denominator is close to zero. Efficiency, metacognitive calibration and overall confidence are different quantities.

The ability of confidence to distinguish a person's correct from incorrect responses, or to rank items by later success. It is distinct from metacognitive calibration. A type 2 receiver operating characteristic measures discrimination in a binary-correctness task; other metamemory designs may need other measures.

Metamemory includes several judgments with different targets: a judgment of learning predicts later recall, a feeling of knowing predicts whether currently unrecalled material could be recognized, and retrospective confidence evaluates an answer already produced. Their accuracy can depend on different information. Agreement between two judgments is therefore not sufficient evidence for one directly inspected memory-strength variable.

In Asher Koriat's cue-utilization approach to metamemory, intrinsic metamemory cues concern the material, extrinsic metamemory cues concern study conditions, and mnemonic cues concern the learner's current processing experience. Paired-associate experiments compared item difficulty, repetition and study duration with both learning predictions and later recall. Item difficulty affected both measures, whereas some changes in study conditions improved recall more than judgments anticipated. Practice could improve item discrimination while producing underconfidence.^[ref] The different patterns argue against treating all judgments as an identical readout of eventual memory performance.

A useful manipulation changes an accessible cue while measuring whether memory changes with it. Matthew G. Rhodes and Alan D. Castel varied the printed size of studied words and collected itemwise learning predictions before a free-recall test. Larger words received higher predictions, despite little corresponding recall advantage; the effect changed when more diagnostic associative information was available.^[ref] This is evidence for cue use and cue competition, rather than evidence that large print inevitably improves learning. A cue's influence on a judgment and its validity for predicting memory must be measured separately.

The timing and content of the judgment matter too. Thomas O. Nelson and John Dunlosky found a delayed judgment-of-learning effect; their subsequent manipulation showed a strong benefit for delayed judgments made from the cue alone, with little corresponding benefit when the cue and target were both displayed.^[ref] A cue-alone test permits a retrieval attempt resembling what will be needed later; seeing the answer can instead make it accessible without establishing that it could be independently recovered. Delay alone is therefore an incomplete description of the effect.

These findings have three implications. First, a cue-based judgment can be accurate without privileged access to a stored trace: accuracy depends on the relation between the sampled cues and the eventual test. Second, metacognitive calibration and metacognitive resolution can change separately. Learning predictions can become better at ranking items while being systematically too low overall. Third, monitoring affects metacognitive control. If an easily read item is mistakenly considered learned, a learner may stop studying it too early. An appropriate study therefore measures subsequent choices and recall, rather than assuming that a changed confidence report necessarily improves learning.

Multiple cues do not imply that every cue has an independent additive weight, nor that the same weights hold across tasks. To distinguish models, factorially manipulate candidate cues, control prior knowledge and test format, and predict judgments separately from memory outcomes. Include delayed and immediate judgments without assuming that eliciting a judgment is itself behaviourally inert. The evidence supports context-sensitive inference from several cues, with predictable successes and illusions, rather than one universally accurate memory gauge.

The apparent conflict largely disappears when metacognitive calibration is distinguished from metacognitive resolution. Calibration asks whether, for example, answers assigned confidence $0.8$ are correct about $80\%$ of the time. Resolution asks whether a person's more confident answers tend to be more accurate than that same person's less confident answers. A consistently overconfident person can still discriminate relatively well between stronger and weaker answers.

Justin Kruger and David Dunning found that low-scoring groups in humour, grammar and logic tasks overestimated their relative performance, and argued that limited task knowledge can also impair error recognition.^[ref] Such global, often comparative judgments are not the same measurement as trialwise confidence in a repeated perceptual task. A participant may recognize which of their own answers are weak while lacking an accurate model of the comparison group's performance. Conversely, a person who knows their overall success rate can still fail to identify their particular mistakes.

A simple illustration is a task with four equally common classes of answers, whose success probabilities are $0.55,0.65,0.75,0.85$. Reporting these probabilities is calibrated. Reporting $0.67,0.77,0.87,0.97$ instead creates an average overestimate of $0.12$, but preserves every ordering and tie between confidence values. Consequently the type 2 receiver operating characteristic and its area under the receiver operating characteristic curve remain unchanged. This is the general result that a strictly increasing confidence transformation preserves metacognitive resolution.

There is also a statistical selection issue. Let standardized observed performance be $X=A+\epsilon$ and self-estimated performance be $S=A+\eta$, where latent ability $A$ and the two errors are centred normal random variables and are independent random variables. Then selection on noisy performance can produce apparent overconfidence:For a low observed score $x<0$ and $0<\lambda <1$, the expected self-estimate exceeds $x$ despite being unbiased before selection. The highest observed scores show the reverse pattern. This toy model demonstrates why a quartile plot alone cannot identify a monitoring deficit. It does not prove that all observed miscalibration is a statistical artifact or that the original skill-training results are uninformative.

Perceptual findings also require qualification. Good resolution can reflect variation in stimulus difficulty; it need not establish optimal monitoring at fixed difficulty. Metacognitive sensitivity depends partly on first-order discrimination, and raw confidence–accuracy correlations can differ because task accuracy or confidence criteria differ. Meta-d-prime estimates the signal-detection sensitivity compatible with observed confidence ratings under its model assumptions. The original method found visual metacognitive sensitivity close to, but below, its optimal benchmark.^[ref]

A decisive comparison would measure global and trialwise estimates in the same people, match task performance where possible, estimate test reliability, distinguish raw scores from percentile beliefs, and examine the uncertainty of sensitivity estimates. Transfer of feedback or training should be tested rather than presumed. Overconfidence, useful trialwise discrimination and genuine limits in domain knowledge can coexist; their relative contributions require separable measurements.

An individual's expressed certainty in a judgment. Its relation to objective correctness is measured separately from the certainty itself. Metacognitive calibration concerns numerical agreement with outcomes; metacognitive sensitivity concerns discrimination between better and worse performance.