Standard deviation for proficiency assessment (source code)

= Standard deviation for proficiency assessment
{title2=$\sigma_{\mathrm{pt}}$}

The \b[standard deviation for proficiency assessment] $\sigma_{\mathrm{pt}}$ is the positive dispersion parameter used to scale a quantitative <proficiency testing> assessment. In a <proficiency-test z-score>, it specifies how large a deviation is relative to the scheme's performance criterion. It is not necessarily the observed standard deviation of participant results, a standard error of their mean, or one participant's <standard uncertainty>.<Selection, Use and Interpretation of Proficiency Testing (PT) Schemes by Brian Brookman and Ian Mann>[§7.2.3, printed p. 16; §8.2.3, printed p. 21]{ref}

The parameter can be chosen from fitness-for-purpose requirements, expert judgment, previous rounds, a general model, precision-study data or the current round's results. The choice must match the intended assessment. The guide favors fitness-for-purpose criteria where possible because a denominator determined by each round's participant spread can change the apparent performance scale.<Selection, Use and Interpretation of Proficiency Testing (PT) Schemes by Brian Brookman and Ian Mann>[§7.2.3 and Annex D.3, sources of assessment standard deviation]{ref}

An original example fixes the laboratory's deviation from the <assigned value in proficiency testing> at 4 units. Choosing $\sigma_{\mathrm{pt}}=1$ gives $z=4$, while choosing $\sigma_{\mathrm{pt}}=2$ gives $z=2$. The measurement did not improve when the denominator changed. This is why the criterion must be justified independently of a desire to obtain a favorable score.

The assessment scale can also depend on the quantity level. For positive <mass concentration> $x_{\mathrm{pt}}$, a stipulated relative criterion $\sigma_{\mathrm{pt}}=0.05x_{\mathrm{pt}}$ gives scales of 5 and 50 when the assigned levels are 100 and 1000 in the same concentration units. Under the usual $|z|\le2$ interpretation, deviations up to10% are then satisfactory at either level. A constant absolute scale of 5 would instead impose a much tighter relative criterion at the higher level. Neither choice is universal; the intended measurement use decides which is meaningful.

Uncertainty of the assigned value is a separate input. The guide's criterion $u(x_{\mathrm{pt}})<0.3\sigma_{\mathrm{pt}}$ treats that contribution as negligible for ordinary score interpretation. Algebraically, adding its square to the denominator increases the scale by less than $\sqrt{1+0.3^2}\approx1.044$, illustrating the size of the approximation. If the contribution is not negligible, the provider must account for it through an appropriate evaluation rather than relabel it as participant variation.<Selection, Use and Interpretation of Proficiency Testing (PT) Schemes by Brian Brookman and Ian Mann>[§7.2.6, printed p. 18, assigned-value uncertainty]{ref}

Document the parameter, its units and rationale with the reported score. A useful assessment criterion remains interpretable across rounds and reflects the performance needed for the task, rather than merely rewarding agreement within a poorly performing group.