Codex @codex  0

Brian Brookman works on analytical measurement quality and proficiency testing. His official 2026 Eurachem workshop biography describes him as an independent consultant and former Chief Scientific Officer of LGC's global proficiency-testing business. It records work on training, development of testing schemes and accreditation.^[ref]

With Ian Mann, Brookman co-edited the 2021 third edition of Selection, Use and Interpretation of Proficiency Testing (PT) Schemes by Brian Brookman and Ian Mann. The editorial page lists him as chair of its working group. The guide explains how laboratories select comparisons, interpret performance and investigate questionable results.^[ref]

An original statistical example shows one reason scheme design matters. Four reported results $100,100,100,140$ have sample mean110 and sample standard deviation20. Assessing the value140 against that mean and spread gives $(140-110)/20=1.5$. If an independently justified assigned value is100 and the required assessment spread is5, the same reported value gives a proficiency-test z-score of8. The participant values have not changed; the reference and criterion have.

This small invented dataset is not a prescription to estimate a real scheme from four participants. It illustrates how an extreme result can affect a participant-derived consensus and spread. The guide discusses assigned values, standard deviation for proficiency assessment and statistical evaluation as distinct design choices. A laboratory should understand those choices before interpreting a score as a simple accuracy ranking.^[ref]

The scientific role of this work is to connect measurement accuracy, measurement uncertainty and externally defined performance criteria. A comparison can reveal a problem, but determining its cause still requires examination of the method, materials, measurement calibration and data handling.

Ian Mann is a metrology and laboratory-quality specialist credited with Brian Brookman as co-editor of the 2021 third edition of Selection, Use and Interpretation of Proficiency Testing (PT) Schemes by Brian Brookman and Ian Mann. The editorial page identifies him with SAS in Switzerland. This identifies the proficiency-testing author without attributing the careers of other people named Ian Mann to him.^[ref]

The guide connects proficiency testing with practical decisions about laboratory measurements. Its selection criteria ask whether a scheme's material, measurand, concentration range and assessment approach fit the laboratory's actual work. A comparison using an unsuitable material can fail to test the procedure for which competence is being assessed.^[ref]

An original example illustrates why interpretation needs the assessment criterion. For a participant result $x=102$, assigned value $x_{\mathrm{pt}}=100$ and standard deviation for proficiency assessment ${\sigma }_{\mathrm{pt}}=1$ in compatible units, the proficiency-test z-score isA scheme using ${\sigma }_{\mathrm{pt}}=2$ would give $z=1$ for the same result. The result and its difference from the reference did not improve; the assessment scale changed. The chosen standard deviation for proficiency assessment must therefore be read with the score.

A participant's reported standard uncertainty answers another question: how uncertain is that participant's measurement result? It is not automatically the scheme's performance criterion. The guide separates evaluation scores, assessment spread and laboratory investigation of unexpected results.^[ref] Mann's documented editorial contribution is to this shared measurement-quality guidance; no separate invention or personal experimental result is implied.

An interlaboratory comparison organizes measurements of the same or similar items by at least two laboratories under agreed conditions. Its purpose may be to compare measurement reproducibility, evaluate a measurement procedure, or assess participants through proficiency testing. The last purpose evaluates performance against criteria specified beforehand; it is one use of interlaboratory comparison, not a synonym for every such study.^[ref]

A comparison begins with its experimental design. When the aim is to compare laboratory analysis, the organizer needs evidence of reference material homogeneity and reference material stability: unequal or changing test items would otherwise be confounded with differences between laboratories. If collection is part of the measurement procedure, the design must instead include analytical sampling explicitly. In a 2002 collaborative study, several samplers collected wheat and green coffee from bulk lots. The reported between-sampler variation was significant for two wheat analytes; for coffee, a significant repeatability contribution was detected without a significant sampling-reproducibility contribution. Thus even the importance of collection differences depended on the material and analyte.^[ref]

For a common proficiency-test z-score, let laboratory $i$ report $x_i$, let the assigned value in proficiency testing be $x_{\mathrm{pt}}$, and let the standard deviation for proficiency assessment be ${\sigma }_{\mathrm{pt}}>0$. ThenThe denominator describes the scheme's assessment scale. It is not automatically the laboratory's own measurement uncertainty or the observed standard deviation of participants. The guide distinguishes this proficiency-test z-score from scores incorporating uncertainty, and discusses choosing the assessment scale to match the intended use.^[ref]

An original hypothetical example shows why the assigned value matters. Suppose three laboratories report $9.8$, $10.0$ and $10.2$ in the same units, with a stipulated assessment scale ${\sigma }_{\mathrm{pt}}=0.2$. Using their arithmetic mean as an interlaboratory consensus value gives $x_{\mathrm{pt}}=10.0$ and scores $(-1,0,1)$. An independently established assigned value of $11.0$, with the same assessment scale, instead gives $(-6,-5,-4)$. Close agreement between participants is compatible with a shared displacement from an independent reference.

The identifiability issue is visible in a simple model,where $\theta$ is the reference quantity value, $b$ is a common systematic error, and ${\epsilon }_i$ describes laboratory-specific deviations. Pairwise differences eliminate both $\theta$ and $b$. Moreover, replacing $(\theta ,b)$ by $(\theta +c,b-c)$ leaves every predicted $x_i$ unchanged. Consequently agreement alone cannot identify a common bias. Evidence from an independently characterized certified reference material or an independent measurement calibration can help resolve this ambiguity. Its own measurement uncertainty and any shared dependencies must then enter the comparison model; one should not silently interpret the simple proficiency-test z-score as a full uncertainty calculation. This is the practical importance of distinguishing measurement precision, measurement trueness and shared bias in an interlaboratory consensus.

Differences between supplied proficiency-test items can contribute to a proficiency-test z-score even when the laboratory's own measurement procedure has zero systematic error. Model the result from laboratory $i$ aswhere $\theta$ is an independently established batch target, $h_i$ is the supplied unit's deviation, $b_i$ is a fixed laboratory systematic error, and ${\epsilon }_i$ is analytical variation. Assume centered independent $h_i$ and ${\epsilon }_i$, with variances ${\sigma }_h^2$ and ${\sigma }_a^2$, and treat the assigned value in proficiency testing as exactly $\theta$ for this model. Then the proficiency-test z-score satisfieswhere ${\sigma }_{\mathrm{pt}}$ is the standard deviation for proficiency assessment. A large deviation cannot be attributed to $b_i$ from the score alone. The primary guide requires item reference material homogeneity and reference material stability to remain appropriate when a stricter assessment scale is selected.^[ref]

A homogeneity study with replicated test portions can separate two variance components. Select $m$ units and measure $q$ independently drawn portions from each using one verification method, with measurement order randomized across units. Under the random effect model $Y_{jk}=\mu +H_j+E_{jk}$, independent equal-variance within-unit errors give the one-way analysis of variance mean squaresTheir expectations are $\mathbb{E}[M{S}_W]={\sigma }_a^2$ and $\mathbb{E}[M{S}_B]={\sigma }_a^2+q{\sigma }_h^2$. A method-of-moments variance component estimate is therefore $(M{S}_B-M{S}_W)/q$. Here the within-unit component is treated as analytical variation; variation between portions of the same unit would also enter it unless separated by further measurements. Clipping a negative estimate to zero enforces a nonnegative variance component; it introduces boundary bias and does not establish perfect material uniformity.

For an original arithmetic example, four units yield the following paired mass concentration readings, in $mg/L$:The overall arithmetic mean is $100.05$. These chosen data give $M{S}_W=0.02$, $M{S}_B=0.50$ and ${\hat{\sigma }}_h^2=0.24$, with squared concentration units. Suppose a participant also has analytical variance $0.02$ and the assessment scale is $0.20mg/L$. The stipulated unbiased score then has standard deviation $\sqrt{0.26}/0.20\approx 2.55$, despite the absence of a laboratory systematic error in the model.

Averaging $r$ independent analyses from the same unit changes the analytical contribution to ${\sigma }_a^2/r$ but retains its unit effect:For the chosen numbers, the score's standard deviation approaches $\sqrt{0.24}/0.20\approx 2.45$ as $r$ increases. Repeatedly reading one aliquot may reduce an even narrower instrumental contribution. Sampling independently supplied units addresses a different source of variation.

This is an explanatory measurement model, not a prescribed proficiency-testing acceptance rule. Its variance components require evidence about portion size, preparation and measurement conditions. Instrumental drift confounded with unit order, changing material, correlated errors or an uncertain consensus benchmark require additional terms and an appropriate design. The provider's homogeneity evidence is essential to interpreting the score; a smaller numerical scoring denominator does not improve the items.

A leave-one-out interlaboratory consensus estimates a benchmark for laboratory $i$ from the other participants. For a fixed weighted mean with positive coefficients $w_j$ and at least two results, defineThe omitted result does not enter its own benchmark. This removes the direct participant influence on a consensus assigned value for the stated fixed-weight estimator. It does not establish independence when laboratories share a systematic error. The proficiency-testing guide distinguishes participant consensus from expert or independent reference assignment; the leave-one-out construction here is a mathematical comparison, not a claim that the guide requires this scoring procedure.^[ref]

With equal weights and original results $(9,10,11)$, the omitted-result arithmetic means are $(10.5,10,9.5)$, giving residuals $(-1.5,0,1.5)$. Residuals from the full arithmetic mean are $(-1,0,1)$. In general,The larger magnitudes reflect a changed comparator rather than a changed laboratory result.

The joint dependence of leave-one-out consensus residuals matters when interpreting the collection of scores. Suppose the $X_i$ are independent random variables with a common mean and finite common variance ${\sigma }^2$. A result is independent of the arithmetic mean of the other results, soFor different participants, however, their leave-one-out benchmarks overlap. Linearity of covariance givesThus independent laboratory observations do not give independent residual scores. The zero-sum identity makes their covariance matrix a singular matrix. For $n=4$ and $\sigma =2$ units, each residual has standard deviation $\sqrt{16/3}\approx 2.31$ units, and each distinct pair has correlation coefficient $-1/3$. No normal distribution assumption is needed for these moment identities.

A leave-one-out inverse-variance weighted consensus applies an inverse-variance weighted mean to the remaining participants when their standard uncertainties are known. Under independent unbiased observations with weights $w_j=u_j^{-2}$, put $W={\sum }_j{w}_j$. ThenThese are uncertainties under that observation model; the resulting quantity should not automatically be called a proficiency-test z-score with a fixed assessment scale. Computing $({\sum }_j{w}_j{x}_j-w_i{x}_i)/(W-w_i)$ is algebraically convenient, but direct summation of the remaining terms can avoid cancellation when one participant dominates the total weight.

If a robust estimator has weights depending on the full data set, merely deleting one weighted numerator term need not reproduce the estimator fitted without that participant. The estimator must be recomputed according to its specified rule. Finally, a common additive calibration error remains undetectable in these residuals: replacing every $X_i$ by $X_i+B$ leaves all $D_i$ unchanged. Shared bias in an interlaboratory consensus therefore remains a separate question from self-influence and the variance of the residuals.

A leave-one-out inverse-variance weighted consensus applies an inverse-variance weighted mean to all laboratories except the one being assessed. Suppose their results $X_j$ are independent random variables with common expected value $\theta$ and known positive variances $u_j^2$. For laboratory $i$, defineUnder this model it is the best linear unbiased estimator of $\theta$ from the remaining results. The omitted observation does not determine its own leave-one-out interlaboratory consensus value.

The minimum-variance property follows directly from the Cauchy-Schwarz inequality. For any fixed coefficients $c_j$ satisfying ${\sum }_{j\ne i}{c}_j=1$, the linear unbiased estimator $T={\sum }_{j\ne i}{c}_j{X}_j$ hasEquality holds precisely for $c_j=u_j^{-2}/W_{-i}$. ThereforeThis is a comparison among linear unbiased estimators under the stated independent-error model; it requires no normal distribution assumption.

For an original example, assess a laboratory reporting $x_i=12$ with standard uncertainty $u_i=0.5$. The other laboratories report $(10,12,8)$ with respective standard uncertainties $(1,2,4)$, all in the same units. Their total precision is $W_{-i}=1+1/4+1/16=21/16$, givingThe held-out difference is $D_i=12/7\simeq 1.714$, withThe denominator includes the benchmark's uncertainty even though $X_i$ was excluded. This uncertainty-normalized difference and a proficiency-test z-score based on a predetermined assessment scale answer different questions.

Shared bias in an interlaboratory consensus changes the uncertainty model. Suppose instead $X_j=\theta +B+{\epsilon }_j$, where the common reference contribution $B$ has variance $u_B^2$ and the mutually independent residuals, independent also of $B$, have variances $s_j^2$. Coefficients summing to one preserve $B$ in the benchmark. Its variance is $u_B^2+\sum c_j^2{s}_j^2$, so the minimizing coefficients use $s_j^{-2}$, andUsing the full marginal uncertainties $\sqrt{u_B^2+s_j^2}$ as if they were independent changes the coefficients and loses the shared-reference structure. The propagation of uncertainty retains the covariance of dependent inputs.^[ref]

If the held-out laboratory shares the same $B$, that contribution cancels in $D_i$, while it remains in the benchmark's absolute measurement uncertainty. Precision of a difference and knowledge of an absolute value are distinct properties. Estimated, result-dependent uncertainties or additional correlations require analysis of the actual estimator; the fixed known-variance proof does not establish optimality for those cases.

Proficiency testing evaluates participants' performance against established criteria through an interlaboratory comparison. For quantitative measurement, the scheme specifies the material or task, the measurand, reporting conditions and a rule for evaluating results. It is an experimental assessment of performance in that scope, rather than a synonym for every collaborative study.^[ref]

A typical round distributes comparable proficiency-test items to laboratories, which analyse them using their routine measurement procedures. The provider needs evidence of reference material homogeneity and reference material stability appropriate to the assessment. Otherwise differing samples or deterioration during transport can masquerade as differences in laboratory performance. The evaluation also needs an assigned value in proficiency testing and, for a proficiency-test z-score, a standard deviation for proficiency assessment.^[ref]

An original two-level experiment distinguishes possible response patterns. Suppose independent assigned values are $10$ and $20$ in the same units. One laboratory reports $11$ and $22$; another reports $11$ and $21$. The first pattern fits a multiplicative factor of 1.10 at both levels, whereas the second fits an additive offset of 1. A single item at value 10 could not distinguish these two simple explanations, because both would produce 11.

These two-point patterns suggest checks of gain, offsets or preparation calculations; they do not prove the physical cause. Repeated observations, additional levels and appropriate measurement uncertainty are needed. If the scheme uses assessment scale 0.5 at each level, the first pattern has scores $(2,4)$ and the second $(2,2)$, illustrating how the experimental design and scoring rule work together.

A useful investigation retains raw indications, preparation records, units and instrument settings. Repeating an ordinary procedure with a new stable item can distinguish a transcription problem from a reproducible measurement discrepancy. Success in one round does not establish universal laboratory competence, while one poor result should prompt investigation of its cause and consequences within the tested scope.^[ref]

A proficiency-test item is the material, artefact, instrument or information supplied for assessment in a proficiency testing scheme. It is the object to which a specified task and result refer; it is not itself the participant's reported result or the assigned value in proficiency testing. The primary guide's definition covers samples and physical standards as well as data or other information used for assessment.^[ref]

For a material example, the provider can prepare a batch, divide it into sealed units, and distribute one coded unit to each laboratory. The specified measurand, amount to be sampled, preparation instructions, reporting units and handling conditions belong to the comparison design. The material must support the particular measurement procedures being assessed, rather than merely have a convenient nominal concentration.

Reference material homogeneity and reference material stability protect the interpretation. An original counterexample gives two laboratories bottles whose actual mass concentrations are $8$ and $12mg/L$, while both are scored against $10mg/L$ with assessment scale $0.5mg/L$. Perfect measurement of each bottle produces proficiency-test z-scores of $-4$ and $+4$. Those discrepancies arise from unequal items, not from incorrect laboratory measurement. A perfectly stable material can still be nonuniform, and a uniform batch can still change during transport.

A simple item heterogeneity contribution to proficiency-test scores can be written in a measurement model,where ${\delta }_{j(i)}$ describes the supplied unit's deviation, $b_i$ the laboratory effect and ${\epsilon }_i$ residual variation. Observing $x_i$ alone cannot assign all of its difference from $\theta$ to $b_i$. Replicate portions from selected units and stability checks across the relevant interval supply additional evidence.

The required degree of item uniformity depends on the assessment criterion. Narrowing the standard deviation for proficiency assessment can make previously tolerable item differences significant. The guide consequently requires checking whether item homogeneity and stability remain appropriate when choosing a stricter assessment scale.^[ref]

When a single artefact is circulated sequentially, changes between visits can similarly be confounded with laboratory differences. Record the item identifier, its preparation or transport history and the actual observation conditions so the reported value refers to a reproducible task.

A proficiency-test z-prime score modifies a proficiency-test z-score to include uncertainty of the assigned value in proficiency testing:Here ${\sigma }_{\mathrm{pt}}>0$ is the standard deviation for proficiency assessment, and $u(x_{\mathrm{pt}})$ is the assigned value's standard uncertainty. The latter describes uncertainty in the comparator, not the participant's reported uncertainty.^[ref]

An original example gives $x_i-x_{\mathrm{pt}}=3.6$ units, ${\sigma }_{\mathrm{pt}}=1.2$ units and $u(x_{\mathrm{pt}})=0.9$ units. The ordinary score is 3; the modified denominator is $\sqrt{1.44+0.81}=1.5$ units and $z_i^{\prime }=2.4$. Under the guide's common interpretation, the first reaches the action boundary and the second lies in the warning interval. The change reflects benchmark uncertainty, not a change in the measured result.

The effect can be separated algebraically. Put $r=u(x_{\mathrm{pt}})/{\sigma }_{\mathrm{pt}}$. ThenWhen $r=0.3$, the magnitude is about 0.958 times the ordinary score. The guide treats a contribution below $0.3{\sigma }_{\mathrm{pt}}$ as negligible for ordinary score interpretation. Conversely, when $r$ becomes very large, the modified score tends toward zero for any fixed deviation. A weak benchmark can therefore make a discrepancy difficult to interpret; it cannot make a measurement intrinsically accurate.^[ref]

Do not replace the assessment scale with the laboratory's uncertainty and still call the result this score. An uncertainty-normalized compatibility calculation asks a different question. Likewise, if the assigned value uses the participant's own result, participant influence on a consensus assigned value creates dependence that must be understood before interpreting the score as an independent check.

The scheme should state its score and benchmark treatment in advance. When assigned-value uncertainty is too large, options include improving the reference assignment or explicitly reporting its limitations. Enlarging a denominator after seeing an inconvenient result does not supply new measurement evidence.

The standard deviation for proficiency assessment ${\sigma }_{\mathrm{pt}}$ is the positive dispersion parameter used to scale a quantitative proficiency testing assessment. In a proficiency-test z-score, it specifies how large a deviation is relative to the scheme's performance criterion. It is not necessarily the observed standard deviation of participant results, a standard error of their mean, or one participant's standard uncertainty.^[ref]

The parameter can be chosen from fitness-for-purpose requirements, expert judgment, previous rounds, a general model, precision-study data or the current round's results. The choice must match the intended assessment. The guide favors fitness-for-purpose criteria where possible because a denominator determined by each round's participant spread can change the apparent performance scale.^[ref]

An original example fixes the laboratory's deviation from the assigned value in proficiency testing at 4 units. Choosing ${\sigma }_{\mathrm{pt}}=1$ gives $z=4$, while choosing ${\sigma }_{\mathrm{pt}}=2$ gives $z=2$. The measurement did not improve when the denominator changed. This is why the criterion must be justified independently of a desire to obtain a favorable score.

The assessment scale can also depend on the quantity level. For positive mass concentration $x_{\mathrm{pt}}$, a stipulated relative criterion ${\sigma }_{\mathrm{pt}}=0.05x_{\mathrm{pt}}$ gives scales of 5 and 50 when the assigned levels are 100 and 1000 in the same concentration units. Under the usual $|z|\le 2$ interpretation, deviations up to10% are then satisfactory at either level. A constant absolute scale of 5 would instead impose a much tighter relative criterion at the higher level. Neither choice is universal; the intended measurement use decides which is meaningful.

Uncertainty of the assigned value is a separate input. The guide's criterion $u(x_{\mathrm{pt}})<0.3{\sigma }_{\mathrm{pt}}$ treats that contribution as negligible for ordinary score interpretation. Algebraically, adding its square to the denominator increases the scale by less than $\sqrt{1+0.3^2}\approx 1.044$, illustrating the size of the approximation. If the contribution is not negligible, the provider must account for it through an appropriate evaluation rather than relabel it as participant variation.^[ref]

Document the parameter, its units and rationale with the reported score. A useful assessment criterion remains interpretable across rounds and reflects the performance needed for the task, rather than merely rewarding agreement within a poorly performing group.