Participant influence on a consensus assigned value

Participant influence on a consensus assigned value is the dependence created when a result being assessed also contributes to its benchmark. The deviation from that benchmark is then generally smaller than the deviation from a corresponding consensus formed solely from the other participants. This is a property of the estimator, not evidence that the laboratory improved.

For an original equal-weight calculation with $n>1$ results, writewhere ${\bar{x}}_{-i}$ is the leave-one-out interlaboratory consensus computed as the arithmetic mean of the other results. With results $(8,10,10,12)$, the overall mean is 10. The participant reporting 12 differs from it by 2, but differs from the other three laboratories' mean $28/3$ by $8/3$. The factor $3/4$ accounts for the difference exactly.

A fixed-weight version makes the effect more general. If normalized weights $a_j$ sum to one, $0\le a_i<1$, andthen $x_i-x_c=(1-a_i)(x_i-x_{c,-i})$. A participant with weight 0.8 retains only one fifth of its difference from the remaining weighted consensus. These are fixed-weight identities; robust statistics with data-dependent weights requires analysis of the actual estimator, for example through an influence function.

Dependence also enters uncertainty. For independent results $X_i$ having common mean $\theta$ and variance ${\sigma }^2$,Treating $X_i$ and $\bar{X}$ as independent would instead add their variances and give the wrong result. This is a stated sampling model, not a universal uncertainty formula for every interlaboratory comparison.

The primary guide distinguishes assigned values based on participant consensus, expert laboratories, formulation and certified reference materials. It also requires the assigned value's uncertainty to be evaluated and reported.^[ref] Using an independent reference or a documented leave-one-out method changes the dependence, but neither is automatically appropriate for every scheme.

Finally, reducing self-influence does not resolve shared bias in an interlaboratory consensus. All remaining laboratories can still share the same calibration error. Benchmark independence, robustness to an extreme result and absolute accuracy are separate questions.

A leave-one-out interlaboratory consensus estimates a benchmark for laboratory $i$ from the other participants. For a fixed weighted mean with positive coefficients $w_j$ and at least two results, defineThe omitted result does not enter its own benchmark. This removes the direct participant influence on a consensus assigned value for the stated fixed-weight estimator. It does not establish independence when laboratories share a systematic error. The proficiency-testing guide distinguishes participant consensus from expert or independent reference assignment; the leave-one-out construction here is a mathematical comparison, not a claim that the guide requires this scoring procedure.^[ref]

With equal weights and original results $(9,10,11)$, the omitted-result arithmetic means are $(10.5,10,9.5)$, giving residuals $(-1.5,0,1.5)$. Residuals from the full arithmetic mean are $(-1,0,1)$. In general,The larger magnitudes reflect a changed comparator rather than a changed laboratory result.

The joint dependence of leave-one-out consensus residuals matters when interpreting the collection of scores. Suppose the $X_i$ are independent random variables with a common mean and finite common variance ${\sigma }^2$. A result is independent of the arithmetic mean of the other results, soFor different participants, however, their leave-one-out benchmarks overlap. Linearity of covariance givesThus independent laboratory observations do not give independent residual scores. The zero-sum identity makes their covariance matrix a singular matrix. For $n=4$ and $\sigma =2$ units, each residual has standard deviation $\sqrt{16/3}\approx 2.31$ units, and each distinct pair has correlation coefficient $-1/3$. No normal distribution assumption is needed for these moment identities.

A leave-one-out inverse-variance weighted consensus applies an inverse-variance weighted mean to the remaining participants when their standard uncertainties are known. Under independent unbiased observations with weights $w_j=u_j^{-2}$, put $W={\sum }_j{w}_j$. ThenThese are uncertainties under that observation model; the resulting quantity should not automatically be called a proficiency-test z-score with a fixed assessment scale. Computing $({\sum }_j{w}_j{x}_j-w_i{x}_i)/(W-w_i)$ is algebraically convenient, but direct summation of the remaining terms can avoid cancellation when one participant dominates the total weight.

If a robust estimator has weights depending on the full data set, merely deleting one weighted numerator term need not reproduce the estimator fitted without that participant. The estimator must be recomputed according to its specified rule. Finally, a common additive calibration error remains undetectable in these residuals: replacing every $X_i$ by $X_i+B$ leaves all $D_i$ unchanged. Shared bias in an interlaboratory consensus therefore remains a separate question from self-influence and the variance of the residuals.

A leave-one-out inverse-variance weighted consensus applies an inverse-variance weighted mean to all laboratories except the one being assessed. Suppose their results $X_j$ are independent random variables with common expected value $\theta$ and known positive variances $u_j^2$. For laboratory $i$, defineUnder this model it is the best linear unbiased estimator of $\theta$ from the remaining results. The omitted observation does not determine its own leave-one-out interlaboratory consensus value.

The minimum-variance property follows directly from the Cauchy-Schwarz inequality. For any fixed coefficients $c_j$ satisfying ${\sum }_{j\ne i}{c}_j=1$, the linear unbiased estimator $T={\sum }_{j\ne i}{c}_j{X}_j$ hasEquality holds precisely for $c_j=u_j^{-2}/W_{-i}$. ThereforeThis is a comparison among linear unbiased estimators under the stated independent-error model; it requires no normal distribution assumption.

For an original example, assess a laboratory reporting $x_i=12$ with standard uncertainty $u_i=0.5$. The other laboratories report $(10,12,8)$ with respective standard uncertainties $(1,2,4)$, all in the same units. Their total precision is $W_{-i}=1+1/4+1/16=21/16$, givingThe held-out difference is $D_i=12/7\simeq 1.714$, withThe denominator includes the benchmark's uncertainty even though $X_i$ was excluded. This uncertainty-normalized difference and a proficiency-test z-score based on a predetermined assessment scale answer different questions.

Shared bias in an interlaboratory consensus changes the uncertainty model. Suppose instead $X_j=\theta +B+{\epsilon }_j$, where the common reference contribution $B$ has variance $u_B^2$ and the mutually independent residuals, independent also of $B$, have variances $s_j^2$. Coefficients summing to one preserve $B$ in the benchmark. Its variance is $u_B^2+\sum c_j^2{s}_j^2$, so the minimizing coefficients use $s_j^{-2}$, andUsing the full marginal uncertainties $\sqrt{u_B^2+s_j^2}$ as if they were independent changes the coefficients and loses the shared-reference structure. The propagation of uncertainty retains the covariance of dependent inputs.^[ref]

If the held-out laboratory shares the same $B$, that contribution cancels in $D_i$, while it remains in the benchmark's absolute measurement uncertainty. Precision of a difference and knowledge of an absolute value are distinct properties. Estimated, result-dependent uncertainties or additional correlations require analysis of the actual estimator; the fixed known-variance proof does not establish optimality for those cases.

The joint dependence of leave-one-out consensus residuals arises because different laboratories' benchmarks reuse many of the same observations. Excluding each result from its own leave-one-out interlaboratory consensus does not make the complete collection of residuals independent.

Let $a_i$ be fixed positive coefficients with ${\sum }_i{a}_i=1$, and let ${\bar{X}}_a={\sum }_i{a}_i{X}_i$ be the full weighted mean. With at least two observations, every $a_i<1$, and the omitted-result residual isWrite $P_a=I-1a^T$ for the weighted centering matrix, and $M=\mathrm{diag}(1-a_1,\dots ,1-a_n)$ for a diagonal matrix. ThenEach row of $A$ sums to zero, so each residual is a contrast: a common additive level cancels.

For independent observations with positive variances $v_i$, define $S={\sum }_k{a}_k^2{v}_k$. Expanding linearity of covariance givesEqual coefficients and equal variances reduce these expressions toThe familiar negative covariance belongs to this model. Arbitrary fixed coefficients can produce a different sign.

An original three-laboratory counterexample sets $v_1=v_2=v_3=1$ and $a=(0.05,0.05,0.90)$. Then $S=0.815$, andThis positive covariance has a simple cause: both benchmarks are dominated by the third result. Increasing $X_3$ pushes both $D_1$ and $D_2$ downward. The coefficients are stipulated to expose the dependence; with equal observation variances, they are not inverse-variance coefficients.

For positive inverse-variance weights $a_i=v_i^{-1}/W$, where $W={\sum }_k{v}_k^{-1}$, one has $a_i{v}_i=1/W$ and $S=1/W$. The off-diagonal expression becomesThus the leave-one-out inverse-variance weighted consensus has a special sign pattern under its independent known-variance assumptions.

The coefficients also impose a deterministic constraint,Consequently the residual covariance matrix is a singular matrix. Under the independent-observation model with every $v_i>0$, its matrix rank is $n-1$: the weighted centering matrix removes the common-level direction and the invertible diagonal matrix $M$ preserves that rank. For equal coefficients the constraint is simply ${\sum }_i{D}_i=0$.

These dependencies matter when evaluating several laboratories together. Products of single-residual probabilities generally do not describe joint events. A specified joint model or a set of appropriately chosen contrasts can represent the available information without treating redundant residuals as independent observations.