Past exam of the digital social research: statistics core course of the University of Oxford / 2019 / 1

Here the coding identifies group 0 as male and group 1 as female in this dataset. The counts are 1215 and 1251, respectively, or approximately 49.27% and 50.73% of 2466 observations. These are sample compositions, not evidence that the sample represents every population of interest. The continuous efficacy scale runs from 0 to 16, with higher scores indicating stronger efficacy. Its printed sample median is 10 and its quartiles are 8 and 12, so its interquartile range is 4 points. The printed overall sample mean, 9.348, requires the separate consistency qualification below.

Let ${\mu }_0$ and ${\mu }_1$ denote the two population means. The null hypothesis is $H_0:{\mu }_0-{\mu }_1=0$, and the alternative hypothesis is the two-sided test $H_1:{\mu }_0-{\mu }_1\ne 0$. The two sample means giveThe positive sign means higher mean efficacy in group 0, given this coding; it does not say that every member of that group has higher efficacy than every member of group 1.

Welch's t-test estimates uncertainty from the two groups separately rather than assuming equal population variances:The printed $t=8.1616$ implies a standard error of approximately $1.033838/8.1616=0.126671$ points. The noninteger 2277.4 statistical degrees of freedom comes from the Welch approximation; it is not a count of fractional observations. Using the corresponding Student t-distribution gives the displayed p-value, approximately $5.408\times 10^{-16}$. The test and interval calculations agree to the precision of the printed numbers.^[ref]

A p-value is the probability, under the null hypothesis and the assumed sampling model, of a test result at least as incompatible with equal means as the observed result. It is not the probability that the null hypothesis is true, the probability that the finding is a chance mistake, or the size of the mean difference. At a prespecified significance level such as $\alpha =0.05$, this result rejects the equal-mean null hypothesis; it also rejects at 1%. A Type I error is rejection when that hypothesis is true, and a valid level-$\alpha$ procedure controls its long-run probability. A Type II error is failure to reject a false hypothesis; statistical power is the probability of rejection under a specified alternative. Neither error probability for this one realized claim is supplied by its p-value.

The 95% confidence interval comes from $\hat{\Delta }\pm t_{0.975,2277.4}\mathrm{SE}(\hat{\Delta })$, with critical value about 1.961. Its positive endpoints support the same direction and exclude zero. The frequentist 95% statement concerns coverage over repeated samples, not a 95% posterior probability assigned to this fixed interval. Reporting the difference in scale points supplies an effect size; its practical importance also depends on what a point means substantively. A very small p-value alone does not answer that question.

The inferential calculation requires independent observations and an appropriate sampling model. With these large groups, approximately normal sampling distributions of the means can be more plausible than normality of individual bounded scores, but a large sample size does not correct dependence, clustering, selection bias or measurement problems. The output does not establish random assignment or remove possible confounding, so the estimated difference is an association.

There is a material source inconsistency. For exhaustive groups with complete outcome data, the sample mean must be the sample-size-weighted mean of the group means:which cannot round to 9.348. Figure 7 independently reports all 2466 efficacy values present and again gives 9.348; Question 6 refers back to the same continuous scale. The pooled-mean compatibility under rounding rules out explaining the discrepancy by the displayed decimal precision. Thus the two-group test supports the stated conclusion if its own group summaries and test fields are trusted, while the overall mean cannot simultaneously be accepted as a coherent summary of those groups. No unprinted exclusion or uniquely correct replacement mean is established.