Past exam of the experimental design and methods course of the year 2 of the psychology (experimental) course of the University of Oxford / 2022 / 2

TODO: the missing output is needed to name the test actually used. A correlation, multiple linear regression adjusting for age, or another model would answer different questions. The presence of power, friendliness and age variables does not identify which were included, which was the response variable, how age was encoded, or whether an interaction was tested. Read the model call and coefficient or test table before identifying the procedure.

TODO: the numerical results, association direction and statistical evidence are unavailable. The required output includes the actual fitted model, estimates and uncertainty, test statistics, statistical degrees of freedom, p-values and an appropriate model-fit summary. An age-adjusted regression coefficient, if reported, would describe a conditional association under that model rather than the same estimand as an unadjusted correlation. Irrespective of its numerical outcome, this observational study does not by itself show that executive power changes real-life friendliness.

Several mechanisms could explain an observed association without a causal effect of power on friendliness. No exposure was assigned, so this is an observational study. Age, organizational role, wealth, gender, industry, public prominence or other common causes could produce confounding. Reverse direction is also possible: interpersonal behavior might influence advancement and hence power. Adjusting for age alone would not remove every alternative explanation.

The measured outcome is perceived friendliness from a photograph. Facial expression, pose, lighting, clothing, attractiveness and familiarity could affect those ratings independently of everyday behavior. If raters recognize an executive or know their status, expectations could alter the outcome; financial experts' power ratings may themselves involve judgment and measurement error. These are construct validity issues, since a photographic impression need not measure friendliness toward colleagues.

The preselected thirty executives are not a probability sample of all workers or even necessarily of all executives. Selection bias, narrow representation and restriction of range could alter associations and limit external validity. Multiple ratings of the same executive are also not independent executive observations; ignoring that structure would be pseudoreplication. These possibilities apply whether the reported association is positive, negative or inconclusive; the unavailable numerical findings are not assumed.

Improve the sampling, measurement and dependence model before interpreting the association. Define the target population and recruit a broader sample with a transparent sampling frame, ideally using probability sampling where feasible and adequate representation across the relevant ages and roles. Justify the sample size for the intended effect size and precision. Preselection on reputation or the outcomes should be avoided or explicitly modeled.

Operationalize workplace friendliness with several relevant measures, such as independently obtained colleague reports and standardized behavioral tasks, rather than relying only on photographs. Use standardized photographs if impressions are themselves the research target, several independent raters, and blinding to power and the specific hypothesis. Measure inter-rater reliability and retain the distinction between impression and behavior. Establish what the power measure actually assesses and record plausible confounders with a causal rationale.

For a linear regression analysis, use friendliness as an explicitly declared outcome and power as a continuous predictor. Retain actual age as a continuous covariate when available instead of treating three age groups as equally spaced measurements; consider justified nonlinearity. If only age categories are available, use explicit categorical contrasts and nonoverlapping category definitions. Check regression diagnostics, influential observations, heteroscedasticity and multicollinearity. Account for executive and rater dependence with an appropriate mixed-effects model when repeated or crossed ratings are analyzed; do not count each rating as a new executive. Report estimates with confidence intervals and disclose exclusions and missing observations.

Preregistration can distinguish planned analyses from exploratory ones. A longitudinal study with repeated measures can establish temporal ordering more clearly, though it remains vulnerable to confounding. A feasible, ethical randomized experiment manipulating assigned power or role in a controlled task could test a more narrowly defined causal effect; its external validity to senior executives must still be assessed. No observational adjustment alone guarantees removal of all unmeasured alternatives.