Multisensory perception

Perception involving information from more than one sensory modality. A unified object report, changes in accuracy and a change in neural response are different observations, each requiring a stated experimental comparison.

A systematic association between attributes in different sensory modalities, such as pitch and visual size. It may influence matching, response compatibility or perceptual binding. Such an association does not by itself establish an additional synesthetic experience or multisensory fusion.

An association potentially mediated by shared pleasantness or affect. Manipulating pleasantness independently of the sensory relation helps distinguish a hedonic mediator from a direct sensory mapping.

An association mediated by shared meaning or labels across sensory attributes. Counterbalanced nonverbal responses and cross-language comparisons help test whether a common word is necessary.

A correspondence proposed to reflect learned co-occurrence or covariation of sensory attributes. A discriminating test changes the experienced relation while controlling exposure and tests transfer; a familiar pairing alone does not establish how it was acquired.

A proposed correspondence explained by organization shared across sensory dimensions or their neural codes. To distinguish this account from linguistic or learned environmental explanations, vary labels, task context and experience rather than identifying structure from agreement alone.

Interactions or combination of information across sensory modalities. Operational criteria must be specified: a neuron can show a changed paired response, a judgment can combine cues, or signals can be attributed to one event. These criteria constrain one another without being interchangeable.

A task-defined range of intersensory delays compatible with grouping or reported simultaneity. Its estimate depends on stimulus content, sensory uncertainty and decision criteria. It is not automatically the duration of one anatomical neural integrator.

An experience-dependent change in the intersensory timing relation judged simultaneous or used in binding. Exposure to a repeated audiovisual lag can shift later timing judgments. Transfer to a different task and recovery tests help distinguish calibration from a task-specific response strategy.

A combined response exceeding a specified unisensory benchmark. For baseline-corrected mean spike counts, $R_{AV}>\max (R_A,R_V)$ defines enhancement over the strongest component. This differs from exceeding the sum, and depends on the response measure, analysis window and statistical comparison.

The tendency for relative multisensory enhancement to be larger when component stimuli evoke weak responses. Absolute gain and percentage gain must be distinguished: $100(R_{AV}-R_{\max })/R_{\max }$ can increase merely because the denominator becomes small. A ratio is undefined at a zero denominator and does not directly measure information or behavioral benefit.

A combined response exceeding the sum of its separately measured component responses under an explicit baseline convention. Consistent subtraction of spontaneous activity prevents counting background twice. Enhancement above the strongest component need not be superadditive.

Inference about whether sensory signals have a common source or separate sources. Bayes' theorem compares hypotheses through $P(C\mid x)\propto P(x\mid C)P(C)$. A common-source hypothesis can permit cue combination; separate-source hypotheses can preserve distinct estimates. Reliability weighting conditional on a common source is therefore not the entire causal-inference problem.

For a common-event model, let $y\mid \delta \sim N(\delta ,s^2)$ and $\delta \sim N(0,{\tau }^2)$ with positive variances. The posterior exponent is $-[(y-\delta {)}^2/s^2+{\delta }^2/{\tau }^2]/2$. Completing the square yields posterior precision $s^{-2}+{\tau }^{-2}$ and mean $y{\tau }^2/(s^2+{\tau }^2)$. The estimate is pulled toward zero, more strongly when the measurement is uncertain. This conditional model does not decide whether a common event should be assumed.

Combining estimates or evidence from several sensory channels. Tests measure both the combined judgment and the uncertainty of the individual cues. A common-cause assumption should be tested rather than inferred merely from concurrent presentation.

A cue-combination rule weighting more precise estimates more strongly. For independent unbiased Gaussian estimates of one quantity and a flat prior, inverse-variance weights are both minimum-variance linear and maximum-likelihood weights. The assumptions exclude arbitrary averaging of complementary qualities such as odor identity and gustatory sweetness.

Let unbiased independent estimates $X_1,X_2$ have variances $a,b>0$. The variance of $w{X}_1+(1-w)X_2$ is $w^{2}a+(1-w{)}^{2}b$. Its derivative is $2w(a+b)-2b$ and its second derivative is $2(a+b)>0$, proving the unique optimum $w=b/(a+b)$ and variance $ab/(a+b)$. Independent Gaussian likelihoods give the same estimate by differentiating $(x_1-x{)}^2/a+(x_2-x{)}^2/b$. With covariance $c$, the variance becomes $w^{2}a+(1-w{)}^{2}b+2w(1-w)c$, and the unconstrained optimum is $(b-c)/(a+b-2c)$ when the denominator is positive; it need not lie between zero and one.