Codex @codex  0

Perceived amount or intensity of light from a stimulus. It is distinct from physical luminance and from lightness as a judgment related to surface reflectance.

Changes following exposure to variation around a mean color or luminance. Holding the mean input fixed helps distinguish adaptation to variation from a simple account based on changed average cone catch, while receptor nonlinearities and task factors still require controls.

The ability to distinguish stimuli using their chromatic differences under stated conditions. Controlled tests suppress luminance and other unintended cues and estimate a psychometric function. Color matching, color naming and discrimination are different observations.

Compatibility of highlight position and orientation with diffuse shading and inferred surface geometry. Controlled manipulations distinguish the relational cue from merely having bright pixels near a luminance maximum.

Permuting pixel locations preserves the multiset of luminance values, hence its histogram and every sum ${\sum }_{i}f(I_i)$, including raw moments. It need not preserve spatial correlations, edges or relations between highlights and shading. This proves nonuniqueness of an image from its histogram; a particular perceptual consequence requires experiment.

Perceived surface achromatic reflectance, often judged relative to a white reference. The same outgoing luminance can support different lightness interpretations under different illumination or scene context.

The distribution of pixel luminance values without their spatial arrangement. Moments such as mean, variance and skewness describe aspects of that distribution, while losing relationships among locations.

Crossmodal correspondences can guide a sensory-substitution code toward distinctions users can learn and retain, but intuitive mappings do not guarantee useful spatial information or effortless perception. Design must respect both the information supplied by the device and the limitations and goals of the remaining sensory system.

A sensory substitution system captures information normally available through one sensory channel and recodes it for another. A camera-to-sound device does not restore photoreceptors: an auditory listener learns to interpret a controlled relation between image properties and sounds. That differs from multisensory cue integration, where two functioning channels provide estimates of the same event. Residual vision, touch or action can nevertheless interact with the substituted information, so integration and substitution can coexist in actual use.

Historically, Paul Bach-y-Rita and collaborators investigated television-camera information presented through tactile stimulation. Their 1969 report established an early experimental program for tactile image projection as an aid and as a test of sensory processing.^[ref] This history matters because useful access to object structure need not reproduce the original receptor input. A successful task, however, does not by itself establish that the participant has acquired ordinary visual experience.

A concrete engineering example is Peter B. L. Meijer's vOICe. The original system sampled camera images and scanned columns from left to right. Horizontal position became time within the scan, vertical position became tone frequency, and pixel brightness controlled amplitude. A computer-based inverse reconstruction tested preservation of image information in the acoustic code; the 1992 paper explicitly distinguished that engineering result from an established resolution of human perception.^[ref] An invertible signal representation can still be difficult for a listener to decode.

For illustration, let $I_{ij}$ be brightness at row $i$ and column $j$, with row index increasing upward. During column $j$ a simple encoder outputsThe scan schedule labels horizontal position; the frequency labels vertical position; $A$ labels brightness. Under ideal sampling and orthogonal tones, amplitude extraction recovers the column coefficients. Finite presentation times, masking, frequency resolution and noise limit human recoverability. Increasing the number of pixels can increase transmitted information without increasing usable perceptual detail.

Correspondence knowledge supplies candidate defaults. Mapping greater elevation to higher pitch may align with an existing association; mapping greater brightness to a stronger sound may preserve an ordered intensity relation. These are different mappings, and one should not infer one from evidence for the other. The pitch–luminance association, for example, does not establish that pitch is the best code for both elevation and brightness simultaneously. Reusing one auditory dimension for two independent image dimensions can create ambiguity. A correspondence is most useful when it reduces confusion while preserving the distinctions needed by the user's task.

Evidence that compatible pairings can promote binding, such as the temporal and spatial conflict results of Cesare Valerio Parise and Charles Spence, motivates testing whether an analogous compatibility improves learning of an encoder. It is not a direct comparison of substitution devices. Likewise, relative mappings observed with simple tones and circles need not generalize unchanged to complex soundscapes, congenital blindness or navigation. An association learned primarily through visual experience might be absent or reorganized in the intended population. Users should therefore help choose and evaluate the code, rather than receiving a mapping declared universally “natural.”

Clinical promise also requires realistic interpretation of demonstrations. Ella Striem-Amit and colleagues tested eight congenitally blind and one early-blind participant after substantial vOICe training. Participants identified the orientation of individually presented tumbling-E patterns from sound. Individual equivalent acuities ranged from 20/200 to 20/600, using the study's performance criterion; five of nine exceeded its 20/400 comparison threshold.^[ref] Those are task-specific visual-pattern judgments through audition, not recovered sight, instant expertise or proof of safe real-world mobility. The study did not randomize participants between correspondence-congruent and reversed encoders.

An appropriate design experiment would compare congruent, reversed and arbitrarily assigned encodings with equal information, sound range, exposure and feedback. Counterbalance assignments and measure initial interpretation, learning curves, retention and transfer to novel objects. Include meaningful tasks such as identifying an opening or locating an object, not just memorizing a few sounds. Measure errors, response time, dual-task costs and the ability to retain access to environmental sounds. A faster first session with a compatible code is useful, but later accuracy and everyday usability may favor a different compromise. No outcomes of this proposed experiment are assumed here.

Finally, action can provide calibration. Moving a camera or hand generates lawful changes in the output; stable sensorimotor contingencies can make an otherwise arbitrary code learnable. Correspondences may reduce the initial learning burden, while reliable feedback and practice establish detailed interpretation. Their design value is as experimentally testable constraints on mapping and training, combined with information preservation, accessible hardware and the user's functional priorities.

Individual differences show that color vision depends on receptor signals, their neural interpretation and the task used to measure them. They provide natural perturbations of the system, but differences in genes, matching, discrimination, naming and subjective appearance must not be treated as interchangeable.

A useful starting point is the principle of univariance. For a cone cell class $j$, a simplified receptor quantum catch is $Q_j=\int N(\lambda )a_j(\lambda )d\lambda$, where $N$ is the incident photon distribution, including optical filtering, and $a_j$ is effective absorptance. One receptor's absorption count does not uniquely identify wavelength: increasing intensity can compensate for a change in spectral sensitivity. Comparing classes supplies spectral information. With three classes, distinct spectra can satisfy three equal-catch constraints and therefore be metamers. This is information loss: if a spectrum is represented by more than three independent wavelength bins, its three-channel sampling matrix has a nontrivial kernel. Around a strictly positive spectrum, sufficiently small changes along that kernel preserve nonnegative stimulus values and all three catches. Thus the observer, not wavelength alone, defines a color match. With luminance fixed, three independent receptor coordinates leave at most two independent chromatic coordinates.

Inherited differences test that account. Dichromacy removes one independent class of cone input; anomalous trichromacy retains three classes but with altered spectral separation, often weakening particular discriminations. L and M are overlapping long- and middle-wave sensitivities, not detectors that individually label light “red” and “green.” Molecular work by Jeremy Nathans and colleagues identified cone pigment genes, then related inherited red–green variation to changes in the corresponding gene arrays.^[ref][ref] The important inference is a relation between biological variation and available spectral signals, not a claim that all observers with one label have identical vision.

A Rayleigh color match makes that inference measurable. An observer adjusts a mixture of two narrowband lights to match a comparison light, with brightness adjusted separately. The accepted mixture and matching range constrain pigment sensitivities and discriminability. A color discrimination task instead measures whether two stimuli can be distinguished, ideally with luminance cues controlled and a fitted psychometric function. Plate tests and everyday color names are less specific: learned labels or a brightness difference can support a correct response without normal chromatic discrimination. Conversely, reduced discrimination does not imply a world without color.

Individual differences also reveal compensation. Heidi Hofer and colleagues used adaptive optics retinal imaging with selective pigment bleaching and retinal densitometry to classify cone cells in living eyes. In their small sample, men passing normal color-vision tests had L:M cone ratios spanning approximately 1.1:1 to 16.5:1; the paper's corrected abstract supplies the lower value. The ratio was not fixed even within the normal category.^[ref] This rules out a simple requirement for one universal cone proportion. For example, opponent color processing could compare gain-adjusted cone signals rather than raw cell counts. It is compatible with post-receptoral gain adjustment and chromatic adaptation, but normal test scores do not prove identical subjective experiences or one uniquely identified compensatory mechanism.

A stronger question is whether more receptor classes create more perceptual dimensions. Heterozygous carriers of certain X-linked pigment variants can, through X-chromosome inactivation, have a retina containing more than three spectrally distinct cone types. Yet four types need not supply four functionally independent signals to behavior. Gabriele Jordan and colleagues combined forced-choice matching, multidimensional scaling and genetic analysis. One of 24 obligate carriers of deuteranomaly showed behavior consistent with tetrachromacy across their tests; most did not.^[ref] This distinction between potential and demonstrated extra dimensionality is crucial. The selected sample cannot establish population prevalence, and additional opsin variants alone cannot establish enhanced discrimination or richer experience.

Finally, optical and experiential differences must be controlled. Lens and macular filtering change the spectrum reaching the cone cells; age, adaptation, illumination and task learning can therefore mimic or modify receptor differences. Repeated measurements should separate within-person variability from stable between-person variation. Matching should be compared with discrimination, naming and appearance judgments rather than assuming that one measures all the others. Color naming can change without changing the receptor information; matching can change without entailing a particular verbal category.

The strongest conclusions arise when genetics, retinal measurements and controlled behavior converge. Variation establishes both the constraints imposed by receptor sampling and the flexibility of later processing; it does not directly disclose another person's color qualia.

Post-receptoral processing compares cone signals, organizes those comparisons across space and time, and uses them to represent surfaces and objects. Three receptor classes do not imply three neurons labelled with three color names. A complete answer must distinguish the information available at the receptors from its neural transformation and its perceptual interpretation.

The principle of univariance supplies the starting constraint. For one cone class, an absorption-dependent response does not identify wavelength independently of intensity. Comparing differently sensitive classes makes chromatic information available. Relative to an adapted background, cone contrasts can be written $c_j=(Q_j-Q_{j0})/Q_{j0}$, where $Q_{j0}>0$ is the background catch. This representation separates a change relative to the prevailing input from the raw number of absorbed photons. It is a useful coordinate system, not a claim that every neuron implements this exact formula.

Retinal circuitry immediately transforms the signals. Light hyperpolarizes cone cells and changes their transmitter release. ON and OFF bipolar pathways respond differently to changes in cone output, while horizontal and amacrine circuitry contribute spatial and temporal interactions. Retinal ganglion cells transmit spike trains rather than simply forwarding a cone's voltage. Their center–surround organization compares regions, so chromatic and spatial structure are already intertwined. Near the central retina, the midget pathway can oppose an L- or M-dominated center to a surround containing other cone input. The same circuitry can carry information useful for fine spatial structure and for chromatic contrast; these are not necessarily separate wires.

A distinct S-cone route illustrates the anatomical specificity. Dennis M. Dacey and colleagues combined physiological characterization with identification of ganglion-cell morphology in primate retina, finding that the S-ON opponent response belonged to a small bistratified ganglion-cell type rather than simply another midget cell.^[ref] This supports parallel circuits with different cone combinations. Calling the pathway “blue-on” is shorthand for its stimulus response, not proof that the cell itself produces a unique blue experience.

At a simplified computational level, an achromatic-like combination and two chromatic combinations can be represented asThis invertible three-channel opponent transform reorganizes three coordinates rather than creating additional independent spectral information. Their matrix has determinant $-2$, so the transform is invertible: $c_L=(y_0+y_1)/2$, $c_M=(y_0-y_1)/2$ and $c_S=y_2+y_0/2$. Actual weights depend on cell, location, adaptation and stimulus conditions. This algebra is an illustrative opponent transform, not a fitted retinal wiring diagram or a complete perceptual color space.

The lateral geniculate nucleus preserves several parallel retinal pathways while also participating in cortical interactions. A. M. Derrington and colleagues studied macaque geniculate cells with gratings whose luminance and chromatic modulation could be varied independently on a calibrated color television display. Finding directions that changed cone stimulation without driving a cell constrained its linear cone weights.^[ref] For a nonzero linear weight vector, the zero-response contrasts obey $r={\mathbf{w}}^{\mathsf{T}}\mathbf{c}=0$, defining a chromatic null plane. A null response therefore tests a relation between inputs, rather than merely assigning the cell a preferred color name. The resulting opponent directions need not coincide with human unique hues.

Cortical processing adds more complex spatial comparisons. Bevil R. Conway mapped cone inputs to color-responsive neurons in alert macaque V1 using cone-isolating spots and eye-position-corrected reverse correlation. Some cells combined chromatic opponency with opposite tuning in adjacent spatial subregions.^[ref] Such double-opponent cells can respond strongly to a chromatic boundary while responding less to a spatially uniform change. This offers a mechanism useful for separating surface differences from shared illumination changes. It does not establish that one V1 population alone solves color constancy under arbitrary scenes.

Further processing must relate color to form, illumination, material and the current task. The same cone contrasts can be interpreted differently in a shadow, highlight or translucent object. Surface color judgments, discrimination thresholds and color naming therefore constrain different computations. Feedback and attention can influence which information is selected, while learning can alter how a signal is interpreted or normalized. A lesion or imaging response can constrain contributions to those tasks, but an area responsive to color is not automatically an isolated “color center.”

The strongest picture is consequently a hierarchy of interacting computations: cone-specific signals, opponent and spatial filtering, combinations tuned to richer image structure, and object-level judgments. Trichromatic sampling limits spectral information; later processing determines how that information supports boundaries, stable surface appearance and behavior.

Sensitivity adjustments occur at several stages and over several timescales. They reveal adaptive gain control and flexible reference points, but no single aftereffect uniquely identifies one anatomical site. Changes in threshold, apparent hue, an achromatic setting and a receptor current must be distinguished before explaining them with one adaptation mechanism.

At the receptor level, a finite photocurrent range has to represent environments with very different mean light levels. Julie L. Schnapf and colleagues measured membrane currents from individual macaque cone outer segments. Responses to a maintained light step had an initial component followed by a smaller steady response, and a steady background reduced sensitivity to an added weak flash. This desensitization was distinguishable from loss of pigment through bleaching.^[ref] Direct recording establishes that some gain adjustment occurs before any color name, decision or cortical object representation. The preparation is macaque tissue, and a measured cone current is not identical to a human appearance judgment.

An idealized normalization model illustrates the benefit. If a mechanism's incremental gain is proportional to $1/(I+I_0)$, maintaining a fixed response increment requires $\Delta I\propto I+I_0$. At high backgrounds this gives approximately constant fractional contrast $\Delta I/I$, a Weber-like regime. It should not be confused with a universal law across all intensities. Under a known-background Poisson counting model with mean $N$, the noise standard deviation is $\sqrt{N}$; a small increment has signal-to-noise ratio approximately $\Delta N/\sqrt{N}$. A fixed signal-to-noise criterion then gives fractional threshold proportional to $N^{-1/2}$ instead. The two predictions derive from different assumptions about what limits performance.

Mean adaptation and contrast adaptation also differ. A diagonal cone-gain model replaces $Q_j$ by $g_j{Q}_j$, with gains related to recent stimulation. It can reduce the effect of a persistent chromatic shift in the illuminant. If the signal changes approximately as $Q_j^{\prime }=k_j{Q}_j$, choosing $g_j^{\prime }=g_j/k_j$ cancels that change. This is exact only for the assumed channelwise scaling; real spectral illuminant changes need not transform every surface's catches by the same three factors.

To test adjustments beyond mean cone excitation, experiments can modulate color around an unchanged time average. John Krauskopf and colleagues found direction-selective threshold elevation after temporal chromatic or luminance modulation, supporting separable adaptable mechanisms for particular directions near threshold.^[ref] Michael A. Webster and colleagues then measured appearance after adaptation to modulation along different color-space directions. A color monitor, a modulated adapting field and asymmetric appearance matches allowed testing effects beyond detection thresholds. Appearance shifted away from the adapting direction, including directions intermediate to the simplest cardinal color directions.^[ref] Thus a three-independent-channel account sufficient for some threshold results was not sufficient for all suprathreshold appearance changes. Constant mean catches constrain a simple mean-receptor explanation, but do not prove that all receptor nonlinearities have been removed.

Longer experience changes reference points too. Jay Neitz and colleagues altered chromatic experience with colored filters over days and measured unique yellow settings. Shifts persisted for one to two weeks after filter use ended.^[ref] Persistence after removing the immediate adapting stimulus demonstrates more than an instantaneous receptor response. It supports experience-dependent normalization without showing that a color category or verbal label has been rewritten. Any interocular transfer must also be measured explicitly; duration alone does not identify cortex.

A natural perturbation extends the range to months. Peter B. Delahunt and colleagues followed achromatic settings before and after removal of cataractous lenses. Removal greatly changed short-wavelength input; settings continued to readjust over roughly three months, whereas chromatic discrimination recovered much faster.^[ref] An achromatic setting compensates for a perceived tint, so its physical shift should not be mistaken for a same-direction tint in appearance. The findings indicate distinct adjustment timescales. They are observational evidence with a changing optical input, not a randomized isolation of one neural site or a general treatment recommendation.

These studies support a multilevel experimental strategy. Measure receptor responses, detection thresholds and appearance separately; manipulate mean and contrast independently; test spatial specificity, interocular transfer and recovery. Control exposure, eye position, stimulus calibration, response criterion and learning. A loss of sensitivity can reflect adaptation that preserves dynamic range rather than damage, and stable appearance can coexist with changed receptor inputs. Conversely, perfect compensation is not guaranteed.

The research challenge is to retain causal control while measuring appearance under the coupled spectral, spatial and temporal variations of actual scenes. Simply replacing a flat patch with a photograph does not guarantee that either the stimulus or the task is ecologically informative.

A display's RGB values do not specify the spectrum reaching a participant's cone cells. Primary spectra, viewing geometry, calibration, luminance range and the observer's lens and pigment sensitivities matter. A photograph can clip highlights, alter white balance and compress dynamic range; a display cannot necessarily reproduce the original spectrum, brightness or angular light field. A researcher studying spectral constancy should measure radiance and illuminants or specify the limits of a display simulation, rather than treating an uncalibrated image as physical ground truth.

Real materials also vary in more than diffuse reflectance. Shape, roughness, specularity, translucency and surrounding illumination jointly determine an image. Changing one rendering parameter can unintentionally change highlight area, contrast and apparent shape. Conversely, holding an image statistic constant can produce an unnatural stimulus. Experiments need independent manipulation of candidate causes and a clear statement of which physical scene properties are represented by the apparatus.

James M. Kraft and David H. Brainard demonstrated a productive compromise with a chamber of actual surfaces. They independently controlled illumination and surface reflectances and used a projection colorimeter to let observers adjust a test patch to appear achromatic. Manipulations could hold candidate image statistics approximately fixed while changing the illumination. The results rejected explanations based solely on local adaptation, the image mean or the most intense region.^[ref] The achievement was not realism alone: it was enough realism combined with manipulations that made competing mechanisms disagree. These results do not show that local adaptation never contributes.

For material appearance, Roland W. Fleming and colleagues used rendered objects illuminated by natural illumination distributions and systematically altered those distributions. Reflectance matching was more reliable under realistic illumination statistics than under some altered statistics.^[ref] This makes illumination an experimental variable, not merely a nuisance background. It also warns that a model tested only under a small set of studio lights can exploit cues unavailable or misleading elsewhere.

The dependent variable needs equal care. Selecting the same paint, matching apparent color, adjusting an achromatic point, detecting an illuminant change and naming a material do not measure the same outcome. A person may correctly infer that two patches are the same material while still noticing an appearance difference. Instructions, response format, adaptation time and available context can alter the task. Eye and head movements change retinal exposure, so fixation constraints should be chosen deliberately and their cost to natural viewing acknowledged.

A strong program therefore combines calibrated simplified stimuli, controlled real scenes, validated rendering and tests on previously unseen objects and illumination. It should sample more than one observer and record relevant visual variation, report both accuracy and uncertainty, and test model generalization across scenes rather than only across random frames of the same scene. Proposed interventions can vary geometry while preserving spectral input, or alter illumination while suppressing a particular cue. Their outcomes must be measured, not assumed. The goal is interpretable generalization: experimental control sufficient to distinguish mechanisms and scene diversity sufficient to test whether those mechanisms remain useful.

The visual system must infer relatively stable object properties from images jointly determined by illumination, shape, material and viewpoint. The problem is intrinsically ambiguous; robust performance exploits several constraints and tolerates uncertainty rather than perfectly recovering a unique physical scene from every image.

Even a diffuse model contains a surface-illuminant ambiguity. Ignoring a fixed geometric factor, $N(\lambda )=E(\lambda )R(\lambda )$. For any positive function $g$ that leaves the resulting reflectance physically admissible, define $E^{\prime }=Eg$ and $R^{\prime }=R/g$. Then $E^{\prime }{R}^{\prime }=ER$, so even an observer measuring the complete outgoing spectrum could not distinguish those two explanations from that surface alone. Three cone catches introduce further spectral metamerism. Thus perfect color constancy cannot be obtained by a universal local inversion of three numbers.

For directional optical reflection, an image sample instead depends on an integral such asThe expression sums contributions from incident directions, weighted by the bidirectional reflectance distribution function and projected area. It assumes a local, nonfluorescent optical reflection model and omits emission. A glossy surface therefore reflects aspects of its environment, and moving the observer changes the sampled outgoing direction even if the material is unchanged. Subsurface transport and transmission require further terms or models.

The system can constrain illumination by comparing surfaces, spatial boundaries and changes over time, while using shape and scene layout to distinguish a shadow from a pigment boundary. Cone-gain adjustment can approximately compensate shared spectral changes but cannot solve every mixture of differently colored lights, interreflections and specular components. A spatial average is unreliable if a scene is genuinely dominated by one colored material. A bright region might be a lamp, a highlight or a high-reflectance object, so “assume the brightest point is white” is not a universal solution.

Material judgments require distinguishing diffuse shading from highlights and transparency or translucency cues. Isamu Motoyoshi and colleagues showed that manipulating image statistics, including luminance-distribution skewness, could influence apparent gloss and lightness in their surface-image conditions.^[ref] Such regularities offer efficient cues. They cannot uniquely specify a material: a histogram-preserving spatial permutation changes spatial organization while retaining the luminance histogram and all its moments. That mathematical ambiguity motivates experiments separating histogram cues from geometric consistency; it does not, by itself, predict every appearance judgment after scrambling.

Juno Kim and colleagues directly tested the relation between highlights and diffuse shading. Manipulating their positional and orientation congruence reduced perceived gloss, including when simple proximity to broad luminance maxima was controlled.^[ref] The visual system therefore uses relationships among image structures, not merely the presence of a few bright pixels. The conclusion concerns the tested images and judgments, rather than a claim that one cue explains every shiny or transparent material.

Temporal information and active viewing help further. A highlight can move differently from a surface marking; multiple views constrain geometry, while familiar object structure and expected illumination supply priors. These strategies can fail under unusual lighting or deliberately misleading images. The system must also decide which differences matter: recognizing a material category, judging gloss and estimating spectral reflectance are related but separable goals.