Sensory substitution system

An apparatus capturing, transforming and presenting information through an available sensory channel. Evaluation must distinguish transmitted resolution from human recoverability, and laboratory identification from practical use.

A camera-to-sound sensory-substitution system developed by Peter B. L. Meijer. Its column scan encodes vertical position by frequency, horizontal position by scan timing and image brightness by amplitude. Later implementations add other cues; properties of one version should not be attributed to every prototype.

A rule converting image properties into acoustic features. One column-scanning example assigns vertical row to $f_i$, horizontal column to scan time, and brightness to amplitude $A(I_{ij})$, yielding $s_j(t)={\sum }_{i}A(I_{ij})\sin (2\pi f_{i}t+{\varphi }_i)$. Distinct orthogonal tones can preserve column coefficients in ideal conditions, while finite windows, masking and limited hearing resolution constrain perception.