Stereoscopic triangulation

Stereoscopic triangulation reconstructs a scene point from corresponding image positions and known camera geometry. Each image position determines a viewing direction. Two correctly matched views constrain the point to lie on both viewing rays; one view alone leaves its position along a ray undetermined.

For an ideal aligned pair, put the left camera at the origin and the right one at $(B,0,0)$ in a Cartesian coordinate system, with their axes pointing in the positive $Z$ direction. Assume a common focal length $f$, square pixels, corrected distortion and image coordinates measured from aligned principal points. Forward perspective projection givesThe stereo disparity $d=u_L-u_R$ therefore determines all three coordinates:Here $Z$ is depth along the camera axis; the Euclidean distance from the left camera is instead $\sqrt{X^2+Y^2+Z^2}$. If $f,u_L,v_L,d$ use pixels and the camera baseline $B$ uses metres, the recovered coordinates use metres. Without a known length scale, the images alone do not determine metric size.^[ref]

Real cameras require calibration before these simplified equations apply. Stereo rectification transforms the images to an aligned geometry. With noisy measurements, viewing rays need not intersect exactly; minimizing reprojection error instead seeks a point whose projected positions agree as closely as possible with the observations. Reconstruction also depends on stereo correspondence: a mathematically precise intersection of mismatched viewing directions can still describe the wrong scene point.^[ref]

A reprojection error compares an observed image point with the position predicted by projecting a reconstructed scene point through a camera model. If $q_i$ is an observed position and ${\pi }_i(P)$ its prediction, the residual is $e_i=q_i-{\pi }_i(P)$. One possible aggregate is the per-point root mean square Euclidean norm:Coordinates in pixels give an error in pixels. Least squares instead minimizes the associated sum of squares; normalization conventions should be stated when reporting a summary.^[ref]

For an observation $(100,50)$ and prediction $(103,54)$, the Euclidean norm of the residual is 5 pixels. Camera calibration can adjust model parameters to reduce many such residuals, while stereoscopic triangulation can adjust a reconstructed point with the camera parameters fixed. Low reprojection error tests agreement with observations under that model; incorrect stereo correspondence or an incorrect camera baseline can still produce wrong scene geometry.