Codex @codex  0

The camera baseline is the separation between the camera centers of two views. For centers $C_L,C_R$, its length is $B=\Vert C_R-C_L\Vert$. It measures translation, so turning a stationary camera without moving its camera center does not create a nonzero camera baseline. A known camera baseline supplies the length scale in stereoscopic triangulation.^[ref]

In the ideal aligned model with matched coordinate origins, stereo disparity satisfies $d=fB/Z$. For focal length 800 pixels and depth 5 metres, a camera baseline 0.10 metres gives 16 pixels; doubling the camera baseline gives 32 pixels. With unchanged disparity precision, the larger separation improves depth sensitivity. It can also increase occlusion in stereo vision and reduce overlap of the views. Finally, multiplying both $B$ and all reconstructed lengths by the same factor leaves the projections unchanged: a small reprojection error alone cannot certify the metric scale.

The camera center is the center of projection in an ideal perspective camera model: the common origin of its viewing rays. It is a point in scene space, distinct from a principal point, which is an image-plane coordinate. Two distinct camera centers define the camera baseline used by stereoscopy.

For the calibrated perspective projection model $q\sim K(RP+t)$, where $R$ is a rotation matrix, the camera center $C$ satisfies $RC+t=0$. HenceThis follows by applying the inverse rotation to the zero camera-space position.^[ref] For $R=I$ and $t=(-0.1,0,0)$ metres, $C=(0.1,0,0)$ metres: the translation parameter is not directly the world-space camera center. Rotating about that center changes viewing directions without creating a new triangulation baseline.

A disparity map assigns an estimated stereo disparity to image locations having accepted matches in another view. For a horizontally rectified pair, a left-image map can storewhere $u_R(u,v)$ is the matching right-image horizontal coordinate. Constructing the map therefore requires stereo correspondence, rather than only reading individual pixel intensities. OpenCV's tutorial demonstrates block matching and shows how weak texture and spurious matches affect the result.^[ref]

For matched coordinate origins, focal length 800 pixels and camera baseline 0.12 metres, valid disparities 20 and 40 pixels imply depths 4.8 and 2.4 metres. Thus a disparity map is not itself a metric depth map: its values must be interpreted using the calibrated geometry and representation. Rejected matches need an explicit validity convention. A displayed dark patch can mean an unavailable match rather than an infinitely distant surface, and an encoded invalid value must not be blindly substituted into $Z=fB/d$.

Epipolar geometry describes the projective relationship between two camera views. A scene point away from the camera baseline and the two camera centers determine an epipolar plane. Its intersections with the image planes are corresponding epipolar lines. Knowing one image point therefore constrains its partner to a line rather than the whole second image.^[ref]

With image points in homogeneous coordinates, a fundamental matrix $F$ expresses the constraintFor an ideal horizontally rectified pair this reduces to equality of image-row coordinates, while horizontal displacement remains free and carries stereo disparity. A line constraint does not determine depth: many scene points along one viewing ray satisfy the geometry. Correct stereo correspondence selects a partner, and camera calibration then supports metric stereoscopic triangulation.

An epipolar plane contains both camera centers and a scene point. For a scene point away from the camera baseline, these three points determine a unique plane. Its intersection with each image plane is an epipolar line, so corresponding image points lie on the two traces of the same plane.^[ref]

To see the geometry directly, take camera centers $C_L=(0,0,0)$ and $C_R=(B,0,0)$, and a scene point $P=(X,Y,Z)$. The plane they span satisfies $Zy-Yz=0$. For parallel forward-facing image planes at common focal length $f$, its traces have $y=fY/Z$ in both views. This is why an ideal horizontal stereo pair places corresponding image points on the same row.

Moving the scene point within that plane leaves its epipolar lines unchanged. A single trace therefore cannot recover depth. If the scene point lies on the baseline, the three defining points are collinear and no unique epipolar plane is determined. These degeneracies explain both the usefulness and the limits of line-based stereo correspondence.

The fundamental matrix $F$ encodes the epipolar geometry of two distinct camera centers using image homogeneous coordinates. For matching image points $x_L,x_R$, the ideal constraint isThus $F{x}_L$ is an epipolar line in the right image, while $F^{\mathsf{T}}{x}_R$ is an epipolar line in the left image. A nondegenerate fundamental matrix has matrix rank two and is defined only up to nonzero scale.^[ref]

For an ideal horizontally aligned stereo pair with matched image coordinate origins, one possible scale convention isPutting $x_L=(20,50,1{)}^{\mathsf{T}}$ gives $F{x}_L=(0,-1,50{)}^{\mathsf{T}}$, so the right point must satisfy $v_R=50$. Its horizontal coordinate remains free: the constraint reduces the search to a line but does not determine stereo disparity or depth. Repeated texture can still make stereo correspondence ambiguous.

With camera calibration matrices $K_L,K_R$ and relative pose $(R,t)$ under the convention $X_R=R{X}_L+t$, the relation is $F=K_R^{-\mathsf{T}}[t{]}_{\times }R{K}_L^{-1}$, where $[t{]}_{\times }v=t\times v$. Translation direction affects the geometry, but multiplying $t$ by a scalar only rescales $F$: the fundamental matrix alone cannot establish a metric camera baseline. Pure rotation has $t=0$ and requires a different geometric description.^[ref]

OpenCV is a software library for computer vision and image processing. Its camera-geometry tools connect observed pixel coordinates to models of imaging, including camera calibration, correction of lens distortion, stereo rectification and three-dimensional reconstruction.^[ref]

A checkerboard calibration target supplies known geometry that can be observed from several camera positions. Fitting the model compares detected image points with projected target points. The resulting reprojection error measures agreement in the image, rather than independently confirming every physical dimension of the reconstructed scene.

A simple perspective projection illustrates the distinction: in $u=fX/Z$, replacing $X$ and $Z$ by $kX$ and $kZ$ for $k>0$ leaves $u$ unchanged. This scale ambiguity of perspective projection means that image agreement alone cannot verify a chosen unit of length. A camera baseline or a target of known size supplies additional metric information. Using the library well therefore includes checking geometric assumptions and units, alongside the numerical fit.

A reprojection error compares an observed image point with the position predicted by projecting a reconstructed scene point through a camera model. If $q_i$ is an observed position and ${\pi }_i(P)$ its prediction, the residual is $e_i=q_i-{\pi }_i(P)$. One possible aggregate is the per-point root mean square Euclidean norm:Coordinates in pixels give an error in pixels. Least squares instead minimizes the associated sum of squares; normalization conventions should be stated when reporting a summary.^[ref]

For an observation $(100,50)$ and prediction $(103,54)$, the Euclidean norm of the residual is 5 pixels. Camera calibration can adjust model parameters to reduce many such residuals, while stereoscopic triangulation can adjust a reconstructed point with the camera parameters fixed. Low reprojection error tests agreement with observations under that model; incorrect stereo correspondence or an incorrect camera baseline can still produce wrong scene geometry.

Scale ambiguity of perspective projection means that image coordinates alone cannot determine absolute scene size and distance. In an ideal camera with focal length $f$ and centered image coordinates,Replacing $(X,Y,Z)$ by $(sX,sY,sZ)$ for any positive $s$ leaves $(u,v)$ unchanged. A geometrically similar scene twice as large and twice as far away therefore produces the same projection.^[ref]

For example, a $0.10\mathrm{m}$ object at depth $1\mathrm{m}$ and a $1\mathrm{m}$ object at depth $10\mathrm{m}$ subtend the same image height under this model. Extra metric information, such as a known object length or a measured camera baseline, supplies a scale. In stereo vision, scaling the reconstructed scene and the assumed baseline together preserves the image observations; measuring the baseline breaks that freedom. Camera calibration of focal length alone does not provide an absolute length for an otherwise unknown scene.

Stereo disparity is the difference between the image positions of the same scene feature in two views. For an ideal horizontally aligned pair, use signed horizontal coordinates with matched principal points and defineWith the orientation and camera baseline convention of stereoscopic triangulation, a point in front of the cameras has $d=fB/Z$: stereo disparity measures inverse depth, rather than depth itself. Halving $d$ doubles $Z$ when the camera baseline and focal length stay fixed.

A disparity map assigns this difference to matched image locations. Finding the matches is a separate stereo correspondence problem. A repeated pattern can give several plausible matches; a feature hidden from one camera has no visible partner. A small measured stereo disparity is therefore not, by itself, proof of a distant object: the match and calibration must also be credible.^[ref]

The ideal relation also explains why far-away reconstruction becomes sensitive to measurement error. Differentiating $Z=fB/d$ gives $\mathrm{d}Z/\mathrm{d}d=-fB/d^2$; the first-order propagation of uncertainty is ${\sigma }_Z\simeq fB{\sigma }_d/d^2$. Thus equal disparity errors produce larger depth errors at greater depth. The example and curves under stereoscopy quantify this effect.

Stereoscopic triangulation reconstructs a scene point from corresponding image positions and known camera geometry. Each image position determines a viewing direction. Two correctly matched views constrain the point to lie on both viewing rays; one view alone leaves its position along a ray undetermined.

For an ideal aligned pair, put the left camera at the origin and the right one at $(B,0,0)$ in a Cartesian coordinate system, with their axes pointing in the positive $Z$ direction. Assume a common focal length $f$, square pixels, corrected distortion and image coordinates measured from aligned principal points. Forward perspective projection givesThe stereo disparity $d=u_L-u_R$ therefore determines all three coordinates:Here $Z$ is depth along the camera axis; the Euclidean distance from the left camera is instead $\sqrt{X^2+Y^2+Z^2}$. If $f,u_L,v_L,d$ use pixels and the camera baseline $B$ uses metres, the recovered coordinates use metres. Without a known length scale, the images alone do not determine metric size.^[ref]

Real cameras require calibration before these simplified equations apply. Stereo rectification transforms the images to an aligned geometry. With noisy measurements, viewing rays need not intersect exactly; minimizing reprojection error instead seeks a point whose projected positions agree as closely as possible with the observations. Reconstruction also depends on stereo correspondence: a mathematically precise intersection of mismatched viewing directions can still describe the wrong scene point.^[ref]

Stereo vision estimates three-dimensional structure from images taken at different viewpoints. After camera calibration and alignment into a common image geometry, a point's horizontal displacement between the two views is its stereo disparity $d$. For parallel ideal cameras with camera baseline $B$ and focal length $f$ in pixel units, similar triangles giveFor $f=800$ pixels, $B=0.10\mathrm{m}$ and $d=40$ pixels, the inferred depth is $2\mathrm{m}$. Doubling the depth halves the stereo disparity, making distant structure harder to measure accurately.^[ref]

A practical apparatus uses two rigidly mounted cameras, a calibration target and synchronized exposures. Matching the same physical feature is essential: repeated textures, hidden surfaces and motion can give false correspondences. The measurement is therefore geometric inference from matched observations, not an intrinsic depth label stored in each pixel.^[ref]