= Laplace regression
{c}
= Least absolute deviations
{synonym}
For <linear regression> with fixed predictor vectors $X_i$, assume conditionally <independent> responses with a common scale $b>0$:
$$
Y_i\mid X_i\sim\operatorname{Laplace}(X_i^T\beta,b).
$$
Each conditional <median> and <mean> is $X_i^T\beta$. The <log-likelihood function> is $-n\log(2b)-S(\beta)/b$, so <maximum likelihood estimation> minimizes the sum of absolute <regression residuals>:
$$
S(\beta)=\sum_i|Y_i-X_i^T\beta|.
$$
This is <least absolute deviations>. With only an intercept, it reduces to the <Laplace location and scale maximum likelihood> problem, so a <sample median> is a fitted intercept. Provided the minimum $S(\widehat\beta)>0$, the scale estimate is $\widehat b=S(\widehat\beta)/n$; the conditional <standard deviation> is $\sqrt2\,b$. If the scale is denoted $\sigma$ instead of $b$, the conditional <standard deviation> is $\sqrt2\,\sigma$.
Minimizing a sum of <absolute losses> does not automatically give a unique coefficient vector. Even an intercept-only fit to an even <sample size> can have an interval of solutions, and redundant predictor columns can introduce further nonuniqueness. If a fit makes every <regression residual> zero, the joint <likelihood function> is unbounded as $b\downarrow0$, so there is no finite joint <maximum-likelihood estimate> with positive scale.
The linear growth of <absolute loss> limits sensitivity to an extreme response at fixed predictors compared with <squared-error loss>. It does not protect against every <outlier>: an extreme predictor can have large <regression leverage>. The numerical example above changes a response in an intercept-only problem, and makes no general claim of immunity to such predictor changes.
Back to article page
