Control theory

Control theory studies dynamical input-output systems, feedback, stability, and response.

A chosen input to a dynamical system, commonly written $u(t)$ in the state equation $\dot{x}=f(t,x,u)$. An optimal control problem specifies both the admissible input set and any information or regularity constraints on the choice.

Choice of a control input to minimize an objective subject to differential equations for a state and constraints on the control, state or terminal time. Necessary conditions such as the Pontryagin maximum principle require regularity hypotheses and do not by themselves prove global optimality.

A stochastic control problem chooses an admissible control input using information available in a filtration to optimize an expected objective for a random dynamical system. Dynamic programming often characterizes its value function through a Hamilton-Jacobi-Bellman equation.

In singular stochastic control, a control is a finite-variation process rather than only a rate integrated against time. Its increments can have atoms or a part singular with respect to time. The Hamilton-Jacobi-Bellman equation becomes a variational inequality with a constraint on directional derivatives of the value function.

Direct-adjoint looping repeatedly integrates a constrained state equation forward, integrates its costate or formal adjoint backward using terminal data, extracts the control gradient, and updates the control input within its constraints. It gives a practical route to local optimization but does not by itself prove a global minimum.

For a differentiable objective on an admissible real Hilbert space $H$, with $\Vert {\mathbf{u}}_0{\Vert }^2/2=E_0>0$, stationarity on the sphere in a normed vector space of fixed kinetic energy means that the projected gradient is normal to the sphere in a normed vector space. Thus $P_H\nabla J=\lambda {\mathbf{u}}_0$ for a real Lagrange multiplier. It follows by testing all tangent variations orthogonal to ${\mathbf{u}}_0$. The multiplier need not be positive; second-order conditions are still needed for a local minimum.

A control input taking only extreme admissible values except possibly on a switching or singular set. If the Hamiltonian of an optimal-control problem is affine in a scalar control $u\in [0,1]$, a nonzero coefficient selects $u=0$ or $u=1$; a zero coefficient does not determine the control.

An adjoint variable enforcing the state dynamics in optimal control. In the normal minimization convention it satisfies $\dot{p}=-H_x$ for the Hamiltonian of an optimal-control problem, with terminal conditions determined by the endpoint constraints.

In a normal minimization convention for optimal control, the running cost $L$ and dynamics $\dot{x}=f$ define $H=L+p\cdot f$. The multiplier $p$ is a costate; this Hamiltonian is a device for variational necessary conditions rather than necessarily a physical energy.

For a regular finite-horizon optimal control problem, an optimal trajectory admits nontrivial multipliers and a costate obeying adjoint equations, together with pointwise optimization of the Hamiltonian of an optimal-control problem. In the normal minimization convention $H=L+p\cdot f$, the conditions are $\dot{x}=H_p$, $\dot{p}=-H_x$ and minimization of $H$ over admissible controls; endpoint and state constraints change the terminal conditions. Abnormal extremals use a zero cost multiplier and must not be silently excluded.

For a smooth normal optimal control problem with freely variable terminal time, fixed terminal state independent of that time and terminal cost $\Phi$, the optimized Hamiltonian of an optimal-control problem at completion satisfies $H(T)+{\Phi }_T=0$. It follows by varying the terminal time in the augmented objective; restrictions on terminal time or a moving endpoint introduce additional terms or inequalities.

State estimation infers an unobserved dynamical state from noisy observations and a model of its evolution.

The Kalman filter recursively computes the conditional mean and covariance of the state in a linear system with Gaussian initial state and Gaussian noise.

Suppose $x\mid \mathcal{F}\sim N(\hat{x},V)$ and the new observation is $y=x+\eta$, where $\eta \sim N(0,R)$ is independent. ThenThe coefficient $V/(V+R)$ is the scalar Kalman gain.

Linear-quadratic-Gaussian control combines a linear system, Gaussian uncertainty, a quadratic cost, and feedback based on a Kalman filter state estimate.

In a linear-quadratic-Gaussian control problem, the optimal control applies the full-information linear-quadratic feedback law to the conditional state estimate. The control gain is independent of the state-estimation error covariance.

The continuous Lyapunov equation is $AX+X{A}^T+Q=0$. If every eigenvalue of $A$ has negative real part, it has a unique solution for each $Q$, and a positive-definite $Q$ gives a positive-definite $X$.

For a linear time-invariant system with zero initial data, the transfer function is the Laplace-domain ratio $G(s)=Y(s)/F(s)$. A rational causal system is BIBO stable when all poles lie in the open left half-plane.

Let $L$ be the open-loop transfer function. The winding of its Nyquist plot around $-1$, together with the number of unstable open-loop poles, determines the number of unstable zeros of $1+L$, which are the closed-loop poles.