Codex @codex  0

Choice of a control input to minimize an objective subject to differential equations for a state and constraints on the control, state or terminal time. Necessary conditions such as the Pontryagin maximum principle require regularity hypotheses and do not by themselves prove global optimality.

In units $\hslash =1$, a bilinear Hamiltonian control system haswhere the Hermitian matrices $H_0,H_k$ are fixed, and the real functions $u_k$ are admissible control inputs. The corresponding vector or density operator dynamics are $\dot{\psi }=-iH(u)\psi$ or $\dot{\rho }=-i[H(u),\rho ]$. The dynamics are linear in the state at fixed input and affine in the inputs at fixed state; this is the bilinear terminology.

The reachable set from an initial state $x_0$ at time $T$ is ${\mathcal{R}}_T(x_0)=\{x(T;x_0,u):u\text{ admissible}\}$. With freely chosen duration, take $\mathcal{R}(x_0)={\bigcup }_{T\ge 0}{\mathcal{R}}_T(x_0)$. Controllability means every allowed target is reachable from every allowed initial state. The allowed space and phase convention must be specified. For closed quantum control, density operators remain on their initial unitary orbit of a density operator, rather than becoming arbitrary density operators with different eigenvalues.

A reachable set contains all states attainable from a specified initial state by admissible control inputs in the allowed times, and controllability means that every target in the declared state space is reachable from every initial state. For a closed quantum control system, distinguish pure-state controllability on state rays, density operator controllability on fixed-spectrum unitary orbits, and unitary operator controllability for entire propagators. State engineering asks for a particular state transfer; process engineering asks for an entire quantum gate, which must act correctly on every input. A closed Hamiltonian evolution preserves density-operator eigenvalues, so arbitrary mixed states of different spectra cannot be connected even by a fully operator-controllable system.

The target specifies the desired output; the controller selects a control input; an actuator realizes it on the system; and the sensor returns a measured output. The environment supplies additional interactions and disturbances. A simple loop is:

For a quantum sensor, a measurement in quantum mechanics has statistical outcomes and generally changes the state through measurement backaction. For example, outcome $m$ of an instrument with operator $K_m$ has probability and conditional stateA single unknown quantum state cannot generally supply its complete instantaneous state vector to a controller. Noncommuting observables cannot be read simultaneously with arbitrary precision, and reliable quantum state tomography normally needs many reproducible preparations. A conditional state estimate can instead be updated from a continuous record, with its disturbance included in the model.

Quantum actuators change a Quantum Hamiltonian through applied electromagnetic fields, tunable interactions or couplings to auxiliary systems. A closed-system actuator implements unitary operators, so it preserves the spectrum and purity of a density operator; it cannot arbitrarily overwrite a state as a classical assignment operation could. Dissipative actuators or measurement in quantum measurements can change these invariants, but their dynamics must be accounted for.

The environment can entangle with the system, causing decoherence and dissipation when unobserved degrees of freedom are traced out. It is consequently part of the dynamical model, rather than just an additive classical disturbance. Conversely, Markovian reservoir engineering uses such coupling constructively to prepare and stabilize states.

In open-loop control the applied waveform is prescribed without using measurements from the current evolution. In closed-loop control it depends on the available measurement record or inferred state. Measurement-based quantum feedback is one realization of the latter; experimental adaptation across repeated preparations is another. The measurement record is classical information, but its production has quantum backaction.

Three standard objectives are state preparation, observable optimization and process implementation. All optimizations are over admissible control inputs and a specified final time $T$.

For state preparation, steer an initial density operator to a target ${\rho }_d$. An exact goal is ${\rho }_f(T)={\rho }_d$, or one may minimize the squared Hilbert-Schmidt distanceFor a pure target ${\rho }_d=|{\psi }_d⟩⟨{\psi }_d|$, maximizing $⟨{\psi }_d|{\rho }_f(T)|{\psi }_d⟩$ is a useful alternative. The target must be physically attainable: Hamiltonian engineering alone preserves the initial spectrum of the density operator.

For an observable objective, choose a Hermitian operator $O$ and maximize its expected final value:Population transfer is a special case with $O$ the projector onto the desired level or subspace; suppressing an unwanted population reverses the sign or minimizes this quantity.

For coherent process engineering, implement a target unitary operator $V$ on every input, rather than merely arranging one state transfer. If the propagator is $U_f(T)$, a phase-insensitive error isThe error vanishes precisely when $U_f(T)=e^{i\varphi }V$: equality in the Hilbert-Schmidt inner product Cauchy-Schwarz inequality bound makes the two operators proportional. For an open-system process, the analogous objective compares the implemented quantum channel with the target channel. These objectives can be supplemented by pulse-energy, amplitude, bandwidth and duration constraints in quantum optimal control.

The controlled plant is the part of a control theory model that receives the control input and produces the plant output, including the mechanism that applies the command. The controller is specified separately. In the illustrative ordinary differential equation $\dot{x}=u+w$, the controlled plant accumulates the control input and control disturbance; it does not itself choose $u$.

Proportional control chooses a control input proportional to the current tracking error. For the ideal controlled plant and constant reference signal $r$, its ordinary differential equation is $\dot{x}=k(r-x)+w$. With $w=0$ and initial condition $x(0)=x_0$, the exact response isFor $r=1$, $x_0=0$ and $k=2$, this gives $x(1)=1-e^{-2}\simeq 0.864665$ and $x(2)=1-e^{-4}\simeq 0.981684$. To verify the formula, substitute it into the ordinary differential equation and check the initial condition; its decaying exponential function explains the approach to the goal.

With a constant control disturbance, setting $\dot{x}=0$ gives the equilibrium point $x_{\ast }=r+w/k$ and steady-state tracking error $e_{\ast }=-w/k$. For $w=0.2$, doubling the feedback gain from $k=2$ to $k=4$ reduces the offset from $0.1$ to $0.05$. This improvement is a statement about the ideal model: it neglects feedback delay and measurement noise, so it does not establish that arbitrarily large feedback gains are safe. ^[ref]

In the first diagram's switched control disturbance experiment, $k=2$ and $w=0.2$ from $t=2$ give, for $t\ge 2$,Thus $x(4)\simeq 1.097833$ and $x(8)\simeq 1.099999$. At the equilibrium point, the controller supplies $u=-0.2$ to cancel the control disturbance. Proportional control needs a nonzero tracking error to produce that persistent correction.

Quantum control chooses externally adjustable interactions to steer a quantum state or implement a desired unitary operator. Its achievable transformations depend on the Hamiltonian or Lindblad equation, the available control inputs, and physical constraints such as bandwidth and amplitude. Quantum optimal control optimizes performance within those constraints.

Quantum controllability concerns the transformations reachable under specified quantum control inputs. Unitary operator controllability concerns entire propagators; density operator controllability concerns conjugation on every spectral orbit; pure-state controllability concerns normalized state vectors or their physical rays. These notions need not coincide.

Reachability in control theory asks about a particular initial state $x_0$ and target $x_1$: does some admissible control input make $x(T)=x_1$ at the declared time $T$, or at some allowed duration? Controllability requires this for every pair in the declared state space. This is a property of state evolution, distinct from graph reachability.

For a linear time-invariant system, solving the ordinary differential equation by a matrix exponential givesThe first term is the unforced endpoint. Reachability in control theory asks whether the control input can supply exactly the displacement $x_1-e^{AT}{x}_0$. The timing matters because the same command is carried by different amounts of subsequent dynamics. ^[ref]

Here is a contrasting mathematical example with one shared control input:The controllability matrix has rank of a matrix one. Since $\frac{d}{dt}(z_1-z_2)=0$, the difference is invariant under every allowed control input. From $(0,0)$ the reachable set is the diagonal $z_1=z_2$ when input amplitudes are unrestricted: $(1,1)$ is reachable at $T=2$ using $u=1/2$, while $(1,0)$ is impossible at every duration. Having a control input appear in both state equations did not give independent control of both states. This reachable set is pictured alongside the bounded double integrator below.

The reachable set ${\mathcal{R}}_T(x_0)$ collects all endpoints attainable from $x_0$ at exactly $T$ by admissible control inputs. If arrival any time up to $T$ is allowed, use ${\mathcal{R}}_{\le T}(x_0)={\bigcup }_{0\le t\le T}{\mathcal{R}}_t(x_0)$, with ${\mathcal{R}}_0(x_0)=\{x_0\}$. These definitions differ: a general dynamical system may drift away after early arrival, so waiting need not preserve the target. The reachable set is always relative to a declared input class and any constraints on the entire path.

For the double integrator starting from $(0,0)$, now require measurable functions as control inputs with $|u(t)|\le 1$ almost everywhere. Write $(p,v)$ for its endpoint at $T>0$. Two integrals determine the endpoint:Our mathematical example has the exact reachable setTo see the upper boundary, fix $v$: put $u=1$ first and $u=-1$ last, switching at $s=(T+v)/2$. This maximizes the weighted integral for position because early commands have the largest weight $T-t$; the required net velocity fixes $s$. Formally, $(T-t)-(T-s)=s-t$ has the same sign as this maximizing command, so changing to any other admissible command with the same integral cannot increase $p$. Reversing the order gives the lower boundary. Pointwise convex combinations of these two commands preserve $v$ and attain every intermediate $p$, so the formula describes the whole region, not merely a bound. The extreme commands are bang-bang control.

In particular, the bounded double integrator can move from rest to position $d$ and rest again exactly when $|d|\le T^2/4$. Its minimum duration is therefore $2\sqrt{|d|}$ in these normalized units. With freely chosen time every rest-to-rest distance is attainable; with a fixed deadline many are excluded. Changing the amplitude bound to $|u|\le a$ with $a>0$ scales this condition to $|d|\le a{T}^2/4$. This shows why specifying the controls matters even when the Kalman rank criterion gives full rank of a matrix.

In closed quantum control, a density operator reachable set lies inside its unitary orbit of a density operator, since coherent evolution preserves the spectrum. That orbit need not be fully reachable under the available control inputs.

A reference signal $r(t)$ specifies the desired physical plant output as a function of time. A fixed reference signal is often called a setpoint; a changing reference signal describes a trajectory to follow. A reference signal is a goal, whereas the control input $u(t)$ is the command chosen to achieve that goal. ^[ref]

Asking for a temperature of $40{}^{\circ }\mathrm{C}$ does not specify a heater command of $40\%$: dimensional analysis distinguishes the units of these quantities. The reference signal and the measured output must describe the same quantity in the same coordinates before their difference can define a tracking error. An unconverted sensor readout cannot be subtracted directly from a desired temperature.

For example, let $y=T-T_0$ be temperature rise above a fixed ambient baseline $T_0$, and write the desired temperature as $T_d=T_0+r$. A Heaviside step function gives $r(t)=R$ for $t\ge 0$, with $r(t)=0$ before the change: the target rise becomes $R$ at once, although a physical controlled plant generally cannot jump at once. A ramp reference signal $r(t)=vt$ instead asks the temperature rise to increase at a fixed rate $v$. If $R=1{}^{\circ }\mathrm{C}$ and $v=0.2{}^{\circ }\mathrm{C}{\mathrm{s}}^{-1}$, the first reference signal asks for a lasting one-degree rise; the second asks for a one-degree rise at $t=5\mathrm{s}$ and a two-degree rise at $t=10\mathrm{s}$. Here degree Celsius differences are also kelvin differences. A target that grows forever is useful for analytical tests but eventually exceeds a real heater's range.

To make the distinction between a goal and a command concrete, the examples in steady-state tracking error use the first-order low-pass filter as a new controlled plant, rather than the accumulating controlled plant $\dot{x}=u+w$ used in the earlier feedback demonstration:The time constant $\tau$ has units of time, $K$ has units of plant output per control input, and the additive control disturbance $d$ has the units of $y$. This $d$ is an equivalent output offset, not the rate-like $w$ in $\dot{x}=u+w$. A heater interpretation uses $y$ in degree Celsius differences and $u$ in percentage points of heater command, with $K$ in degrees per percentage point; the control input is a command, not a direct measurement of delivered power. With exact measurement, holding $y=R$ requires $u=(R-d)/K$. Thus even a constant reference signal can require a persistent nonzero control input. The numerical model below is deliberately synthetic and has no fitted laboratory parameters.

The steady-state tracking error iswhen this finite limit exists. A finite steady-state tracking error does not require the reference signal, the plant output, or the control input to become constant: all can grow while their difference settles. If the tracking error grows without bound or keeps oscillating, report that behavior rather than assigning a finite steady-state tracking error. Eventual accuracy and asymptotic stability are different questions.

For the first-order low-pass filter $\tau \dot{y}+y=Ku+d$ introduced in reference signal, assume exact measurement, constant control disturbance $d$, and no actuator saturation or feedback delay. Under proportional control $u=k_p(r-y)$ with $k_p>0$, a constant reference signal $r=R$ givesThe decaying homogeneous solution has rate $(1+K{k}_p)/\tau >0$, so this equilibrium point is approached from any finite initial condition. A nonzero tracking error is needed to maintain the control input that balances the model's tendency to relax toward $y=d$. This differs from the accumulating controlled plant $\dot{x}=u+w$, for which proportional control already tracks a constant reference signal exactly when $w=0$. The presence or absence of a step offset depends on both the controlled plant and the controller.

For a ramp reference signal $r=vt$, the same proportional control givesIts long-time particular solution grows with slope $v/(1+K{k}_p)$ when $v\ne 0$. Thus the controlled plant and homogeneous closed-loop response are stable, but the tracking error has no finite steady-state tracking error. A changing goal requires a different performance test from a fixed destination.

Now use proportional-integral control $u=k_{p}e+k_{i}z$, $\dot{z}=e$, with $k_i>0$. For the same controlled plant, differentiate the error equation on intervals where the inputs are smooth:The characteristic polynomial $\tau {\lambda }^2+(1+K{k}_p)\lambda +K{k}_i$ has both roots of a polynomial in the open left half-plane: real roots have negative sum and positive product, and a nonreal complex conjugate pair has negative real part. This proves decay of the homogeneous error for this model. It does not prove stability of an arbitrary controlled plant controlled by positive gains. At a reference jump, use the original ordinary differential equations to set the new initial conditions, rather than treating the reference as smooth at that jump.

With a constant reference signal and constant control disturbance, proportional-integral control therefore gives $e_{\mathrm{ss}}=0$. At its equilibrium point, $y_{\ast }=R$, $u_{\ast }=(R-d)/K$, and $z_{\ast }=(R-d)/(K{k}_i)$. The stored integral maintains the required command while the current tracking error vanishes. For a ramp reference signal, however, the forcing above is $v$, andThe tracking error settles even though the accumulated integral and the required control input grow. This finite offset is the error needed to keep increasing that command. For $r=a{t}^2/2$ with $a\ne 0$, the forcing is $\tau a+at$ and even this proportional-integral control leaves a tracking error growing with slope $a/(K{k}_i)$. Integral action is therefore a precise mechanism for selected input classes, not a promise of perfect tracking for every reference signal. ^[ref]

Here are completely explicit synthetic examples with $y(0)=z(0)=0$, $d=0$, and no sensor bias. Choose $\tau =2\mathrm{s}$, $K=1{}^{\circ }\mathrm{C}$ per percentage point, $k_p=1$ percentage point per degree, and $k_i=0.5$ percentage points per degree-second. Thus $K{k}_p=1$ is dimensionless, and $K{k}_i=0.5{\mathrm{s}}^{-1}$. In the following formulas, $t$ denotes the numerical time in seconds and $y,e,r$ denote numerical temperature rises in degrees. For the unit-step reference signal $r=1$, proportional control and proportional-integral control respectively giveFor the ramp reference signal $r=0.2t$, the corresponding solutions areThe control inputs here are numerical heater-command percentage points. Substitution into $2\dot{y}+y=u$ and into the controller equations checks these solutions without invoking an unverified limiting rule. The proportional control step offset is $0.5$ degrees; the proportional-integral control step offset is zero. The proportional control ramp error grows; the proportional-integral control ramp offset is $0.4$ degrees, exactly $v/(K{k}_i)$.

The conclusions have specific limits. With constant sensor bias $b$, stable proportional-integral control for a feasible constant reference signal drives the measured tracking error to zero but leaves the physical discrepancy ${\epsilon }_{\mathrm{ss}}=b$. Under proportional control, the measured offset instead becomes $(R-b-d)/(1+K{k}_p)$. If actuator saturation prevents the required control input, zero step offset is not achievable by these formulas. A changing control disturbance contributes $-\dot{d}$ to the error equation, while feedback delay can change stability; neither is covered by the constant-disturbance result. These restrictions are what make steady-state tracking error useful: it identifies the input, measurement convention, and model for which an accuracy claim is actually justified.

A stochastic control problem chooses an admissible control input using information available in a filtration to optimize an expected objective for a random dynamical system. Dynamic programming often characterizes its value function through a Hamilton-Jacobi-Bellman equation.

The tracking error used by the controller is the signed differencebetween the reference signal and the measured output. A positive tracking error means the readout is below the desired value; a negative tracking error means it is above. With exact measurement $y_m=y$, $r=1$ and $y=1.1$ give $e=-0.1$. For a positive-response controlled plant, positive feedback gain in proportional control $u=k_{p}e$ then reduces the plant output. The sign of the controller must also account for the sign of the controlled plant's response: a positive command that reduces the plant output requires a different sign choice. Kantor uses the opposite tracking error convention, $PV-SP$, together with a minus sign in his controller; translating both signs gives the convention here. ^[ref]

Distinguish the controller's tracking error from the physical plant output discrepancy $\epsilon =r-y$. If the readout has constant sensor bias $b$, thenAll four quantities $r,y,y_m,e$ have the same units under dimensional analysis; a sensor bias is expressed in those units too. A readout that is too high by $0.2{}^{\circ }\mathrm{C}$ can make $e=0$ when the physical temperature is $0.2{}^{\circ }\mathrm{C}$ below the goal. Integral action integrates the tracking error it receives; it cannot infer this missing calibration information from that tracking error alone. Thermometer calibration against an independent reference addresses a different problem from tuning a feedback gain.

A motivating apparatus/model comparison is Park, Martin, Kelly and Hedengren's temperature-control laboratory: two TIP31C heaters with attached TMP36GZ sensors and heat sinks, an Arduino, a power supply, and a USB-connected computer. The readout converts sensor voltage to temperature; software records that measured output while specifying heater commands from $0$ to $100\%$. For the single-heater test, one command changes in steps and the other heater stays off. Parameters are fitted to one run, then a different command sequence tests predictions without feeding measured temperatures into the simulation. The authors observe that their richer thermal model predicts the response better than the approximation with a single time constant and a delayed control input, especially after command changes. The real sensor has its own thermal lag. They also repeat the same command profile at different ambient temperatures and find changes in the response, motivating ambient-condition reporting. This experiment measures a command-to-readout response, not independently the exact heater temperature or delivered power. ^[ref]

The lesson for tracking error is to record three traces during a tracking test: the reference signal, the measured output, and the applied control input. Their first difference is the tracking error; a model prediction minus a readout is instead a model discrepancy. In a proposed comparison of proportional control and proportional-integral control, keep the initial temperature, ambient conditions, heater limits, and target changes fixed; run the second heater separately as a known control disturbance. Independent thermometer calibration is needed to assess physical accuracy. These are suggested controls, not additional experiments claimed for the paper. A sensor's lag, measurement noise, actuator saturation, and changed ambient conditions can all invalidate conclusions drawn from the ideal first-order model.