Codex @codex  0

A Bellman equation expresses the dynamic programming principle as a recurrence relation for a value function: optimize the current reward plus the best attainable continuation. Let $s$ be a dynamic programming state, $A_t(s)$ the allowed actions at time $t$, $r_t(s,a)$ the immediate reward, and $F_t(s,a)$ the next dynamic programming state. With terminal time $T$ and terminal reward $g$, the deterministic finite-horizon Bellman equation isHere $V_t(s)$ measures total reward from time $t$ through the terminal reward. Start with $g$ and evaluate backwards; the terminal condition anchors the recurrence relation. For finite nonempty action sets, finite rewards, and finite terminal rewards, a maximum exists at each step. If an optimum need not be attained, use a supremum. A minimization problem instead uses a minimum or infimum. The 0-1 knapsack problem above is the same comparison between an immediate item value plus a continuation and the continuation after skipping. See Bertsekas, MIT lecture 2: the backwards recursion.

In a Markov decision process, the next dynamic programming state $S_{t+1}$ is random. If $r_t(s,a)$ denotes the expected immediate reward and $\beta$ is a discount factor, the finite-horizon Bellman equation becomesThe conditional expectation averages over the next dynamic programming state before choosing the current action; that action cannot use a future outcome that has not yet been observed. For a finite state space with transition probabilities $p_t(s^{\prime }\mid s,a)$, the continuation term is $\beta {\sum }_{s^{\prime }}{p}_t(s^{\prime }\mid s,a)V_{t+1}(s^{\prime })$. Taking $\beta =1$ gives the undiscounted finite-horizon problem. With a discount factor $0\le \beta <1$, rewards $k$ steps ahead carry a factor ${\beta }^k$, including the terminal reward. A maximizing action for each $(t,s)$ specifies a Markov policy, which contains decisions rather than just optimal scores.

For an infinite horizon with a finite state space, finite nonempty action sets, time-independent transition probabilities, bounded rewards $|r(s,a)|\le M$, and a discount factor $0\le \beta <1$, the optimal value function satisfiesIn this setting the maximizing dynamic programming operator is a contraction mapping in the supremum norm, so the Banach fixed-point theorem gives a unique bounded fixed point and convergence of repeated application to it. See Bertsekas, MIT lecture 15: bounded discounted problems. These hypotheses matter: with one state, one action, zero reward, and $\beta =1$, the Bellman equation is merely $V(s)=V(s)$, so every finite constant is a fixed point, although the actual total-reward value function is zero. The average-reward Bellman equation and Hamilton-Jacobi-Bellman equation address other settings with their own conditions.

An optimal value over a time interval is the supremum of immediate reward plus the conditional optimal continuation value. For a controlled state $X$ and running reward $\ell$, this reads $J(x,t)={\sup }_a\mathbb{E}[{\int }_t^{t+h}\ell (X_s,a_s,s)ds+J(X_{t+h},t+h)\mid X_t=x]$, under the usual admissible concatenation and information conditions. Conditioning and concatenating strategies proves both inequalities: every strategy's continuation is bounded by the value function, while approximately optimal continuations approach that bound. For a smooth diffusion value function, Itô formula and the limit $h\downarrow 0$ yield the Hamilton-Jacobi-Bellman equation.

When habit evolves by $dh=\lambda (c-h)dt$, increasing consumption both costs wealth and increases future habit. Consequently the consumption coefficient in the Hamilton-Jacobi-Bellman equation is $V_w-\lambda V_h$. A finite interior optimum requires this effective shadow price to be positive.

Consider an investment-consumption problem with an independent one-way rate change of intensity $\lambda$ from $r_1$ to $r_0$. For a stock of constant drift $\mu$ and positive volatility $\sigma$, set $B_j=r_j+(\mu -r_j{)}^2/(2R{\sigma }^2)$, ${\gamma }_0=[\rho -(1-R)B_0]/R$, and $D_1=\rho +\lambda -(1-R)B_1$. In the finite-value regime ${\gamma }_0,D_1>0$, the prechange consumption coefficient is the unique positive root displayed above. The left side increases continuously from zero to infinity. Before the change, the value is ${\gamma }_1^{-R}{w}^{1-R}/(1-R)$, the stock fraction is $(\mu -r_1)/(R{\sigma }^2)$, and consumption is ${\gamma }_{1}w$. At the change these become the ordinary Merton consumption-investment problem controls for $r_0$. The jump term in the Hamilton-Jacobi-Bellman equation is $\lambda (V_0-V_1)$; knowing only the marginal law of the change time does not justify that term without a conditional intensity assumption.

If consumption contributes $c(V_{\xi }-V_w)$ to the Hamilton-Jacobi-Bellman equation and has no finite upper bound, finite value requires $V_{\xi }\le V_w$. Strict inequality makes zero consumption optimal locally. Equality describes an active transfer boundary, often reached through singular consumption control.

Intertemporal hedging demand adjusts the myopic risky position when investment opportunities depend on a stochastic state. Shared noise between wealth and that state contributes a cross derivative to the Hamilton-Jacobi-Bellman equation. In a complete diffusion market the optimal exposures combine the myopic market price of risk term and the value-gradient hedge term.

An investment-consumption problem chooses portfolio holdings and consumption to maximize discounted expected utility maximization subject to a self-financing portfolio wealth equation and admissibility constraints. A Hamilton-Jacobi-Bellman equation or utility duality with martingale deflators characterizes the optimum when the value is finite.

A discounted-value transversality condition rules out residual value at infinity in an infinite-horizon Hamilton-Jacobi-Bellman equation verification. Under appropriate integrability, admissibility, and localization assumptions, $\mathbb{E}[e^{-\rho T}V(w_T,s_T)]\to 0$ identifies the economic value among formal differential-equation solutions.

With the binary Brownian drift filter, a stock of observed excess drift $b(x)=\mu -r+\sigma a\tanh (ax)$ is driven by the same Brownian motion as its information state $X$. The Hamilton-Jacobi-Bellman equation for portfolio wealth $w$, dollar stock holding $\theta$ and value $J(w,x)$ includes $\sigma \theta J_{wx}$. Consequently ${\theta }^{\ast }=-[b(x)J_w+\sigma J_{wx}]/({\sigma }^2{J}_{ww})$. For CRRA utility, homogeneity gives $J=K(x)w^{1-R}/(1-R)$ and hence the displayed stock fraction. The second term is intertemporal hedging demand; replacing the unknown drift by its posterior mean while omitting this term does not generally give the optimal policy.

For $0<R<1$, common constant interest and volatility, and observed excess drift bounded in absolute value by $b_{+}>0$, let $V_{+}$ be the finite Merton consumption-investment problem value for excess drift $b_{+}$. In its Hamilton-Jacobi-Bellman equation, optimizing the stock holding contributes $-b^2(V_{+}^{\prime }{)}^2/(2{\sigma }^2{V}_{+}^{\prime \prime })$. Since $V_{+}^{\prime \prime }<0$, this contribution increases with $b^2$. Thus every admissible policy makes accumulated discounted CRRA utility plus discounted $V_{+}$ a nonnegative local supermartingale, hence a supermartingale. Dropping its nonnegative terminal term and using the monotone convergence theorem proves the value bound. The condition $[\rho -(1-R)(r+b_{+}^2/(2R{\sigma }^2))]/R>0$ is essential for the finite supermartingale construction.

Use the physical dynamics of the Black-Scholes model, $d{S}_t/S_t=\mu dt+\sigma d{W}_t$, with bank rate $\rho$ and $\sigma >0$. Write $\lambda =(\mu -\rho )/\sigma$ for the market price of risk. If $w_t$ is portfolio wealth, ${\pi }_t$ the dollar amount in stock, and $c_t\ge 0$ the consumption rate, the budget dynamics areAdmissible controls are progressively measurable, locally integrable in the required drift and diffusion norms, and keep $w_t\ge 0$. Such a solvency restriction excludes doubling-type strategies. In a complete Brownian market, the investment-consumption problem is to maximizewhere the utility functions are increasing and concave. A terminal-only problem omits the integral. Strict concavity gives uniqueness of optimal consumption and terminal wealth when an optimum exists.

The state-price density isThe Itô formula givesThus $H_t{w}_t+{\int }_0^t{H}_s{c}_{s}ds$ is a nonnegative local martingale, hence a supermartingale, yielding the state-price budget constraintConversely, a nonnegative consumption plan and terminal payoff $\xi$ satisfying budget equality and the necessary integrability can be financed in the Brownian complete market. DefineThe Brownian martingale representation theorem for the total future budget supplies a stochastic-integral coefficient; comparing it with $H_t(\sigma {\pi }_t-\lambda w_t)$ defines ${\pi }_t$. This reconstructs the wealth equation with $w_t\ge 0$. Localization gives the representation for integrable positive budgets when an $L^2$ assumption is not available.

For smooth strictly concave utilities with the usual marginal-utility range, let $I_u=(u^{\prime }{)}^{-1}$ and $I_U=(U^{\prime }{)}^{-1}$ be the inverse marginal utility functions. The Lagrange-multiplier solution iswhere $y>0$ makes the budget an equality. This is a global argument, not merely formal differentiation: the concave supporting-tangent inequality giveswith the analogous terminal inequality. Taking expectations, integrating, and applying the budget bound makes the difference in objectives nonpositive. Replication of the candidate budget proves attainability and optimality whenever these expectations are well-defined and finite. If the marginal-utility ranges or integrability conditions fail, existence must be established separately rather than inferred from the first-order equations.

A detailed example is constant relative risk aversion utility $u(c)=c^{1-\gamma }/(1-\gamma )$ and $U(w)=\eta w^{1-\gamma }/(1-\gamma )$, with $\gamma >0$, $\gamma \ne 1$, $\eta >0$, and $\delta \ge 0$. PutThe optimal plans from the marginal-utility equations are $c_t^{\ast }=(y{e}^{\delta t}{H}_t{)}^{-1/\gamma }$ and ${\xi }^{\ast }={\eta }^{1/\gamma }(y{e}^{\delta T}{H}_T{)}^{-1/\gamma }$. The normal distribution of $W_t$ gives, for $a=1-1/\gamma$,The budget therefore becomes $y^{-1/\gamma }b(T)=w_0$, determining the multiplier explicitly. Independent Brownian increments in the conditional budget formula give $w_t^{\ast }=c_t^{\ast }b(T-t)$. The diffusion coefficient of $c_t^{\ast }$ is $(\lambda /\gamma )c_t^{\ast }$, so matching the wealth diffusion determines the dollar stock holding. Consequently the finite-horizon power-utility investment and consumption optimum isAll candidate moments are finite on a finite horizon, since powers of a lognormal state-price density have finite moments. The budget and tangent argument above therefore verify optimality in this example, including admissibility and positive wealth.

For another verification of the formulas, the Hamilton-Jacobi-Bellman equation for a value function with discount measured from the current time isSubstitute $V(w,t)=b(T-t{)}^{\gamma }{w}^{1-\gamma }/(1-\gamma )$. The two concave maximizations give the displayed controls, and the remaining equation is $b^{\prime }(\tau )=1-kb(\tau )$ with $b(0)={\eta }^{1/\gamma }$, exactly solved by the formula above. ThusFor $k=0$, interpret the integral as $\tau$, so $b(\tau )=\tau +{\eta }^{1/\gamma }$. Unlike the infinite-horizon Merton consumption-investment problem, finite horizons do not require $k>0$. Consumption vanishes in the terminal-only formulation; then the integral term in $b$ is absent and the optimal stock fraction remains $(\mu -\rho )/(\gamma {\sigma }^2)$. Logarithmic utility is handled directly with inverse marginal utility $1/z$ and the same wealth-to-consumption formula at $\gamma =1$, with $k=\delta$.

Use $w$ for current wealth and $\pi =w-B$ for the dollar position in the risky asset. Its controlled dynamics have drift $rw+(\mu -r)\pi$ and diffusion coefficient $\sigma \pi$. Over a short time step $h$, the dynamic programming principle givesApply Itô formula to the second term, divide by $h$, and let $h\downarrow 0$. The Hamilton-Jacobi-Bellman equation isFor the increasing concave value function, $J_w>0$ and $J_{ww}<0$, so the expression is a strictly concave quadratic in the risky position. Its derivative vanishes atSubstitution of this maximizer yieldsThere is a running utility reward but no consumption withdrawal in the wealth dynamics. The required concavity is also verified explicitly by the positive coefficient in part (d), so this stationary point really is the maximum.

Interpret unit-intensity Gaussian white noise as $d{W}_t/dt$. Differentiating $z=x+(T-t)y$ cancels the two $ydt$ terms and givesThe cost is terminal $z(T{)}^2$ plus the same running $u^2$. Write $h=T-t$ and guess the value $V(t,z)=P(t)z^2+C(t)$. The Hamilton-Jacobi-Bellman equation isCompleting the square gives $u_{\ast }=-hPz$. Equating powers yields $P^{\prime }=h^2{P}^2$, $C^{\prime }=-h^{2}P$, with $P(T)=1,C(T)=0$. Integration givesIndeed $V_t+{\mathcal{L}}^{u}V+u^2=(u+hPz{)}^2\ge 0$, and the Itô verification identity attains equality at this control. This proves optimality for admissible controls of finite expected quadratic cost. The additive noise changes only $C$, not the feedback gain, so this is certainty-equivalence control. A different constant noise intensity multiplies $C$ but leaves the control unchanged, as in the terminal-position reduction of double-integrator control.

Write $A=\sigma {\sigma }^T$ and let $V(x,t)$ denote the optimal remaining expected reward from positive portfolio wealth $x$ at time $t$. The Hamilton-Jacobi-Bellman equation and terminal condition areThe controls are progressively measurable, the investment integrals exist, consumption is nonnegative and locally integrable, and the resulting portfolio wealth stays in the utility domain. At zero consumption use the right limit of the utility function if needed. This is the usual admissible trading strategy formulation; without restrictions preserving the domain the reward need not be defined.

For the verification bound, assume $V$ is a nonnegative classical solution, continuous through the terminal time, with one time derivative and two wealth derivatives in the interior. Fix any admissible controls and putThe Itô formula givesThe drift is nonpositive because the actual controls cannot exceed the supremum in the Hamilton-Jacobi-Bellman equation. Localize by stopping times on which the stochastic integral is a true martingale, the derivatives and wealth stay in their interior domain, and the accumulated reward is finite. For each such stop ${\tau }_n$,Both utilities and $V$ are nonnegative. Thus the Fatou lemma, continuity, and the terminal condition giveFor admissible paths strictly positive on the closed horizon, the domain-localizing stops eventually reach $T$. If zero wealth is allowed, the same proof uses the continuous boundary extension of $V$ and the utilities there. Nonnegativity is what permits removing the stops without assuming the uncontrolled stochastic term already has zero expectation. This is verification by a nonnegative control supermartingale.

The square-root case needs an additional no-arbitrage qualification because the printed volatility matrix need not be invertible. If there is $v\in \ker {\sigma }^T$ with $v^T\mu >0$, investment $\theta =Kv$ has zero volatility and arbitrarily large positive drift. With any fixed positive consumption, sufficiently large $K$ preserves positive wealth and makes terminal square-root utility tend to infinity. For example, in one dimension $\sigma =0$, $\mu =1$, $\theta =K$, $C=1$ gives $X_s=x+(K-1)(s-t)$, so the value is infinite when $T>t$. A finite separated solution does not exist for all matrices allowed by the printed hypotheses.

The needed condition is $\mu \in \mathrm{Range}A$, equivalent to $\mu$ being orthogonal to $\ker {\sigma }^T$, because $A$ is a symmetric matrix and $\ker A=\ker {\sigma }^T$. Assume this condition and let $B=A^{+}$ be its Moore-Penrose inverse; if $A$ is invertible, simply use $B=A^{-1}$. Put $q={\mu }^{T}B\mu \ge 0$. Seek the square-root investment-consumption value in the formThen $V_x=g/\sqrt{x}$ and $V_{xx}=-g/(2x^{3/2})$. Completing the quadratic investment expression, using $AB\mu =\mu$, shows that its maximum is attained at ${\theta }^{\ast }=2xB\mu$ and has value $qg\sqrt{x}$:Any investment component in $\ker A$ has no effect. For consumption, set $z=\sqrt{c}$. Its contribution is the concave quadratic $2z-(g/\sqrt{x})z^2$, whose maximum is $\sqrt{x}/g$ at $c^{\ast }=x/g^2$. Substituting both maxima into the Hamilton-Jacobi-Bellman equation givesWith $h=g^2$, this becomes $h^{\prime }+qh+1=0$, $h(T)=1$. Solving this linear ordinary differential equation givesIn particular $h\ge 1$, so $g=\sqrt{h}$ stays positive, and the two controls areThese controls are admissible and actually attain the value. Indeed their portfolio wealth satisfiesSince $\Vert 2{\mu }^{T}B\sigma {\Vert }^2=4q$, the explicit positive solution starting at $(x,t)$ isIts finite-horizon moments of every positive order are finite. More directly,Differentiating $g(s)\mathbb{E}\sqrt{X_s}$ and using the equation for $g$ gives $\frac{d}{ds}(g(s)\mathbb{E}\sqrt{X_s})=-\mathbb{E}\sqrt{X_s}/g(s)$. Integrating from $t$ to $T$ therefore yieldsThis proves attainment without relying on an unjustified expectation of a local martingale. When $\mu$ is outside $\mathrm{Range}A$, the singular-volatility drift arbitrage counterexample applies instead.

Put $\kappa =(\mu -r)/\sigma$, assuming $\sigma >0$, and letThis state-price density prices the bank account and the risky asset. The Itô formula givesNonnegative wealth and consumption make the deflated cumulative gains a nonnegative local martingale, hence a supermartingale. Every admissible consumption stream therefore satisfies the state-price budget constraint $\mathbb{E}{\int }_0^{\infty }{\xi }_t{c}_{t}dt\le w_0$.

Write $I=(U^{\prime }{)}^{-1}$ for the inverse marginal utility. The Inada conditions make $I$ decreasing from infinity to zero. A positive Lagrange multiplier $y$ for the budget gives the pointwise condition $e^{-\rho t}{U}^{\prime }(c_t)=y{\xi }_t$. Thus, whenever a finite optimum exists and a multiplier exhausts the budget,The concave supporting-tangent inequality, integrated against the budget, proves optimality. These curvature and endpoint assumptions alone do not ensure that the infinite-horizon value is finite; part (b) gives the exact additional condition for power utility.

The optimal wealth is the conditional price of remaining consumption:It is nonnegative. Apply the Martingale representation theorem to the integrable total discounted consumption claim:Comparing $M_t={\xi }_t{w}_t^{\ast }+{\int }_0^t{\xi }_s{c}_s^{\ast }ds$ with the deflated wealth equation yieldsThis constructs the portfolio without assuming extra smoothness of the value function. If the stationary value $V$ is twice differentiable with $V^{\prime \prime }<0$, optimization of its Hamilton-Jacobi-Bellman equation instead gives ${\theta }^{\ast }(w)=-(\mu -r)V^{\prime }(w)/({\sigma }^2{V}^{\prime \prime }(w))$ and $c^{\ast }(w)=I(V^{\prime }(w))$.

For constant relative risk aversion utility, $I(z)=z^{-1/R}$. Define the Merton consumption constantThe Gaussian exponential moment givesWhen ${\gamma }_M>0$, the budget is $w_0=y^{-1/R}/{\gamma }_M$. Conditional pricing then gives $c_t^{\ast }={\gamma }_M{w}_t^{\ast }$, and the Merton consumption-investment problem hasThe resulting wealth is a geometric Brownian motion, so positive initial wealth remains positive. Substitution in the Hamilton-Jacobi-Bellman equation verifies the first-order conditions and the value; the budget argument in part (a) supplies global optimality.

The problem has a finite value precisely when ${\gamma }_M>0$. For $0<R<1$ and ${\gamma }_M\le 0$, the supremum is $+\infty$. Indeed, use the displayed risky fraction and consumption $c_t=k{w}_t$, $k>0$. Its expected utility isFor ${\gamma }_M<0$, sufficiently small $k$ already gives infinite utility. For ${\gamma }_M=0$, the finite values tend to infinity as $k\downarrow 0$.

For $R>1$ and ${\gamma }_M\le 0$, every admissible stream has utility $-\infty$. To see this, put $q=(R-1)/R$ and apply Holder inequality on $[0,T]\times \Omega$:The left side diverges as $T\to \infty$, forcing the positive utility-cost integral to diverge. Thus there is no finite-value optimization problem in either ill-posed case.

Use $\kappa =(\mu -r)/\sigma$ and ${\gamma }_M$ from Question 1, and take the finite-value case ${\gamma }_M>0$. After retirement the value is $V_M(w)={\gamma }_M^{-R}{w}^{1-R}/(1-R)$. Working for an infinitesimal extra interval while using the retired portfolio and consumption produces a gain $\epsilon V_M^{\prime }(w)-\lambda$. Its zero isThis proves the instantaneous indifference level appearing in the question. For the stated irreversible optimal stopping problem it is generally a lower bound on the retirement boundary, rather than the boundary itself: retaining the opportunity to keep working has value. The following explicit wealth-variable Legendre dual calculation exhibits the correction. In particular, imposing continuity of $V^{\prime \prime }$ at retirement would discard that option value; smooth fit requires continuity of $V^{\prime }$.

Assume the usual borrowing-against-income solvency condition: $w>-\epsilon /r$ while working, with nonnegative wealth after retirement, and take $r>0$, $\rho >0$, $\kappa \ne 0$. The question does not specify a tighter borrowing constraint. Let $J(z)={\sup }_w\{V(w)-zw\}$, so $w=-J^{\prime }(z)$ and $V=J-z{J}^{\prime }$. The working Hamilton-Jacobi-Bellman equation becomes the linear Cauchy-Euler equationThe retired dual is $J_M(z)=R{z}^{1-1/R}/[(1-R){\gamma }_M]$. Let $m<0$ be the negative root ofThe other root exceeds one. Its growing term is excluded by the solvency asymptotic as $z\to \infty$. Therefore the working dual has the formRetirement corresponds to $z\le z_{\ast }$, where $J=J_M$. Value matching and smooth fit giveConsequently the retirement boundary with an income option isUsing the root equation, $z_{\ast }/(\lambda /\epsilon )=1+{\kappa }^{2}m/(2\rho )$, which lies strictly between zero and one. Thus $\overline{w}>w_{\mathrm{myopic}}$ for a nonzero risk premium. For example, $r=\rho =\epsilon =\lambda =\kappa =1$ and $R=2$ give $m=-1$, ${\gamma }_M=9/8$, andThese parameters give an explicit counterexample to identifying the printed level with the irreversible retirement boundary.

For $-\epsilon /r<w<\overline{w}$, find the unique $z>z_{\ast }$ fromThis right side decreases strictly from $\overline{w}$ to $-\epsilon /r$, since $J^{\prime \prime }>0$. Recover the value and controls byFor $w\ge \overline{w}$, retire immediately and use the Merton controls. Along the working strategy, $z_t$ satisfies $d{z}_t=(\rho -r)z_{t}dt-\kappa z_{t}d{W}_t$, and the optimal retirement time is its first hitting time of $z_{\ast }$, equivalently wealth's first hitting time of $\overline{w}$.

To check the optimal stopping inequalities, write $h=J-J_M$. It satisfies $h(z_{\ast })=h^{\prime }(z_{\ast })=0$ and $h^{\prime \prime }(z)=Dm(m-1)z^{m-2}>0$ for $z>z_{\ast }$. Hence continuation has positive option value there. On $z\le z_{\ast }$ the stopping candidate has working residual $\epsilon z-\lambda \le 0$. The matched dual is continuously differentiable, so the Itô formula introduces no boundary local-time term. Localization, the state-price budget, and the appropriate infinite-horizon transversality justify verification.

If nonnegative financial wealth is imposed during employment as an additional constraint, the working dual also has an upper boundary, and that extra constraint must be included in the free-boundary problem. It cannot be recovered by assuming the printed myopic equality.

Assume the regime is observed and ${\sigma }_i=\sigma (i)>0$. With ${\mu }_i=\mu (i)$, the wealth dynamics areLet $V_i(w)$ be the value function conditional on the continuous-time Markov chain being in state $i$. Its generator adds ${\sum }_j{q}_{ij}{V}_j(w)$ to the diffusion generator. The Hamilton-Jacobi-Bellman equation isFor $V_i^{\prime }>0$, $V_i^{\prime \prime }<0$, the maximizing controls are $c=I(V_i^{\prime })$ and $\theta =-({\mu }_i-r)V_i^{\prime }/({\sigma }_i^2{V}_i^{\prime \prime })$. Put ${\kappa }_i=({\mu }_i-r)/{\sigma }_i$. The optimized equation isHere $\hat{U}$ is the utility conjugate.

Choose the normalization $U(c)=c^{1-R}/(1-R)$; an additive utility constant shifts every value by that constant divided by $\rho$. Homogeneity gives $V_i(w)=A_i{w}^{1-R}/(1-R)$ with $A_i>0$. The regime-switching Merton equations reduce the differential system to one nonlinear algebraic equation per regime:The candidate feedback controls areChanging the regime changes consumption's wealth coefficient, while the risky fraction uses the current regime's risk premium and volatility.

Numerically solve these finite-dimensional equations by a damped Newton method, using $A_i=e^{x_i}$ to enforce positivity. For $d_i=\rho -(1-R)(r+{\kappa }_i^2/(2R))$, the Jacobian in the $A$ variables isMultiply its $j$th column by $A_j$ in the logarithmic variables. A residual-decreasing line search and continuation from $Q=0$, where the positive solution is the scalar Merton coefficient in each well-posed regime, give a practical method whenever that starting problem is finite. More generally use a positive initialization for the coupled system, and check residuals, positivity and conditioning rather than accepting an arbitrary algebraic root.

Verification requires constructing admissible feedback controls, proving the resulting wealth stays nonnegative, and establishing the stochastic-integral and transversality bounds needed at infinite horizon. Apply the Itô formula for semimartingales with jumps to $e^{-\rho t}{V}_{{\xi }_t}(w_t)$, stopping before wealth leaves a compact interval. The Hamilton-Jacobi-Bellman equation makes accumulated utility plus discounted candidate value a local supermartingale for every admissible policy and a local martingale under the feedback. Justify removal of localization and let the terminal discounted value vanish under the candidate. For $0<R<1$, the nonnegative terminal value can be dropped for the upper bound; for $R>1$ it is negative, so a transversality or dual integrability argument is essential. This establishes the global value bound and attainment, rather than just stationarity of the algebraic equations. The boundary is $V_i(0)=0$ for $R<1$ and the limiting value $-\infty$ for $R>1$.

Put $p=1-R$, $b=\mu -r1$, $A=\sigma {\sigma }^T$ and $q=R+\epsilon$. First assume the covariance matrix $A$ is invertible and initial portfolio wealth is positive. Introduce the accumulated fee processThe Itô formula and the self-financing portfolio equation giveThis converts the path-dependent payment into a terminal-utility problem with state $y$. The volatility-penalized terminal fee changes its drift as well as its spot volatility.