User:Alexnally/Notable Contributions/Optimal Control Utility Example

Take a simplified version of the Ramsey–Cass–Koopmans model. We wish to maximize an agent's discounted lifetime utility achieved through consumption


 * $$max\int^\infty_0 e^{-\rho t}u(c(t)) dt$$

subject to the time evolution of capital per effective worker


 * $$\dot{k}=\frac{\partial k}{\partial t} =f(k(t)) - (n + \delta)k(t) - c(t)$$

where $$c(t)$$ is period t consumption, $$k(t)$$ is period t capital per worker, $$f(k(t))$$ is period t production, $$n$$ is the population growth rate, $$\delta$$ is the capital depreciation rate, the agent discounts future utility at rate $$\rho$$, with $$u'>0$$ and $$u''<0$$.

Here, $$k(t)$$ is the state variable which evolves according to the above equation, and $$c(t)$$ is the control variable. The Hamiltonian becomes


 * $$H(k,c,\mu,t)=e^{-\rho t}u(c(t))+\mu(t)\dot{k}=e^{-\rho t}u(c(t))+\mu(t)[f(k(t)) - (n + \delta)k(t) - c(t)]$$

The optimality conditions are


 * $$\frac{\partial H}{\partial c}=0 \Rightarrow

e^{-\rho t}u'(c)=\mu(t)$$
 * $$\frac{\partial H}{\partial k}=-\frac{\partial \mu}{\partial t}=-\dot{\mu} \Rightarrow \mu(t)[f'(k)-(n+\delta)]=-\dot{\mu}$$

If we let $$u(c)=ln(c)$$, then log-differentiating the first optimality condition with respect to $$t$$ yields

$$-\rho \frac{\dot{c}}{c(t)}=\frac{\dot{\mu}}{\mu(t)}$$

Setting this equal to the second optimality condition yields

$$\rho \frac{\dot{c}}{c(t)}=f'(k)-(n+\delta)$$

This is the Keynes–Ramsey rule or the Euler–Lagrange equation, which gives a condition for consumption in every period which, if followed, ensures maximum lifetime utility.