Dynamic Optimization
Variational Methods, Optimal Control, and Dynamic Programming — A Rigorous Reconstruction
Self-contained, fully-derived English lecture notes on dynamic optimization, reconstructed from Zhang Bo’s (张博) graduate course. From static optimization and the qualitative theory of ODEs, through the calculus of variations and Pontryagin’s optimal control, to Bellman/HJB dynamic programming — with economic applications (Ramsey, Solow, Merton) worked in full.
Preface
These are self-contained lecture notes on dynamic optimization — the mathematics of choosing an entire path rather than a single point. They reconstruct, in English and from first principles, a graduate course taught by Zhang Bo (张博), whose original Chinese courseware supplies the structure, the examples, and the economic motivation.
“Reconstruct” is meant literally. Every theorem is stated with its hypotheses and proved; every sign is argued, not asserted; every value or asset equation is derived — typically from a discrete period-\(\Delta\) problem taken to the \(\Delta\to 0\) limit — rather than written down as given. Where the course is terse, these notes are long; the goal is a reference you can read closed-book and reproduce.
The arc of the subject
Dynamic optimization is three methods for one problem — maximize an objective over a path subject to a law of motion — each with its own central tool:
| Method | Central object | Tool |
|---|---|---|
| Calculus of variations | the path \(x(t)\) | the Euler equation |
| Optimal control | a control \(u(t)\) steering the state | Pontryagin’s maximum principle |
| Dynamic programming | the value function \(V\) | the Bellman / HJB equation |
The three are not rivals but views of the same summit: the costate \(\lambda\) of optimal control is the gradient \(V_x\) of the value function, and the maximum principle and the HJB equation imply one another. A recurring aim of these notes is to make those bridges explicit.
How the notes are organized
The book is built bottom-up, so that the dynamic theory rests on solid static and dynamical foundations.
- Foundations. The Introduction frames the optimization problem and the three methods. Static Optimization develops convexity, the KKT conditions, the envelope theorem, and duality. ODEs covers solution methods and — more importantly for us — the qualitative theory: stability, phase diagrams, and bifurcations.
- Continuous-time dynamic optimization. The Calculus of Variations and Optimal Control handle continuous-time paths, with the Ramsey growth model as the running economic example.
- Dynamic programming. Dynamic Programming treats discrete and continuous time, deterministic and stochastic, up to the Itô calculus and the Merton portfolio problem.
Each chapter ends with the course’s problem set, translated in full, with worked solutions to the most instructive problems. Supplementary courseware (LICQ, the brachistochrone sufficiency proof, MRAP, the verification theorems, random-horizon DP) is folded into the relevant chapter.
A word on notation
The notes use standard field notation, which occasionally departs from the courseware — most consequentially, the continuous-time discount rate \(\rho\) is kept distinct from the discrete-time discount factor \(\beta=1/(1+\rho)\), which the course writes with a single symbol. The Notation chapter is the master table and flags every such deviation. The reasoning behind the notation policy and the choice of Quarto for these notes is recorded in docs/adr/.
These notes are a study reconstruction, not Zhang Bo’s official course materials; any errors introduced in the rebuilding are mine.