Trajectory Optimization with Discrete Decisions

This paper proposes a method of incorporating discrete decision making capabilities into an optimal control problem using differential dynamic programming (DDP). First proposed and described in the 1960s, DDP is an indirect method that relies on a Taylor series expansion of the loss function in the neighborhood of some optimal trajectory, and ideally exhibits quadratic convergence. Although DDP is not innately suited to problems having discrete solutions, the work described in this paper shows a straightforward, feasible means of accomplishing this goal without modifying the core DDP algorithm itself. Simulation results suggest that DDP can be made to choose between multiple discrete goals at a particular decision step. The capability to dynamically (and optimally) assign the steps at which these decisions occur is also demonstrated.