Integrated Dynamic Control of Process and Cost in Complicated R&D Projects: A Reinforcement Learning Approach

This study addresses the integrated dynamic control problem of process and cost in an complicated R&D project based on earned value theory. It investigates optimal correlative efforts for correcting project's process and cost performance when confronted with performance deviations from the baseline at sequential checkpoints. The objective is to minimize the total costs, including actual and progress costs. To tackle this problem, we employ a stochastic Markov process to model the stochastic state transitions effected by the level of the efforts and exogenous random noises throughout the project's lifespan. We use Proximal Policy Optimization (PPO), a reinforcement learning technique, to identify a near-optimal control policy. Experimental results demonstrate the effectiveness of our proposed algorithm and managerial insights are derived from the results.