Traffic signal control with macroscopic fundamental diagrams

The recent breakthrough finding of macroscopic fundamental diagram (MFD) establishes the foundation of macroscopic analysis in urban transportation studies. However, the implementation of MFD for traffic signal control remains challenging. This is because the compact network-wide information provided by MFD is insufficient for searching for the optimal microscopic control policy. In this paper, rather than implementing only MFD, we integrate MFD into our microscopic urban traffic flow model to constrain the searching space of control policies. This approach is able to maximize the contribution of MFD, without losing microscopic information in the control model. Specifically, we first build a traffic flow model and introduce the stochastic driver behaviors by a turning matrix. We then implement the approximate Q-learning with restricted control to reduce the computational cost of the large-scale stochastic control problem. Here, the information of MFD is used to design both the heuristic regularization term in the stage cost and the statebased feature vector of the approximate Q-function. By this approximate Q-learning algorithm, the traffic density distribution of the network tends to become homogenous, with the mean value around the optimal density of the MFD. The numerical experiments demonstrate that compared to a fixed policy, our policy could efficiently make a heterogeneous network more homogeneous, and thus guarantee a more robust shape of the MFD. Furthermore, our policy has a better performance on trip completion flow maximization compared to either a fixed or a greedy policy, since it can achieve the optimal density in the MFD.

I. Interoduction

Traffic flow optimization and signal control in a large-scale urban network is a well investigated yet challenging problem in transportation studies, due to the complex spatio-temporal correlations in network traffic. Like that of image and signal processing, a traffic flow represents different characteristics when observed at different scales. Usually, when the scale gets larger, the observation is less stochastic, carrying information of more general system-level patterns. For example, the traffic flow can be modeled as either discrete vehicles at real-time observation scale or continuous flows at an hourly or daily observation scale. As for traffic signal control, to achieve accurate result, we should represent the evolution of dynamic traffic flows at each intersection by micro-modeling simulations, such as the probabilistic queue model [1], or an operational macro-scopic model [2]. However, the microscopic modeling suffers severe computational cost due to the curse of dimensionality. On the other hand, recent studies reveal that if we measure the traffic at network scale (using space-mean) instead of intersection scale, a MFD exists between the traffic density and the network outflow if the density is distributed homogeneously over the network. The theoretical physical model of MFD was proposed early [3], but the supporting evidence was found recently in a real-world experiment [4].