Loading [MathJax]/extensions/MathZoom.js

Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey | IEEE Journals & Magazine | IEEE Xplore

- Donate
- Personal Sign In

ADVANCED SEARCH

Journals & Magazines >IEEE Transactions on Intellig... >Volume: 23 Issue: 1

Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey

Download PDF
Download References
Request Permissions
Save to
Alerts

Abstract:

Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e....Show More

Metadata

Abstract:

Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e.g., in transportation, robotics, IoT and power systems. Combining data-driven applications with transportation systems plays a key role in recent transportation applications. In this paper, the latest deep reinforcement learning (RL) based traffic control applications are surveyed. Specifically, traffic signal control (TSC) applications based on (deep) RL, which have been studied extensively in the literature, are discussed in detail. Different problem formulations, RL parameters, and simulation environments for TSC are discussed comprehensively. In the literature, there are also several autonomous driving applications studied with deep RL models. Our survey extensively summarizes existing works in this field by categorizing them with respect to application types, control models and studied algorithms. In the end, we discuss the challenges and open questions regarding deep RL-based transportation applications.

Published in: IEEE Transactions on Intelligent Transportation Systems ( Volume: 23, Issue: 1, January 2022)

Page(s): 11 - 32

Date of Publication: 22 July 2020

ISSN Information:

DOI: 10.1109/TITS.2020.3008612

Funding Agency:

References is not available for this document.

I. Introduction

With increasing urbanization and latest advances in autonomous technologies, transportation studies evolved to more intelligent systems, called intelligent transportation systems (ITS). Artificial intelligence (AI) tries to control systems with minimal human intervention. Combination of ITS and AI provides effective solutions for the 21st century transportation studies. The main goal of ITS is providing safe, effective and reliable transportation systems to participants. For this purpose, optimal traffic signal control (TSC), autonomous vehicle control, traffic flow control are some of the key research areas.

Select All

1.

R. Trevor, "INRIX global traffic scorecard", Feb. 2019.

2.

Z. Liu, "A survey of intelligence methods in urban traffic signal control", Int. J. Comput. Sci. Netw. Secur., vol. 7, no. 7, pp. 105-112, 2007.

3.

A. L. C. Bazzan, "Opportunities for multiagent systems and multiagent reinforcement learning in traffic control", Auto. Agents Multi-Agent Syst., vol. 18, no. 3, pp. 342, Sep. 2008.

CrossRef Google Scholar

4.

P. Mannion, J. Duggan and E. Howley, "An experimental review of reinforcement learning algorithms for adaptive traffic signal control" in Autonomic Road Transport Support Systems, Cham, Switzerland:Springer, pp. 47-66, 2016.

CrossRef Google Scholar

5.

K.-L. A. Yau, J. Qadir, H. L. Khoo, M. H. Ling and P. Komisarczuk, "A survey on reinforcement learning models and algorithms for traffic signal control", ACM Comput. Surv., vol. 50, no. 3, pp. 34, Oct. 2017.

CrossRef Google Scholar

6.

W. Tong, A. Hussain, W. X. Bo and S. Maharjan, "Artificial intelligence for vehicle-to-everything: A survey", IEEE Access, vol. 7, pp. 10823-10843, 2019.

7.

R. Abduljabbar, H. Dia, S. Liyanage and S. A. Bagloee, "Applications of artificial intelligence in transport: An overview", Sustainability, vol. 11, no. 1, pp. 189, Jan. 2019.

CrossRef Google Scholar

8.

H. Wei, G. Zheng, V. Gayah and Z. Li, "A survey on traffic signal control methods", arXiv:1904.08117, 2019, [online] Available: http://arxiv.org/abs/1904.08117.

9.

M. Veres and M. Moussa, "Deep learning for intelligent transportation systems: A survey of emerging trends", IEEE Trans. Intell. Transp. Syst., Jul. 2019.

10.

B. Ravi Kiran et al., "Deep reinforcement learning for autonomous driving: A survey", arXiv:2002.00444, 2020, [online] Available: http://arxiv.org/abs/2002.00444.

11.

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, Cambridge, MA, USA:MIT, 2018.

12.

C. J. Watkins and P. Dayan, "Q-learning", Mach. Learn., vol. 8, no. 3, pp. 279-292, 1992.

13.

G. A. Rummery and M. Niranjan, "On-line Q-learning using connectionist systems", vol. 37, 1994.

14.

L. M. Rios and N. V. Sahinidis, "Derivative-free optimization: A review of algorithms and comparison of software implementations", J. Global Optim., vol. 56, no. 3, pp. 1247-1293, Jul. 2013.

CrossRef Google Scholar

15.

R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning", Mach. Learn., vol. 8, no. 3, pp. 229-256, May 1992.

CrossRef Google Scholar

16.

L. C. Baird, "Reinforcement learning in continuous time: Advantage updating", Proc. IEEE Int. Conf. Neural Netw. (ICNN), vol. 4, pp. 2448-2453, Jun. 1994.

17.

L. Busoniu, R. Babuska and B. De Schutter, "Multi-agent reinforcement learning: A survey", Proc. 9th Int. Conf. Control Autom. Robot. Vis., pp. 1-6, 2006.

18.

Y. LeCun, Y. Bengio and G. Hinton, "Deep learning", Nature, vol. 521, no. 7553, pp. 436, 2015.

CrossRef Google Scholar

19.

V. Mnih et al., "Human-level control through deep reinforcement learning", Nature, vol. 518, no. 7540, pp. 529, 2015.

CrossRef Google Scholar

20.

T. P. Lillicrap et al., "Continuous control with deep reinforcement learning", arXiv:1509.02971, 2015, [online] Available: http://arxiv.org/abs/1509.02971.

21.

Z. Wang et al., "Sample efficient actor-critic with experience replay", arXiv:1611.01224, 2016, [online] Available: http://arxiv.org/abs/1611.01224.

22.

T. Schaul, J. Quan, I. Antonoglou and D. Silver, "Prioritized experience replay", arXiv:1511.05952, 2015, [online] Available: http://arxiv.org/abs/1511.05952.

23.

H. V. Hasselt, "Double Q-learning", Proc. Adv. Neural Inf. Process. Syst., pp. 2613-2621, 2010.

24.

H. V. Hasselt, "Double Q-learning", Proc. Adv. Neural Inf. Process. Syst., pp. 2613-2621, 2010.

25.

Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot and N. de Freitas, "Dueling network architectures for deep reinforcement learning", arXiv:1511.06581, 2015, [online] Available: http://arxiv.org/abs/1511.06581.

26.

B. O’Donoghue, R. Munos, K. Kavukcuoglu and V. Mnih, "Combining policy gradient and Q-learning", arXiv:1611.01626, 2016, [online] Available: http://arxiv.org/abs/1611.01626.

27.

V. Mnih et al., "Asynchronous methods for deep reinforcement learning", Proc. Int. Conf. Mach. Learn., pp. 1928-1937, 2016.

28.

S. S. Mousavi, M. Schukat and E. Howley, "Traffic light control using deep policy-gradient and value-function-based reinforcement learning", IET Intell. Transp. Syst., vol. 11, no. 7, pp. 417-423, Sep. 2017.

CrossRef Google Scholar

29.

D. Garg, M. Chli and G. Vogiatzis, "Deep reinforcement learning for autonomous traffic light control", Proc. 3rd IEEE Int. Conf. Intell. Transp. Eng. (ICITE), pp. 214-218, Sep. 2018.

30.

X. Liang, X. Du, G. Wang and Z. Han, "A deep reinforcement learning network for traffic light cycle control", IEEE Trans. Veh. Technol., vol. 68, no. 2, pp. 1243-1253, Feb. 2019.

References is not available for this document.