Abstract:
Mobile Edge Computing (MEC) is one of the most promising techniques for next-generation wireless communication systems. In this paper, we study the problem of dynamic cac...Show MoreMetadata
Abstract:
Mobile Edge Computing (MEC) is one of the most promising techniques for next-generation wireless communication systems. In this paper, we study the problem of dynamic caching, computation offloading, and resource allocation in cache-assisted multi-user MEC systems with stochastic task arrivals. There are multiple computationally intensive tasks in the system, and each Mobile User (MU) needs to execute a task either locally or remotely in one or more MEC servers by offloading the task data. Popular tasks can be cached in MEC servers to avoid duplicates in offloading. The cached contents can be either obtained through user offloading, fetched from a remote cloud, or fetched from another MEC server. The objective is to minimize the long-term average of a cost function, which is defined as a weighted sum of energy consumption, delay, and cache contents' fetching costs. The weighting coefficients associated with the different metrics in the objective function can be adjusted to balance the tradeoff among them. The optimum design is performed with respect to four decision parameters: whether to cache a given task, whether to offload a given uncached task, how much transmission power should be used during offloading, and how much MEC resources to be allocated for executing a task. We propose to solve the problems by developing a dynamic scheduling policy based on Deep Reinforcement Learning (DRL) with the Deep Deterministic Policy Gradient (DDPG) method. A new decentralized DDPG algorithm is developed to obtain the optimum designs for multi-cell MEC systems by leveraging on the cooperations among neighboring MEC servers. Simulation results demonstrate that the proposed algorithm outperforms other existing strategies, such as Deep Q-Network (DQN).
Published in: Intelligent and Converged Networks ( Volume: 1, Issue: 2, September 2020)
Citations are not available for this document.
Cites in Papers - |
Cites in Papers - IEEE (81)
Select All
1.
Ritabrata Maiti, A. S. Madhukumar, Tan Zheng Hui Ernest, "MACU: A Multiagent Cache Updating Framework for IIoT Networks", IEEE Internet of Things Journal, vol.12, no.5, pp.5219-5232, 2025.
2.
Shivani Tripathi, Praveen Kumar, Mailram Sai Chaitanya, Nagireddy Sai Tarun Teja, Rajiv Misra, T.N. Singh, "Dueling Double DQN with Attention for Optimized Offloading in Wireless-Powered Edge-Enabled Mobile Computing Networks", 2024 IEEE International Conference on Big Data (BigData), pp.4420-4428, 2024.
3.
Raja A, Prathibhavani P M, Venugopal K R, "Combined-Jelly-Snake optimization with Deep Learning Architecture for Task offloading in Edge Computing", 2024 Global Conference on Communications and Information Technologies (GCCIT), pp.1-6, 2024.
4.
Min Zhang, Yanxiang Jiang, Fu-Chun Zheng, Dongming Wang, Mehdi Bennis, Abbas Jamalipour, Xiaohu You, "Communication-Efficient Federated Deep Reinforcement Learning Based Cooperative Edge Caching in Fog Radio Access Networks", IEEE Transactions on Wireless Communications, vol.23, no.12, pp.18409-18422, 2024.
5.
Youjia Chen, Yuyang Zheng, Jian Xu, Hanyu Lin, Peng Cheng, Ming Ding, Xi Wang, Jinsong Hu, Haifeng Zheng, "Knowledge-Assisted Resource Allocation With Domain Adversarial Neural Networks", IEEE Transactions on Network and Service Management, vol.21, no.6, pp.6493-6504, 2024.
6.
Megha Sharma, Abhinav Tomar, Abhishek Hazra, Zaid Akhter, Daksh Dhangar, Rahul Kumar Singh, "Edge Device Selection For Industrial IoT Task Offloading In Mobile Edge Computing", 2024 IEEE International Conference on Omni-layer Intelligent Systems (COINS), pp.1-4, 2024.
7.
Geng Chen, Jingli Sun, Yuxiang Zhou, Qingtian Zeng, Fei Shen, "A Novel Proactive Cache Decision Algorithm Based on Prior Knowledge and Aerial Cloud Assistance in Internet of Vehicles", IEEE Transactions on Network Science and Engineering, vol.11, no.6, pp.5280-5297, 2024.
8.
Yulong Zhang, Dirk Kutscher, Ying Cui, "Networked Metaverse Systems: Foundations, Gaps, Research Directions", IEEE Open Journal of the Communications Society, vol.5, pp.5488-5539, 2024.
9.
Ashvin Srinivasan, Mohsen Amidzadeh, Junshan Zhang, Olav Tirkkonen, "Adaptive Cache Policy Optimization Through Deep Reinforcement Learning in Dynamic Cellular Networks", Intelligent and Converged Networks, vol.5, no.2, pp.81-99, 2024.
10.
Aman Saurav, Biswadip Bandyopadhyay, Pratyay Kuila, Mahesh Chandra Govil, "DQN-based Multi-user Multi-task Offloading in Mobile Edge Computing", 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), pp.1-6, 2024.
11.
Ming Yan, Meiqi Luo, Chien Aun Chan, André F. Gygax, Chunguo Li, Chih-Lin I, "Energy-Efficient Content Fetching Strategies in Cache-Enabled D2D Networks via an Actor-Critic Reinforcement Learning Structure", IEEE Transactions on Vehicular Technology, vol.73, no.11, pp.17485-17495, 2024.
12.
Sicong Liu, Younan Mou, Hong Zhang, "Sparsity-Aware Channel Estimation for Underwater Acoustic Wireless Networks: A Generative Adversarial Network Enabled Approach", 2024 International Wireless Communications and Mobile Computing (IWCMC), pp.1171-1176, 2024.
13.
Changmao Wu, Zhengwei Xu, Xiaoming He, Qi Lou, Yuanyuan Xia, Shuman Huang, "Proactive Caching With Distributed Deep Reinforcement Learning in 6G Cloud-Edge Collaboration Computing", IEEE Transactions on Parallel and Distributed Systems, vol.35, no.8, pp.1387-1399, 2024.
14.
Gabriele Proietti Mattia, Roberto Beraldi, "Online Decentralized Scheduling in Fog Computing for Smart Cities Based on Reinforcement Learning", IEEE Transactions on Cognitive Communications and Networking, vol.10, no.4, pp.1551-1565, 2024.
15.
Ming Yan, Litong Zhang, Wei Jiang, Chien Aun Chan, André F. Gygax, Ampalavanapillai Nirmalathas, "Energy Consumption Modeling and Optimization of UAV-Assisted MEC Networks Using Deep Reinforcement Learning", IEEE Sensors Journal, vol.24, no.8, pp.13629-13639, 2024.
16.
Seonghoon Yoo, Seongah Jeong, Jeongbin Kim, Joonhyuk Kang, "Cache-Assisted Mobile-Edge Computing Over Space–Air–Ground Integrated Networks for Extended Reality Applications", IEEE Internet of Things Journal, vol.11, no.10, pp.18306-18319, 2024.
17.
James Adu Ansere, Eric Gyamfi, Vishal Sharma, Hyundong Shin, Octavia A. Dobre, Trung Q. Duong, "Quantum Deep Reinforcement Learning for Dynamic Resource Allocation in Mobile Edge Computing-Based IoT Systems", IEEE Transactions on Wireless Communications, vol.23, no.6, pp.6221-6233, 2024.
18.
Langtian Qin, Hancheng Lu, Yao Lu, Chenwu Zhang, Feng Wu, "Joint Optimization of Base Station Clustering and Service Caching in User-Centric MEC", IEEE Transactions on Mobile Computing, vol.23, no.5, pp.6455-6469, 2024.
19.
Zhikai Liu, Navneet Garg, Tharmalingam Ratnarajah, "Multi-Agent Federated Reinforcement Learning Strategy for Mobile Virtual Reality Delivery Networks", IEEE Transactions on Network Science and Engineering, vol.11, no.1, pp.100-114, 2024.
20.
Liqing Liu, Zhichao Chen, "Joint Optimization of Multiuser Computation Offloading and Wireless-Caching Resource Allocation With Linearly Related Requests in Vehicular Edge Computing System", IEEE Internet of Things Journal, vol.11, no.1, pp.1534-1547, 2024.
21.
Ziya Chen, Qian Ma, Lin Gao, Xu Chen, "Price Competition in Multi-Server Edge Computing Networks Under SAA and SIQ Models", IEEE Transactions on Mobile Computing, vol.23, no.1, pp.754-768, 2024.
22.
Minh K. Quan, Dinh C. Nguyen, Van-Dinh Nguyen, Mayuri Wijayasundara, Sujeeva Setunge, Pubudu N. Pathirana, "HierSFL: Local Differential Privacy-Aided Split Federated Learning in Mobile Edge Computing", 2023 IEEE Virtual Conference on Communications (VCC), pp.103-108, 2023.
23.
Sumit Kumar, Yajnaseni Dash, "Caching Techniques in Edge Computing and Challenges", 2023 3rd International Conference on Advancement in Electronics & Communication Engineering (AECE), pp.360-367, 2023.
24.
Zejie Li, Ke Yu, Hao Zhou, Xiaofei Wu, "DQN-Based Collaborative Computation Offloading for Edge Load Balancing", 2023 8th IEEE International Conference on Network Intelligence and Digital Content (IC-NIDC), pp.01-06, 2023.
25.
Chenyi Yang, Xiaolong Xu, Muhammad Bilal, Yiping Wen, Tao Huang, "Deep-Deterministic-Policy-Gradient-Based Task Offloading With Optimized K-Means in Edge-Computing-Enabled IoMT Cyber-Physical Systems", IEEE Systems Journal, vol.17, no.4, pp.5195-5206, 2023.
26.
Yu-Ting Zhang, Jing-Yu Yang, Yang Wu, "Dynamic Resource Allocation Strategy of Multi-Objective Fuzzy Optimization Based on Markov Decision Process", IEEE Access, vol.11, pp.99607-99613, 2023.
27.
Wenliang Cheng, Yueqiang Xu, Quansheng Xu, Heli Zhang, Xi Li, Xun Shao, "CSFRL: A Reinforcement Learning Technology Enabled Computing Power Scheduling Framework Based on Kubernetes", 2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pp.1-6, 2023.
28.
Zhixiu Yao, Shichao Xia, Yun Li, Guangfu Wu, "Cooperative Task Offloading and Service Caching for Digital Twin Edge Networks: A Graph Attention Multi-Agent Reinforcement Learning Approach", IEEE Journal on Selected Areas in Communications, vol.41, no.11, pp.3401-3413, 2023.
29.
Yiping Yuan, Xin Chen, Libo Jiao, Yijie Wang, "Deep Reinforcement Learning-based Computing Offloading and Resource Allocation Algorithm in Internet of Vehicles", 2023 IEEE 11th International Conference on Information, Communication and Networks (ICICN), pp.30-36, 2023.
30.
S. Kumari, Sanjay V, Sanjay Preeth D, Sathish Kumar G, "Maximize Computational Offloading Optimization Multi-User Multi-Access Fog Computing Wireless Networks", 2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN), pp.1059-1064, 2023.
Cites in Papers - Other Publishers (68)
1.
Chunlin Li, Yong Zhang, Long Yu, Kun Jiang, Youlong Luo, Shaohua Wan, "DRL-based Content Caching Strategy With Efficient User Preference Predictions in UAV-assisted VEC", ACM Transactions on Sensor Networks, 2024.
2.
Yiran Lu, Chi Xu, Yitian Wang, "Joint Computation Offloading and Trajectory Optimization for Edge Computing UAV: A KNN-DDPG Algorithm", Drones, vol.8, no.10, pp.564, 2024.
3.
Shudong Wang, Zhi Lu, Haiyuan Gui, Xiao He, Shengzhe Zhao, Zixuan Fan, Yanxiang Zhang, Shanchen Pang, "DDQN-based online computation offloading and application caching for dynamic edge computing service management", Ad Hoc Networks, pp.103681, 2024.
4.
Shan Yin, Lihao Liu, Mengru Cai, Yutong Chai, Yurong Jiao, Zheng Duan, Yian Li, Shanguo Huang, "DNN distributed inference offloading scheme based on transfer reinforcement learning in metro optical networks", Journal of Optical Communications and Networking, vol.16, no.9, pp.852-867, 2024.
5.
Xu Gao, Junhui Zhao, Qingmiao Zhang, Haitao Han, "Optimization of resource allocation strategy for high-speed railway based on deep reinforcement learning", Physical Communication, pp.102455, 2024.
6.
Tiantian Li, Bingchuan He, Huangfei Cheng, Bin Cao, "A Novel MEC Framework for Extractive Summarization Using Semantic Role Graph and Semantic Matching", Artificial Intelligence and Machine Learning, vol.2058, pp.16, 2024.
7.
Hongzhi Li, Lin Tang, Shengwei Chen, Libin Zheng, Shaohong Zhong, "AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning", Electronics, vol.13, no.6, pp.1104, 2024.
8.
Alok Choudhury, Manojit Ghose, Akhirul Islam, Yogita, "Machine learning-based computation offloading in multi-access edge computing: A survey", Journal of Systems Architecture, pp.103090, 2024.
9.
Peizhe Ma, Saurabh Garg, Mutaz Barika, "Research allocation in mobile volunteer computing system: Taxonomy, challenges and future work", Future Generation Computer Systems, vol.154, pp.251, 2024.
10.
Xintong Zhu, Zongpu Jia, Xiaoyan Pang, Shan Zhao, "Joint Optimization of Task Caching and Computation Offloading for Multiuser Multitasking in Mobile Edge Computing", Electronics, vol.13, no.2, pp.389, 2024.
11.
Aristeidis Karras, Christos Karras, Ioannis Karydis, Markos Avlonitis, Spyros Sioutas, "An Adaptive, Energy-Efficient DRL-Based and\xa0MCMC-Based Caching Strategy for\xa0IoT Systems", Algorithmic Aspects of Cloud Computing, vol.14053, pp.66, 2024.
12.
Zeinab Zabihi, Amir Masoud Eftekhari Moghadam, Mohammad Hossein Rezvani, "Reinforcement Learning Methods for Computation Offloading: A Systematic Review", ACM Computing Surveys, vol.56, no.1, pp.1, 2024.
13.
Adlin Sheeba, B. Uma Maheswari, "An efficient fault tolerance scheme based enhanced firefly optimization for virtual machine placement in cloud computing", Concurrency and Computation: Practice and Experience, 2023.
14.
Mohsen Khani, Mohammad Mohsen Sadr, Shahram Jamali, "Deep reinforcement learning‐based resource allocation in multi‐access edge computing", Concurrency and Computation: Practice and Experience, 2023.
15.
Ze Wei, Rongxi He, Yunuo Li, Chengzhi Song, "DRL-Based Computation Offloading and Resource Allocation in Green MEC-Enabled Maritime-IoT Networks", Electronics, vol.12, no.24, pp.4967, 2023.
16.
Qiuying Shen, Wentao Zhang, Mofei Song, "RankXGB-Based Enterprise Credit Scoring by Electricity Consumption in Edge Computing Environment", Computers, Materials & Continua, vol.75, no.1, pp.197, 2023.
17.
Xiaohu Gao, Mei Choo Ang, Sara A. Althubiti, "Deep Reinforcement Learning and Markov Decision Problem for Task Offloading in Mobile Edge Computing", Journal of Grid Computing, vol.21, no.4, 2023.
18.
Xiaodan Gu, Zhihan Zhang, Jinghui Zhang, Liqun Zhu, Fang Dong, "Simulation based QoS aware dynamic caching scheme for heterogeneous content requests in vehicular edge computing", Journal of King Saud University - Computer and Information Sciences, pp.101813, 2023.
19.
Zewu Li, Chen Xu, Zhanpeng Zhang, Runze Wu, "Deep reinforcement learning based trajectory design and resource allocation for task-aware multi-UAV enabled MEC networks", Computer Communications, 2023.
20.
Zhengyi Chai, Haole Hou, Yalun Li, "A dynamic queuing model based distributed task offloading algorithm using deep reinforcement learning in mobile edge computing", Applied Intelligence, 2023.
21.
Sun Yi, Zhang Xiao, , 2023.
22.
Jinghui Zhang, Weilong Xin, Dingyang Lv, Jiawei Wang, Guangxing Cai, Fang Dong, "Multi-exit DNN inference acceleration for intelligent terminal with heterogeneous processors", Sustainable Computing: Informatics and Systems, pp.100906, 2023.
23.
Zhiyang Zhang, Die Wu, Fengli Zhang, Ruijin Wang, "DECCo-A Dynamic Task Scheduling Framework for Heterogeneous Drone Edge Cluster", Drones, vol.7, no.8, pp.513, 2023.
24.
Udayakumar K, Ramamoorthy S, , 2023.
25.
Erana Veerappa Dinesh Subramaniam, Valarmathi Krishnasamy, "Hybrid Optimal Ensemble SVM Forest Classifier for Task Offloading in Mobile Cloud Computing", The Computer Journal, 2023.
26.
Xuehua Li, Jiuchuan Zhang, Chunyu Pan, "Federated Deep Reinforcement Learning for Energy-Efficient Edge Computing Offloading and Resource Allocation in Industrial Internet", Applied Sciences, vol.13, no.11, pp.6708, 2023.
27.
Jixun Gao, Bingyi Hu, Jialei Liu, Huaichen Wang, Quanzhen Huang, Yuanyuan Zhao, "Overbooking-Enabled Task Scheduling and Resource Allocation in Mobile Edge Computing Environments", Intelligent Automation & Soft Computing, vol.37, no.1, pp.1, 2023.
28.
Jiawen CHU, Chunyun PAN, Yafei WANG, Xiang YUN, Xuehua LI, "Edge Computing Resource Allocation Algorithm for NB-IoT Based on Deep Reinforcement Learning", IEICE Transactions on Communications, vol.E106.B, no.5, pp.439, 2023.
29.
Min Yang, Chengmin Ge, Xiaoran Zhao, Huaizhen Kou, "FSPLO: a fast sensor placement location optimization method for cloud-aided inspection of smart buildings", Journal of Cloud Computing, vol.12, no.1, 2023.
30.
Xinyi Qing, Baopeng Ye, Yuanquan Shi, Tao Li, Yuling Chen, Lei Liu, "Lightweight Storage Framework for Blockchain-Enabled Internet of Things Under Cloud Computing", Computers, Materials & Continua, vol.75, no.2, pp.3607, 2023.