A Reinforcement Learning-based Iterated Local Search for Software Modularization

Modularization algorithms are used to recover software structure. These algorithms divide the source code artifacts of the program into smaller and more understandable modules. Since software modularization is an NP-hard problem, heuristics/meta-heuristics methods are usually used to solve this problem. Iterated Local Search (ILS) is a meta-heuristic having two key components, namely a local search method and a perturbation method. Most current ILS models are formulated manually by selecting these components among the different combinations. This paper presents a modified ILS to solve the software modularization problem whose components are selected intelligently by utilizing a reinforcement learning (RL) technique, i.e., ILS is designed automatically at each iteration. More specifically, Q-learning is used to guide the proposed method in selecting the appropriate components in each search state. To demonstrate the applicability of the proposed algorithm, eleven software systems are selected. The experimental results show that the proposed method achieves higher or equal quality modularizations compared with other techniques.