Self-Assembling Modular Robotic Training Using Concurrent Dual Reinforcement Learning

Mobile robots operating in harsh and inaccessible environments, such as space, the battlefield, or disaster scenarios, can be limited by an inability to adapt to unforeseen challenges, damage, or component failure. Modular, self-assembling robots composed of many homogenous robotic modules, which together form a single robot, is one method of alleviating this limitation. A robot composed of these modules has inherent redundancy, as each module can contain all necessary elements for operation, and as each module is identical, a module failure can be corrected by the remaining modules by collectively and autonomously replacing the failed module with a spare working module. Coordinating the movement, actions, and communications between multiple, potentially thousands, of modules remains an ongoing research challenge. With the objective of alleviating some of these challenges, in this paper we propose reinforcement learning as a viable method of developing algorithms for this robotic modular coordination. Here, we propose and present a platform of magnetically connected, hexagonal disk modules, which are not capable of independent operation, but collectively are capable of movement and other actions through multiple successive self-reconfigurations. We then propose a dual concurrent reinforcement learning algorithm that demonstrates these modules can learn to successfully self-reconfigure for movement while concurrently learning to navigate through an environment, searching for a goal.