Data flow structure for multimodal human-robot collaboration in material handling

Material handling systems are challenges by ever increasing complexity due to customization, new product technologies, circularity requirements and increasing supply network interactions. In such a dynamic environment, proper use of robots can boost the efficiency of material handling systems. However, human-robot collaboration in material handling has not been in focus as a research agenda. Thus, it is not clear how the different requirements of human and robots from a data flow perspective should be addressed. In addition, human-robot collaboration in material handling should take into account some specific characteristics such as low visibility caused by constant movement for people, robots, and materials. This paper aims to take a step to cover this gap by proposing a framework that uses multimodal technologies to serve the purpose of safe and efficient human-robot collaboration in material handling. The proposed framework includes aspects such as human thoughts, robot assistance, real-time sensing, and human knowledge. A material handling scenario in a tested environment is presented for demonstration and verification purposes of the proposed framework. The results show that the proposed framework is beneficial to address the different requirements of human and robot in material handling. In addition, it is argued that efficient human-robot collaboration in material handling requires close attention to the monitoring of the work environment to ensure safe and efficient co-existence and collaboration.