Computer Vision-Aided Beamforming for 6G Wireless Communications: Dataset and Training Perspective

Recent progress of deep learning (DL) and computer vision (CV) have paved the way for the application of DL-based CV technologies in 6G wireless communications. DL-based CV is data-hungry, and thus it is important to collect a massive vision dataset designed for wireless applications. An aim of this paper is to propose a vision dataset called Vision Objects for Millimeter and Terahertz Communications (VOMTC) consisting of 20,232 pairs of RGB and depth images, each of which is manually annotated with three object classes (person, mobile, and laptop) and their corresponding boxes. To demonstrate the efficacy of VOMTC, we develop a CV-aided beamforming technique called VOMTC-based beam management (VBM). In VBM, the location of the mobile is extracted via the VOMTC-trained object detector and then a beam heading toward the extracted location is generated. Due to the use of this special object detector tailored for identifying mobiles, VBM enhances the chance of transmitting directional beams to the mobile location. Using the VOMTC test dataset, we show that VBM achieves 15% improvement in the data rate over the conventional CV-aided beam management.