Fast Software-Based Real Time Panoramic Image Processing

Panoramic or stitched image processing has wide applications in areas such as medical imaging, topographical mapping, and deep space exploration. Rapid development of high-speed communication and artificial intelligence technologies have enabled real-time panoramic image processing, essential to autonomous driving, robotics, drones, etc., and critical in the advancement of smart cities, smart hospitals, manufacturer automation, and intelligent military warfare. Image stitching algorithms are able to join sets of images together and provide a wider field of a vision when compared with an image from a single standard camera. Traditional techniques are able to adequately produce a stitch for a static set of images, but suffer when differing lighting conditions exist between the two images, and from processing times too slow for real time use cases. We propose a solution which resolves these two issues encountered by traditional techniques. First, two advanced blending schemes, including the superpixel approach, have been implemented to resolve the lighting difference. Second, we develop a fast validation scheme to rapidly detect invalid solutions, and package all the system components in a parallel processing architecture to ensure a 0.008 second frame processing time, sufficiently small for most real-time applications. The proposed solution, including its fault-detection system, is implemented in the software level and the need for specialized hardware is completely eliminated. To the best of our knowledge, this is the first software-based solution fast enough for real-time panoramic image processing; it would contribute significantly to the advancement of real-time image processing, especially for its wide applications in the modern smart society.