2D to 3D Segmentation: Inclusion of Prior Information using Random Walk Kalman Filters

Augmented reality is a quickly advancing field that has the potential to provide surgeons with computer generated diagnostic results during surgery. Visual classification of diseased tissue generated during a diagnostic procedure, for example, trans-urethral cystoscopy of the urinary bladder, can aid a surgeon during the following resection to ensure no tissue is inadvertently missed. Work with 2D segmentation of camera images is well developed and frameworks already exist to fuse this data real-time in a 3D reconstruction. These existing frame-works, however, maintain only the most recent segmentation information when building the 3D reconstruction. This work proposes a method to build a 3D point cloud classification using random walk Kalman filters. The method enables retention of prior classification information and additionally provides a framework to include additional sensor classifications contributing to a single, final 3D segmentation result. The method is demonstrated using a simulated environment intended to emulate the inside of a human bladder.