Improving 3-D Zebrafish Tracking With Multiview Data Fusion and Global Association

Zebrafish behavioral patterns reveal valuable insights for biomedical research. To accurately identify these patterns, visual tracking systems need to reconstruct 3-D trajectories from multiview video sequences. However, 3-D zebrafish tracking faces challenges such as the dynamics in movements, the similarity in appearances, and the distortion caused by different viewpoints. In this article, we propose a new method for robust 3-D zebrafish trajectory reconstruction based on multiview data fusion and global association. Our method generates reliable segments of 2-D/3-D trajectories, called
tracklets
, where we consider short-term cues of appearance similarity and motion consistency and propose corresponding scoring metrics. Moreover, we use a
lazy-reconstruction
strategy to enhance the overall accuracy of 3-D trajectories by taking into account the global context. Extensive experiments on the public 3D-ZeF20 dataset demonstrate the effectiveness of the proposed method, achieving 67.9% multiple object tracking accuracy (MOTA), 64.3% ID F1 Score (IDF1), and 55.0 MTBFm.