MMTS: Multi-Modal Time Series Based Decision Support System for Ventilator Associated Pneumonia

Ventilator-associated pneumonia (VAP) stands out as the predominant nosocomial pneumonia among critically ill patients and poses a significant threat to morbidity and mortality in intensive care units (ICUs). The timely identification of individuals susceptible to VAP allows for early intervention, thereby enhancing patient outcomes. We have proposed a multi-modal time series model to facilitate the early prediction of VAP. This model predicts a patient’s VAP condition early, by examining the previous day’s chest X-rays, Clinical and Micro Biological Analysis(CMBA), and the patient’s medical history, enabling early detection before the actual diagnosis. Our approach involves a two-stage architecture. In the first stage, we train two modality-specific models independently for chest X-rays and CMBA respectively. These encoders are then frozen, and in the second stage, a sequence model is trained on the embeddings of previous days’ chest X-rays and CMBA, integrated in the temporal domain, extracted from the encoders from stage 1, to capture patterns and predict the next day’s VAP condition. Our model achieves an AUROC of 89.2 percent. Notably, our model benefits from an increased number of previous day data points for Early VAP diagnosis.