CrisisFlow: Multimodal Representation Learning Workflow for Crisis Computing

An increasing number of people use social media (SM) platforms like Twitter and Instagram to report critical emergencies or disaster events. Multimodal data shared on these platforms often contain useful information about the scale of the event, victims, and infrastructure damage. The data can provide local authorities and humanitarian organizations with a big-picture understanding of the emergency (situational awareness). Moreover, it can be used to effectively and timely plan relief responses. In our project, we aim to address the challenge of finding relevant information among the vast amount of published SM posts. Specifically, we use deep learning algorithms to produce embeddings that encode the informativeness of multimodal SM data in the context of disaster events. Our method improves the state-of-the-art performance on the informative vs. non-informative classification task for the CrisisMMD dataset. To ensure the reliability and scalability of our solution in real-world scenarios, we implement the resulting crisis computing workflow in the Pegasus Workflow Management System (WMS).