An xAI Thick Data Assisted Caption Generation for Labeling Severity of Ulcerative Colitis Video Colonoscopy

Deep-learning convolutional neural networks (DCNNs) has made significant success in the area of medical image analysis and in particular in the area of colonoscopy. However, DCNNs are largely black-box predictors with no power to provide explanation for the underlying reasons of classification which is so important for evidence-based care practice. Providing thick data as additional heuristics in the form of generating relevant captions according to the features predicted by the machine learning can provide the explainable artificial intelligence (xAI) component to transfer those black boxes into more explainable components. This paper presented an approach that uses Siamese neural network for identifying the ulcerative colitis features from small training samples as well as to use an LSTM model to combine and embed relevant captions for providing that power of explainability to the Siamese classifier. Our modeling uses the Glove as an embedding model but did not use a copra that are dedicated for clinical practice. In our next research we are going to add this Glove based clinical copra to enhance our caption prediction accuracy.