Automatic Clinical Report Generation of Thyroid Scintigraphy using Natural Language Processing and Bayesian Convolutional Neural Network

There are several pathological diseases affecting the thyroid gland, such as thyroiditis, goiter, thyrotoxicosis also malignant and benign nodules. One of the well-established examinations in early diagnosis, differentiation and treatment follow up is thyroid scintigraphy performed using radioactive compounds. Generating reports documenting the diagnosis process commonly performed by nuclear medicine (NM) physicians is usually laborious and time-consuming and can be accelerated by the aid of machine learning algorithms. The main purpose of this study was developing a method to generate clinical reports using planar thyroid images in order to save time as an aid to physicians. In this study, we included 268 adult cases referred for thyroid imaging. The report generated by a NM physician with more than 15 years of experience. Each data consisted of 3 images of a same person which is described by two sections, one complete description and the other is a shorter format in each of which the diagnosis is determined. Using the latter (the shorter), we first, standardize texts of the reports, secondly, we omit the stop words i.e. prepositions and also words occurring fewer than 10 times in the reports. Then, by utilizing stems of the remainder of words, we assumed 23 classes of reports (each containing words like nodular goiter, left lobe, normal, etc.). We took advantages of single words as well as bigrams and trigrams in generating different classes. Furthermore, word collocation probabilities used to re-arrange generated words. The neural network applied in this research was Bayesian Convolutional Neural Network (BCNN) using 75, 10 and 15 percent of the data for train, evaluation and test phases, respectively. The BCNN with two convolutional and two pooling and three fully-connected layers was set to train in 200 epochs using Log-Soft-Max to calculate the probability of each class. In the final step, classes with higher probabilities are given to a text generator using collocations. We evaluated model by mean accuracy where the existence of a class (single word, bigram and trigram) in the reference sentence illustrates the more accurate of the method.The results of this strategy show that the average accuracy for the test data is 74.3%. We developed an automated method to generate reports in human language and automatic diagnosis by means of machine learning.