An Organ-Aware Diagnosis Framework for Radiology Report Generation

Radiology report generation (RRG) is crucial to save the valuable time of radiologists in drafting the report, therefore increasing their work efficiency. Compared to typical methods that directly transfer image captioning technologies to RRG, our approach incorporates organ-wise priors into the report generation. Specifically, in this paper, we propose Organ-aware Diagnosis (OaD) to generate diagnostic reports containing descriptions of each physiological organ. During training, we first develop a task distillation (TD) module to extract organ-level descriptions from reports. We then introduce an organ-aware report generation module that, for one thing, provides a specific description for each organ, and for another, simulates clinical situations to provide short descriptions for normal cases. Furthermore, we design an auto-balance mask loss to ensure balanced training for normal/abnormal descriptions and various organs simultaneously. Being intuitively reasonable and practically simple, our OaD outperforms SOTA alternatives by large margins on commonly used IU-Xray and MIMIC-CXR datasets, as evidenced by a 3.4% BLEU-1 improvement on MIMIC-CXR and 2.0% BLEU-2 improvement on IU-Xray.

I. Introduction

Imaging Techniques such as X-ray, computed tomography (CT), and magnetic resonance imaging (MRI) have become crucial and widely used tools in modern disease diagnosis. During a working day, a radiologist is typically assigned hundreds of radiology images, each of which requires the radiologist to check abnormal organs and write a radiology report. This leads to laborious, time-consuming, and tedious works [1], [2]. In order to alleviate the workload for radiologists and increase the efficiency of diagnosing diseases, automatic radiology report generation has emerged as an attractive potential technology [3], [4], [5], [6] to provide radiologists with assistant information for timely and objective diagnosis.