Diffusion Models for Image Generation to Enhance Health Literacy

Improving health literacy is critical for patient education, and a lack of health literacy has been shown to negatively affect both patients and the overall health care system. Text is the most common form of educational materials. In this paper, we explore how text information can be automatically augmented with images using a text-to-image model to assist in readability and comprehension. To understand what text types are amenable to image generation, we controlled for two text characteristics: concreteness (concrete vs. abstract) and length (sentence vs. noun phrase). We conducted a 2x2 study with N=80 medical prompts across all four conditions: sentence-concrete, sentence-abstract, noun-phrase-concrete, and noun-phrase-abstract. Experts evaluated the images on six metrics. Both the length and the concreteness were found to affect the quality of the generated images significantly. Overall, the Google Image Search results were better than the automatically generated results, highlighting the challenges of generative models.