ChartGPT: Leveraging LLMs to Generate Charts From Abstract Natural Language

The use of natural language interfaces (NLIs) to create charts is becoming increasingly popular due to the intuitiveness of natural language interactions. One key challenge in this approach is to accurately capture user intents and transform them to proper chart specifications. This obstructs the wide use of NLI in chart generation, as users’ natural language inputs are generally abstract (i.e., ambiguous or under-specified), without a clear specification of visual encodings. Recently, pre-trained large language models (LLMs) have exhibited superior performance in understanding and generating natural language, demonstrating great potential for downstream tasks. Inspired by this major trend, we propose ChartGPT, generating charts from abstract natural language inputs. However, LLMs are struggling to address complex logic problems. To enable the model to accurately specify the complex parameters and perform operations in chart generation, we decompose the generation process into a step-by-step reasoning pipeline, so that the model only needs to reason a single and specific sub-task during each run. Moreover, LLMs are pre-trained on general datasets, which might be biased for the task of chart generation. To provide adequate visualization knowledge, we create a dataset consisting of abstract utterances and charts and improve model performance through fine-tuning. We further design an interactive interface for ChartGPT that allows users to check and modify the intermediate outputs of each step. The effectiveness of the proposed system is evaluated through quantitative evaluations and a user study.

I. Introduction

Natural language interfaces (NLIs) have become a popular interactive strategy for data analysis and visualization creation [1], [2]. For example, a user can easily create a histogram showing the distribution of IMDB ratings for a movie dataset by simply saying “create a histogram showing the distribution of IMDB ratings.” Compared to traditional methods, NLIs provide a shortcut for analysts not proficient in visualization programming, such as D3.js or Vega-Lite [3], to create visualizations. Even for senior visualization users, NLIs can free them from tedious programming issues or interactive editings on visualization toolkits (e.g., Tableau [4]).