Plot2API: Recommending Graphic API from Plot via Semantic Parsing Guided Neural Network

Plot-based Graphic API recommendation (Plot2API) is an unstudied but meaningful issue, which has several important applications in the context of software engineering and data visualization, such as the plotting guidance of the beginner, graphic API correlation analysis, and code conversion for plotting. Plot2API is a very challenging task, since each plot is often associated with multiple APIs and the appearances of the graphics drawn by the same API can be extremely varied due to the different settings of the parameters. Additionally, the samples of different APIs also suffer from extremely imbalanced.Considering the lack of technologies in Plot2API, we present a novel deep multi-task learning approach named Semantic Parsing Guided Neural Network (SPGNN) which translates the Plot2API issue as a multi-label image classification and an image semantic parsing tasks for the solution. In SPGNN, the recently advanced Convolutional Neural Network (CNN) named EfficientNet is employed as the backbone network for API recommendation. Meanwhile, a semantic parsing module is complemented to exploit the semantic relevant visual information in feature learning and eliminate the appearance-relevant visual information which may confuse the visual-information-based API recommendation. Moreover, the recent data augmentation technique named random erasing is also applied for alleviating the imbalance of API categories.We collect plots with the graphic APIs used to drawn them from Stack Overflow, and release three new Plot2API datasets corresponding to the graphic APIs of R and Python programming languages for evaluating the effectiveness of Plot2API techniques. Extensive experimental results not only demonstrate the superiority of our method over the recent deep learning baselines but also show the practicability of our method in the recommendation of graphic APIs.

I. Introduction

Figures and plots are the indispensable tools for data visualization which provide people with intuitive understanding of data and interaction with data. In software engineering, almost all the programming languages support such functions and possess a series of relevant APIs as one of core libraries or packages. It is very common for the software developer particularly the beginner to search API on the web based on a case figure for guiding the plot. Figure 1 shows a help post where a developer asks how to draw a figure like the one posts in Stack Overflow. What’s more, people might want to know the APIs starting from a plot, such as imitating visualization styles. In agile development, developers often sufficiently utilize the materials of previous projects for speeding up the development, thereby they expect to convert the figures plotted in one language into APIs of the other directly to reduce the time cost. In these scenarios, a tool that can automatically recommend graphic APIs based on a plot can provide guidance for developers and improve their productivity. Therefore, how to identify API based on Plot (Plot2API) is a meaningful task in software engineering and data visualization.