Human Vs. Machine Eye for Chart Interpretation

Recent advances in machine learning have enabled the creation of models that can read, parse, and to a certain extent, reason about charts. These models are yet to reach up to human interpretation, but given the fast progressive technologies, these models can analyze, interpret and visualize chart images to some extent. Many people are acquainted with data visualization paradigms; still, estimating a users' capability to read complex charts or graphs is not easy. For designing an interpretation and visualization system for the non-specialist user, it becomes necessary to separate the potential efficiency of the system and the comprehension capability of the user. Complex data interpretation requires extraction, analysis, and manipulation of the data repeatedly, with the influence of multiple perceptions leveraging multiple interpretation tools. Moreover, interpreting charts requires the extraction of textual and graphical encodings from chart images. With the complex layout, the retrieval system fails to interpret data relationships. The paper discusses human and machine perception for understanding charts, and the societal need for chart interpretation. Visually impaired users rely more on summaries to understand images. However, neither images are described by the assistive readers, nor chart images are accompanied by any proper contextual messages. To overcome this issue, the paper also proposes a style-invariant automated encoding system that uses inter and intra-relationship of chart elements to interpret data encodings from various types of chart images.

I. Introduction

The influence of scientific chart visualization is due to the solidity of the machine and human perception. Visualizers, machines, or humans are mainly interested in the meaning of the image. Visualizers contemplate ‘how’ and ‘what’ is understandable from an image or data. Structured or unstructured data interpretation initializes the visualization process [1]. It becomes necessary to perpetuate the mythologies and dependent parameters while evolving with complex data. With evolving data visualization, i.e., scientific or non-scientific charts, visualizers need to comprehend and extract overlapping and explicit data for better interpretation. The human brain has the supreme intelligence to understand and summarize scenes, while machines need images and additional intelligence to per-form the same task over iterative learning. The interpretation of chart images by humans and machine is different, i.e., humans can grasp hidden meaning easily, but machines can interpret charts differently [2], i.e., it might not distinguish between a pie chart and a pie-in-a-donut chart discriminately.