Visual Decoding of Phrases from Occipital Neuromagnetic Signals

Orthographic visual perception (reading) is encoded via a widespread dynamic interaction between different language centers of the brain and visual cortex. In this study, we investigated orthographic visual perception decoding with Magnetoencephalography (MEG), where phrases were visually presented to participants. We compared the decoding performance obtained with sensors within the occipital lobe that obtained with sensors covering the whole head. Two naive machine learning classifiers namely support vector machines (SVM) and linear discriminant analysis (LDA) were used. Experimental results indicated that the decoding performance using only occipital sensors is similar to the performance obtained with all sensors within the task period, which were all above chance level. In addition, temporal analysis by taking short-time windows showed that the occipital sensors were more discriminative near onset compared to later time periods, while using the whole head sensor setup at later time periods performed slightly better than occipital sensors. This finding may indicate a sequential order (from visual cortex to other areas beyond occipital lobe) during visual speech perception.

I. Introduction

Visual perception is a neural mechanism that allows the brain to create patterns of activity to receive, interpret, and act upon visual stimuli through a series of transformations of neural signals [1]. Orthographic visual perception is a “mid-level vision” process that acts as a central interface between visual and linguistic processing during reading [2]. The role of the occipital cortex, specifically ventral occipital-temporal cortex (vOT) is found to be crucial by several functional magnetic resonance imaging (fMRI) studies [3]–[6]. In regards to recognition of orthographic perception, several studies have attempted to explore pattern analysis using electroencephalography (EEG) [7], [8], fMRI [9], [10], electrocorticography (ECoG) [11], and magnetoencephalography (MEG) [12]. Very recently, beyond classification, research studies have even attempted for neural visual image reconstruction [13], [14]. These studies have focused on word-based stimuli which constrains the complete understanding of visual lexico-semantic information that can be generated with a complete sentence/phrase.