Deep Learning with Noise Data Augmentation and Detrended Fluctuation Analysis for Physical Action Classification by Brain-Computer Interface

Deep neural networks (DNNs) demonstrate their high potential for investigation of time sequences in many domains, including analysis of electroencephalography (EEG) signals gathered by brain-computer interface (BCI). This work is targeted on physical actions classification problem based on analysis of EEG data collected by BCI. The relatively small DNN with components of fully-connected networks (FCN) and convolutional neural network (CNN) was considered to classify physical activities (namely, hand manipulations) from the grasp-and-lift (GAL) dataset. The noise data augmentation (NDA) allowed us to estimate reliability of the results obtained in the view of the possible influence from environment. The detrended fluctuation analysis (DFA) was applied to investigate the fluctuation properties and calculate the correspondent Hurst exponents. The results of DNN training, validation and testing with and without NDA along with DFA demonstrated that some physical actions can be divided in separate groups of actions that can be characterized by complexity and the feasibility of their classification: the easiest (HandStart), medium (LiftOff, Replace, and BothReleased), and hardest (BothStartLoadPhase and FirstDigitTouch) classification. As far as the statistically reliable results were obtained by the relatively small DNN with the low resource requirements, this approach can be used for porting such models to Edge Computing infrastructures on devices with the very limited computational resources.

I. Introduction

Recently deep learning (DL) methods based on deep neural networks (DNNs) were effectively used for processing different data [1]. In health care and elderly care they become very popular for processing th very complex multimodal medical data [2], [3]. Usage of DL is especially important in the view of availability of various brain-computer interfaces (BCI) used for collection and analysis of electroencephalography (EEG) signals generated by brain activities [4]–[6]. In the context of air-space applications BCIs are intensively used for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks [7]–[9]. This work is targeted on investigation of EEG data collected by BCI to resolve classification problem for some physical activities (namely, hand manipulations) by the relatively simple DNN. The DNN was applied for analysis of preliminary (prior-activity), current (in-activity), and following (post-activity) parts of the relevant brain EEG signals. The paper has the following structure: section II.Background and Related Work contains a short outline of some similar attempts to investigate EEG by DNNs, section III.Methodology presents the dataset, physical activities (namely, hand manipulations), structure of DNN and metrics used, section VI.Experimental describes the results obtained, section V.Discussion gives the analysis of the methods used, and section VI.Conclusions proposes summary as to the further improvements.