Evaluation criteria for closed-loop adaptive dynamic discrete-continuous brain-computer interfaces: clinical study case with tetraplegic patient

Quantifying and evaluating properly the performances is a critical issue in BCI experiments. The choice of the most adapted metrics can be difficult because they are specific to the experimental paradigm, task control, and data. In the current study evaluation criteria for closed-loop adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces are examined. The challenges such as imbalanced multi-class bias for discrete decoding (classification), online computing cost and dynamic analysis of results in time are considered. There are two main objectives of the study: identifying the best suited performances metrics according to the requirements and combining several levels of evaluation in the whole BCI system (decoder and patient oriented). The main novelty of the study is the combination of several common metrics with new temporal metrics and a time dynamic approach which allows adequately reflect the performance of adaptive dynamic and hierarchical discrete-continuous brain-computer interfaces BCI systems. Additional response time and blocking error patterns reveal complementary information for BCI system performance evaluation. Developed criteria are applied to performance evaluation of 8-dimensional exoskeleton control by tetraplegic patient for both decoder and user-centered metrics.