Investigating Human Machine Integration Concepts for Isar Classification

Critical to the development of airborne human-machine surveillance systems is understanding how they perform individually and how to optimally integrate them. We present a case study that explores the integration of automatic classification technology with human judgement for maritime surveillance using inverse synthetic aperture radar (ISAR). We have conducted this study by combining methods from human factors research and constructive simulation. Accordingly, we first performed a forced choice reaction time experiment to objectively quantify human ISAR classification performance which revealed important differences in performance as a function of vessel type. One of the results we present is the asymmetrical confusion matrices that describe the accuracy of observer classification performance. By reviewing current automatic ISAR vessel classification performance, we used algorithms to produce machine classification confusion matrices and subsequently compared them with human performance. Machine performance generally outperformed humans on the common ISAR images used in both studies. Finally, we considered how both human and machine classification capabilities (as measured via their corresponding confusion matrices) are best integrated to result in more accurate performance while maintaining human control for decisions. We investigated two models for combining human and machine classification and tested them using a discrete event simulation experiment. Our findings suggest that the optimal combination depends on the search priorities, for timeliness it is when automatic machine classification is used to pre-filter targets by their Perceptual Class, leaving the Gross class to be classified using human judgement. If accuracy is the dominant priority then automation should be applied for the Gross Naval classification.