Software Architecture Evaluation of a Machine Learning Enabled System: A Case Study

Machine learning components are increasingly being included in the so-called machine learning enabled software systems. Machine learning and software engineering communities have agreed in that it is necessary to consider the particular aspects and issues that arise from the development and deployment of such systems, especially when related to software architecture. In relation to software architecture, architecture evaluation is a relatively mature area which is seen as inextricable with any architecture design. However, we believe machine learning and software engineering communities have not given enough attention to the evaluation of software architectures of machine learning enabled software. We propose in this paper a set of aspects that should be attended in order to enact a machine learning enabled software architecture evaluation such as the diversity of stakeholders’ knowledge. We designed, planned, run and report a case study in which the case was the evaluation of a white matter brain lesions segmentation machine learning enabled system using the Decison-Centric Architecture Review (DCAR) method. In this paper we report some of the decisions that were reviewed and changed, along with interesting insights regarding factors that influence software architecture evaluation adoption in this topic. We conclude and suggest in this paper that both machine learning and software architecture research and practitioner communities should start considering the study of software architecture evaluations in the context of machine learning enabled software architectures.