FDA: Between Process and Product Evaluation

Several institutions and agencies around the world, from both the private and public sectors, have adopted the practice of software validation and certification to ensure higher levels of confidence in software. The US Food and Drug Administration (FDA) is one of these organisations. The FDA has published several guidance documents concerning the validation of medical device software. In its guidance documents for both the medical software industry and FDA staff, FDA recommends certain activities to be undertaken and certain deliverables to be prepared. This is believed by the FDA to achieve higher confidence in the software quality and, accordingly, better validation. In this paper, we question the clarity and practicability of the FDA approach for medical software validation. We examine the FDA approach by evaluating the FDA guidance documents. We base our analysis on considering the product versus process evaluation criterion. By focusing on these two concepts, we believe that the FDA validation approach leads to confusion in the minds of both software producers and evaluators and it may well lead to lower quality software than desired being certified, at the time the FDA considers its approach as the least burdensome. As a demonstration of our claim, we analyze "quality planning" as the first activity in the FDA approach. We discuss the corresponding effort for this activity in another product-oriented evaluation approach, the common criteria (CC) for information technology security evaluation. As its main objective, the comparison aims to highlight both the inconsistency and the vagueness of the FDA evaluation function. Reconsideration of these issues by the FDA will lead to a more objective evaluation function and higher confidence in the medical software, which represents the most important concern among the evaluation community and the general public.