A Preliminary Fault Taxonomy for Multi-tenant SaaS Systems

Multi-tenancy is the key feature for every Software as a Service (SaaS), as it enables multiple customers, so-called tenants, to transparently share a system's resources reducing costs. Tenants can customize a system according to their particular needs, however, such a high level of complexity may open possibilities for a failure. In addition, there is a lack of a reference architecture for such applications and once the implementations differ significantly, ensuring that all executions flows have been verified without impacting the working features for other tenants is a complex task. The clear understanding of the possible faults is fundamental for the identification, tolerance and definition of appropriate testing techniques. This paper presents a preliminary fault taxonomy for multi-tenant cloud applications considering their foundational features. A literature review previously carried out, a survey with practitioners and analysis of some applications were performed to achieve this classification. In addition, an e-commerce called MtShop was developed for a case study. The expressiveness of the proposed taxonomy is illustrated with critical faults identified in the MtShop through the automated and parallel testing. We conclude with the benefits that our taxonomy can bring to testing, prediction and regression testing activity of multi-tenant cloud applications.