Requirements Engineering Practices and Challenges in the Context of Big Data Software Development Projects: Early Insights from a Case Study

This paper reports on the results of an exploratory case study on a large-scale Big Data systems development project in the Oil&Gas domain within a non-profit organisation. The aim of this study was to investigate the RE practices and challenges in such projects, currently bereft in the scientific literature. This investigation was focused on: (a) RE practices; (b) sources and distribution of requirements; (c) the role of Big Data characteristics and technologies in RE and systems design; and (d) RE challenges in engineering Big Data Systems. The main results show that (a) there is a lack of specific project tailored RE practices, tools, and frameworks for elicitation, specification and modelling, analysis, and prioritisation of requirements; (b) 40% of the system's requirements are considered Big Data-related from which 75% are identified from internal sources; (c) Big Data characteristics and technologies play an important role in defining quality requirements and system's architecture; (d) five challenges in eliciting, documenting, and analysing Big Data related requirements were identified and discussed. The findings suggest academics and practitioners opportunities to engage in further research in this area.