Real-time management model for frequent Big Data errors : Automatic Clean Repository For Big Data (ACR)

In this paper we present a solution that manages in real time the frequent errors of big data flows. In this model, we propose a repository in which the metadata, errors, and cleaning correction algorithms are stored for a given domain. In the first step, the system is supervised by an advisor that estimates the corresponding errors cleaning algorithm for a given context, according to the obtained results. At the second step, the system, thanks to its learning module, becomes autonomous in the selection algorithm procedure. To allow this capability, the proposed approach is designed and built on the basis of the Strategy Pattern. This pattern brings the possibility of building a family of algorithms, encapsulate each one, make them interchangeable and allow them to evolve independently of used context.