Kubernetes Container Orchestration as a Framework for Flexible and Effective Scientific Data Analysis

In this paper we present the design and deployment details for Single Particle Imaging (SPI) experiments data analysis pipeline in Kubernetes infrastructure. We have analyzed various software usage patterns for different payloads. Components of the pipeline software include traditional HPC (MPI-based) applications, applications which require GPU computations, GUI-based software and the software which can be parallelized naturally by dividing the data into several independent parts (Data parallelism). For every payload type individual deployment approach was proposed.