Accelerating Time to Science using CRADLE: A Framework for Materials Data Science

Modern materials science research problems present a challenge to data science and analytics as experiments generate Petabyte-scale spatiotemporal datasets that span a number of modalities and formats. Creating computing infrastructure and frameworks that support the scale and diversity of materials science data while remaining accessible for materials scientists to use is a non-trivial task. We have developed the Common Research Analytics and Data Lifecycle Environment (CRADLE) to solve the challenges of materials data science through a scalable research computing framework and cyber infrastructure that can (1) handle large-scale, heterogeneous datasets (2) provide a flexible toolbox for building machine learning pipelines that span from ingestion to model deployment (3) be accessible to research scientists with limited to extensive computational backgrounds and (4) utilize a myriad of low performance to high performance computer systems. CRADLE is a framework that integrates distributed systems like Hadoop and High-Performance Computing (HPC) infras-tructure to handle materials data at scale. This all enables the general materials data scientist to query Petabytes of data and train thousands of models in a parallel, distributed environment. We demonstrate three use cases for CRADLE to benchmark its capability to ingest and analyze spatiotemporal materials data at scale. These tasks span three data modalities: transforming 2.6 billion Photovoltaic time-series power measurements, training hundreds of deep learning models on Atomic Force Microscopy images, and ingesting 27 billion geospatial data points. CRADLE exemplifies an overarching framework that accelerates time to science, extends to other domains with similar challenges, and expands the horizon of data science and research.