Big Data Storage and Analysis System for Space Application

In 2022, China successfully completed the construction of its space station, which boasts a multitude of payload devices within its space application system. These payloads operate year-round, generating various types of time-series data, including both status data and scientific experiment data. However, due to the diverse nature of scientific experiments, the payload data packet sizes and sampling frequencies can dynamically change. Traditional time-series databases designed for the Internet of Things (IoT) cannot fully meet the storage and analysis requirements for this payload data. To address this challenge, this paper presents the development of a storage and analysis system tailored to space application big data by modifying the existing Cassandra time-series database. By intercepting write requests with dynamic partitioning, the system stores data in both an index table and a data entity table, allowing it to adapt to the dynamic characteristics of the data. Additionally, a hierarchical binary tree aggregation algorithm is applied to the incoming data to support the system's aggregation and analysis needs. Experimental validation and task verification have confirmed the effectiveness of the proposed solution and algorithm.