JARVIS2: a data compressor for large genome sequences

Reducing data storage and its associated costs is one of the major bottlenecks in large-scale genomic projects. In this paper, we present a new method for efficient reference-free data compression of genomic sequences, using a mixture of multiple finite-context models (FCMs) and weighted local stochastic repeat models (WLSRMs). In the WLSRMs, we developed a new cache-hash memory model for increasing the compression ratio, while decreasing the RAM. The mixture is provided by a new architecture, that includes a neural network, followed by arithmetic encoding. The method is implemented using the C language as the JARVIS2 tool. Additionally, we provide a C/Bash implementation for using JARVIS2 in FASTA data. The results show an improvement in the compression of two extensive datasets, while using lower computational resources relative to the second best compression ratio tool. JARVIS2 is freely available at https://github.com/cobilab/jarvis2.