Performance Analysis of Higher Order Tensor Storages for Highly Sparse Multidimensional Data

To compute multidimensional data, tensor algebra is a powerful tool and it has many applications in machine learning and data analytics. This multidimensional data which is represented with higher-order tensors for multi-aspect data analysis is very sparse for an actual dataset. The degree of data sparsity increases with the increase of dimensionality. This is because the volume of the tensor increases very fast with the increase of dimensionality but the actual data to the tensor does not increase at such a rate for high-performance scientific computing. Storing and applying operations on this highly sparse multidimensional data is a challenge for data scientists. Therefore, an efficient storage scheme for such sparse tensor is very important. There are many storage schemes in the literature for storing sparse tensors. In this paper, we analyze and compare different tensor storage formats using synthetic and actual datasets. We found that no single storage scheme is efficient for all the operations or applications of tensors. We calculated the memory required for per-nonzero elements among these tensor formats and compared them for different parameters.