Space-efficient sparse matrix storage formats for massively parallel systems

Authors: Šimeček I., Langr D.,


Keywords
Sparse matrix format; space complexity.


Abstract
In this project, we propose and evaluate new storage formats for sparse matrices that minimize the space complexity of information about matrix structure. The motivation of our work are applications with very large sparse matrices that due to their size must be processed on massively parallel computer systems consisting of tens or hundreds of thousands of processor cores and that must be stored in a distributed file system using parallel I/O. The parallel I/O is typically the main performance bottleneck and reading or writing such matrices from/to distributed file system can take significant amount of time. We try to reduce this time by reducing the amount of data to be processed.


What's new
February 1, 2012 -  version 0.5 of this project is released (EBF format).
May 1, 2012 -  version 0.7 of this project is released (AHF format).
October 1, 2012 -  version 0.9 of this project is released (BHF format).

Known bugs


(Possible) future works

Used in papers
Adaptive-Blocking Hierarchical Storage Format for Sparse Matrices
Space-efficient sparse matrix storage formats for massively parallel systems



Parameters and output:
Download:
Space efficient formats sizes overview 0.9

BACK