/big-data-h5py

Showcase the handling of large files (larger than the RAM size) in Python using MPI4PY and H5PY

Primary LanguagePython

big-data-h5py

Showcase the handling of large files (larger than the RAM size) in Python using MPI4PY and H5PY

Contents

  1. mpi-h5py.py: Uses h5py to backup large datasets, to minimize memory footprint.
  2. mpi-numpy.py: Uses numpy arrays for the same task.