ZipDAS: Distributed Acoustic Sensing Data Compression

Installation

pip install -r requirements.txt

Usage

Compression using Wavelet (JPEG2000)

Compression

python run.py --mode compress --data_path noise_data --data_format h5 --result_path results/compressed_noise --method=jpeg --quality=10 --batch_nt 6000 --plot_figure

Decompression:

python run.py --mode decompress --data_path results/compressed_noise/jpeg --result_path results/decompressed_noise --method=jpeg --plot_figure

Compression using Neural Network models

To use GPU acceleration, please follow this instruction: https://www.tensorflow.org/install/pip

Compression and decompression:

python run.py --mode compress --data_path noise_data --data_format h5 --result_path results/compressed_noise --method=neural  --model_path=model --batch_nt=6000  --plot_figure

python run.py --mode decompress --data_path results/compressed_noise/neural --result_path results/decompressed_noise --method=neural --model_path model --plot_figure

Training:

python train.py  --model_path model  --data_path noise_data --result_path training --format=h5

Test:

python train.py --mode test  --model_path model  --data_path tests/data --result_path tmp --format=h5 --batch 0 --nt 2048

Optional: Compression using Curvelet

python run.py --mode compress --data_path noise_data --data_format h5 --result_path results/compressed_noise --method=curvelet --keep_ratio 0.01 --batch_nt 6000 --plot_figure

python run.py --mode decompress --data_path results/compressed_noise/curvelet --result_path results/decompressed_noise --method=curvelet --plot_figure

Application: Ambient Noise

Raw data:

basename noise_data/* > noise_data.txt

python CCTorch/run.py --data_list1=noise_data.txt --data_path=noise_data  --mode=AN  --block_size1 1 --block_size2 10 --fixed_channels 100 300 500 700 900  --dt=0.02 --maxlag=15 --temporal_gradient --result_path results/cctorch_noise

python CCTorch/scripts/plot_ambient_noise.py --result_path results/cctorch_noise --figure_path results/cctorch_noise  --fixed_channels 100 300 500 700 900

Compressed data:

METHOD=jpeg
python CCTorch/run.py --data_list1=noise_data.txt --data_path=results/decompressed_noise/$METHOD  --mode=AN  --block_size1 1 --block_size2 100 --fixed_channels 100 300 500 700 900  --dt=0.02 --maxlag=15  --result_path results/decompressed_noise/$METHOD/cctorch

python CCTorch/scripts/plot_ambient_noise.py --result_path results/decompressed_noise/$METHOD/cctorch --figure_path results/decompressed_noise/$METHOD/cctorch  --fixed_channels 100 300 500 700 900

Optional: Install PyCurvelab for curvelet compression

FFTW 2.1.5

wget http://www.fftw.org/fftw-2.1.5.tar.gz
tar -xvf fftw-2.1.5.tar.gz
cd fftw-2.1.5
./configure --with-pic --prefix=/home/weiqiang/.local/
make
make install

CurveLab

tar -xvf CurveLab-2.1.3.tar.gz
cd CurveLab-2.1.3
vi makefile.opt
make
make install

SWIG

tar -xvf swig-4.1.1.tar.gz
cd swig-4.1.1
./configure --prefix=/home/weiqiang/.local/
make
make install

PyCurvelab

git clone https://github.com/slimgroup/PyCurvelab.git
cd PyCurvelab
export FDCT=/home/weiqiang/Research/DASCompression/tests/comp_curvelet/CurveLab-2.1.3/
export FFTW=/home/weiqiang/.local/
python setup.py build install

PyWavelets

pip install PyWavelets

AI4EPS/ZipDAS