ZipDAS: Distributed Acoustic Sensing Data Compression
Installation
pip install -r requirements.txt
Usage
Compression using Wavelet (JPEG2000)
Compression
python run.py --mode compress --data_path noise_data --data_format h5 --result_path results/compressed_noise --method=jpeg --quality=10 --batch_nt 6000 --plot_figure
Decompression:
python run.py --mode decompress --data_path results/compressed_noise/jpeg --result_path results/decompressed_noise --method=jpeg --plot_figure
Compression using Neural Network models
To use GPU acceleration, please follow this instruction: https://www.tensorflow.org/install/pip
Compression and decompression:
python run.py --mode compress --data_path noise_data --data_format h5 --result_path results/compressed_noise --method=neural --model_path=model --batch_nt=6000 --plot_figure
python run.py --mode decompress --data_path results/compressed_noise/neural --result_path results/decompressed_noise --method=neural --model_path model --plot_figure
Training:
python train.py --model_path model --data_path noise_data --result_path training --format=h5
Test:
python train.py --mode test --model_path model --data_path tests/data --result_path tmp --format=h5 --batch 0 --nt 2048
Optional: Compression using Curvelet
python run.py --mode compress --data_path noise_data --data_format h5 --result_path results/compressed_noise --method=curvelet --keep_ratio 0.01 --batch_nt 6000 --plot_figure
python run.py --mode decompress --data_path results/compressed_noise/curvelet --result_path results/decompressed_noise --method=curvelet --plot_figure
Application: Ambient Noise
Raw data:
basename noise_data/* > noise_data.txt
python CCTorch/run.py --data_list1=noise_data.txt --data_path=noise_data --mode=AN --block_size1 1 --block_size2 10 --fixed_channels 100 300 500 700 900 --dt=0.02 --maxlag=15 --temporal_gradient --result_path results/cctorch_noise
python CCTorch/scripts/plot_ambient_noise.py --result_path results/cctorch_noise --figure_path results/cctorch_noise --fixed_channels 100 300 500 700 900
Compressed data:
METHOD=jpeg
python CCTorch/run.py --data_list1=noise_data.txt --data_path=results/decompressed_noise/$METHOD --mode=AN --block_size1 1 --block_size2 100 --fixed_channels 100 300 500 700 900 --dt=0.02 --maxlag=15 --result_path results/decompressed_noise/$METHOD/cctorch
python CCTorch/scripts/plot_ambient_noise.py --result_path results/decompressed_noise/$METHOD/cctorch --figure_path results/decompressed_noise/$METHOD/cctorch --fixed_channels 100 300 500 700 900
Optional: Install PyCurvelab for curvelet compression
wget http://www.fftw.org/fftw-2.1.5.tar.gz
tar -xvf fftw-2.1.5.tar.gz
cd fftw-2.1.5
./configure --with-pic --prefix=/home/weiqiang/.local/
make
make install
tar -xvf CurveLab-2.1.3.tar.gz
cd CurveLab-2.1.3
vi makefile.opt
make
make install
tar -xvf swig-4.1.1.tar.gz
cd swig-4.1.1
./configure --prefix=/home/weiqiang/.local/
make
make install
git clone https://github.com/slimgroup/PyCurvelab.git
cd PyCurvelab
export FDCT=/home/weiqiang/Research/DASCompression/tests/comp_curvelet/CurveLab-2.1.3/
export FFTW=/home/weiqiang/.local/
python setup.py build install
pip install PyWavelets