1B rides NYC taxi native benchmark
This repository reflects progress of performance and design study for 1B ride NYC taxi benchamarks.
Compilation
- Install and activate miniconda
- Create development environment:
conda env create -n arrow-def -f conda/default-env.yml
- Activate this environment:
conda activate arrow-def
- Configure:
cmake -GNinja -DCMAKE_BUILD_TYPE=release -S . -B ../build
- Build:
cmake --build ../build
- Allocate huge memory pages:
sudo bash -c 'echo 30000 > /proc/sys/vm/nr_hugepages'
- Download trips.xaa.csv.gz,
gzip -d
it - Run
../build/bench_all
for a single run of all execution nodes. - Adjust and run
bash run-aff-bsz.sh
for collecting times of CSV reader in different modes.