vibhatha/pyspark_substrait

pyspark_substrait

Pre-requisites

Create conda environment

conda env create -f environment.yaml

cd pyspark_substrait
git submodule sync --recursive && git submodule update --init --recursive
cd substrait-python && \
    chmod u+x gen_proto.sh && ./gen_proto.sh && \
    pip install .

Developer Mode

cd pyspark_substrait
pip install -e .