Getting error while running schemachange docker on gitlab cicd
binitbhaskar opened this issue · 1 comments
Describe the bug
While running the docker command in gitlab cicd to run schemachange, we are getting error.
Please find the gitlab stage details below
deploy-to-eu-staging-snowflake:
stage: snowflake-staging
when: manual
image: docker:latest
services:
- docker:dind
environment:
name: staging
before_script:
- cat contents/A__insert_data.sql
script:
- >
docker run --rm
--name schemachange-script
-v "$PWD":/usr/src/schemachange
-w /usr/src/schemachange
-e SNOWFLAKE_USER
-e SNOWFLAKE_ROLE
-e SNOWFLAKE_PRIVATE_KEY_PATH
-e SNOWFLAKE_PRIVATE_KEY
-e SNOWFLAKE_AUTHENTICATOR
python:3 /bin/bash -c "export OPENBLAS_NUM_THREADS=1 && pip install schemachange --upgrade --progress-bar off --debug && schemachange -f contents/ -a SNOWFLAKE_URL -u $SNOWFLAKE_USER -r $SNOWFLAKE_ROLE -w TEST_WH -d TEST_DB -c TEST_DB.SCHEMACHANGE.CHANGE_HISTORY"
Steps to reproduce the behavior:
- Go to 'gitlab pi'
- Click on 'Run pipeline'
- Scroll down to 'log'
- See error
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
Schemachange (please complete the following information):
- Version (e.g. 3.5.3): 3.6.0-py3
Additional context
Please find the details logs in the gitlab cicd
$ docker run --rm --name schemachange-script -v "$PWD":/usr/src/schemachange -w /usr/src/schemachange -e SNOWFLAKE_USER -e SNOWFLAKE_ROLE -e SNOWFLAKE_PRIVATE_KEY_PATH -e SNOWFLAKE_PRIVATE_KEY -e SNOWFLAKE_AUTHENTICATOR python:3 /bin/bash -c "export OPENBLAS_NUM_THREADS=1 && pip install schemachange --upgrade --progress-bar off --debug && schemachange -f contents/ -a SNOWFLAKE_URL -u $SNOWFLAKE_USER -r $SNOWFLAKE_ROLE -w TEST_WH -d TEST_DB -c TEST_DB.SCHEMACHANGE.CHANGE_HISTORY"
Unable to find image 'python:3' locally
3: Pulling from library/python
8457fd5474e7: Pulling fs layer
13baa2029dde: Pulling fs layer
325c5bf4c2f2: Pulling fs layer
7e18a660069f: Pulling fs layer
98a59f0ffede: Pulling fs layer
72c7f17f2221: Pulling fs layer
2f40b346325a: Pulling fs layer
f3f08e04e337: Pulling fs layer
Digest: sha256:7b8d65a924f596eb65306214f559253c468336bcae09fd575429774563460caf
Status: Downloaded newer image for python:3
Collecting schemachange
Obtaining dependency information for schemachange from https://files.pythonhosted.org/packages/91/36/828c2caac9933af76893e3f73fe5e63f0b100e49bf1aa52c48160ceb87fd/schemachange-3.6.0-py3-none-any.whl.metadata
Downloading schemachange-3.6.0-py3-none-any.whl.metadata (33 kB)
Collecting jinja2~=3.0 (from schemachange)
Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting pandas~=1.3 (from schemachange)
Downloading pandas-1.5.3.tar.gz (5.2 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: still running...
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pyyaml~=6.0 (from schemachange)
Obtaining dependency information for pyyaml~=6.0 from https://files.pythonhosted.org/packages/b4/33/720548182ffa8344418126017aa1d4ab4aeec9a2275f04ce3f3573d8ace8/PyYAML-6.0.1-cp[31](https://gitlab.com/sinch/sinch-projects/enterprise-and-messaging/beehive/finance/rating/genesis/financial-data-pipeline-selfserve/-/jobs/5489018214#L31)2-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting snowflake-connector-python<4.0,>=2.8 (from schemachange)
Downloading snowflake-connector-python-3.4.0.tar.gz (719 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [331 lines of output]
Collecting setuptools>=40.6.0
Obtaining dependency information for setuptools>=40.6.0 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/fa/7f/4c07234086edbce4a0a446209dc0cb08a19bb206a3ea53b2f56a403f983b/wheel-0.41.3-py3-none-any.whl.metadata
Using cached wheel-0.41.3-py3-none-any.whl.metadata (2.2 kB)
Collecting cython
Obtaining dependency information for cython from https://files.pythonhosted.org/packages/0a/f5/a629b0decba74c6cd1f9e830d1e55b89b6be15507d26c7cc14bc87275b28/Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Downloading Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
Collecting pyarrow<10.1.0,>=10.0.1
Downloading pyarrow-10.0.1.tar.gz (994 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 994.1/994.1 kB 15.4 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting numpy>=1.16.6 (from pyarrow<10.1.0,>=10.0.1)
Obtaining dependency information for numpy>=1.16.6 from https://files.pythonhosted.org/packages/07/c0/ccbb2a4c75b283d6100400a907087bfa4d89cee9df73fa6af85268115d81/numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
Using cached wheel-0.41.3-py3-none-any.whl (65 kB)
Using cached Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.5 MB)
Using cached numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.9 MB)
Building wheels for collected packages: pyarrow
Building wheel for pyarrow (pyproject.toml): started
Building wheel for pyarrow (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for pyarrow (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [289 lines of output]
<string>:36: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-312
creating build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/filesystem.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_generated_version.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/plasma.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/csv.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/types.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/cuda.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/benchmark.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/flight.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/cffi.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/conftest.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/substrait.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/json.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/ipc.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/pandas_compat.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/hdfs.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/feather.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/jvm.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/serialization.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/util.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/orc.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/compute.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/fs.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/dataset.py -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_compute_docstrings.py -> build/lib.linux-x86_64-cpython-312/pyarrow
creating build/lib.linux-x86_64-cpython-312/pyarrow/parquet
copying pyarrow/parquet/core.py -> build/lib.linux-x86_64-cpython-312/pyarrow/parquet
copying pyarrow/parquet/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/parquet
copying pyarrow/parquet/encryption.py -> build/lib.linux-x86_64-cpython-312/pyarrow/parquet
creating build/lib.linux-x86_64-cpython-312/pyarrow/vendored
copying pyarrow/vendored/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/vendored
copying pyarrow/vendored/docscrape.py -> build/lib.linux-x86_64-cpython-312/pyarrow/vendored
copying pyarrow/vendored/version.py -> build/lib.linux-x86_64-cpython-312/pyarrow/vendored
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_extension_type.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cuda.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_flight.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_schema.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_convert_builtin.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_types.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_csv.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_exec_plan.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_json.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_serialization_deprecated.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_table.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/strategies.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cpp_internals.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_sparse_tensor.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_hdfs.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_deprecations.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_serialization.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_tensor.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_substrait.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/read_record_batch.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cython.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/deserialize_buffer.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/conftest.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_orc.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_plasma_tf_op.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/pandas_threaded_import.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_dataset.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_io.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_filesystem.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/arrow_7980.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_scalars.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_misc.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_builder.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_plasma.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cffi.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_adhoc_memory_leak.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_jvm.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/arrow_16597.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_ipc.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_util.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_cuda_numba_interop.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_array.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/util.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_memory.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_gandiva.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_strategies.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/pandas_examples.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_udf.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_compute.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_fs.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_feather.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_gdb.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/test_pandas.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_datetime.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_data_types.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_parquet_writer.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/common.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/__init__.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_parquet_file.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/encryption.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/conftest.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_dataset.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_metadata.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_basic.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_compliant_nested_type.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_encryption.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
copying pyarrow/tests/parquet/test_pandas.py -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/parquet
running egg_info
writing pyarrow.egg-info/PKG-INFO
writing dependency_links to pyarrow.egg-info/dependency_links.txt
writing entry points to pyarrow.egg-info/entry_points.txt
writing requirements to pyarrow.egg-info/requires.txt
writing top-level names to pyarrow.egg-info/top_level.txt
ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any
reading manifest file 'pyarrow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '../LICENSE.txt'
warning: no files found matching '../NOTICE.txt'
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '#*' found anywhere in distribution
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
no previously-included directories found matching '.asv'
writing manifest file 'pyarrow.egg-info/SOURCES.txt'
copying pyarrow/__init__.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_compute.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_compute.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_csv.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_csv.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_cuda.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_cuda.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset_orc.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_dataset_parquet.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_exec_plan.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_feather.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_flight.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_fs.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_fs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_gcsfs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_hdfs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_hdfsio.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_json.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_orc.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_orc.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet_encryption.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_parquet_encryption.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_plasma.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_pyarrow_cpp_tests.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_pyarrow_cpp_tests.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_s3fs.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/_substrait.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/array.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/benchmark.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/builder.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/compat.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/config.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/error.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/gandiva.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/io.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/ipc.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/lib.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/lib.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/memory.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/pandas-shim.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/public-api.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/scalar.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/serialization.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/table.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/tensor.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
copying pyarrow/types.pxi -> build/lib.linux-x86_64-cpython-312/pyarrow
creating build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_dataset.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libplasma.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_python.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/__init__.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/common.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_fs.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_substrait.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_dataset_parquet.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_flight.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_cuda.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libarrow_feather.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
copying pyarrow/includes/libgandiva.pxd -> build/lib.linux-x86_64-cpython-312/pyarrow/includes
creating build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/ArrowPythonConfig.cmake.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/ArrowPythonFlightConfig.cmake.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/CMakeLists.txt -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/arrow-python-flight.pc.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
copying pyarrow/src/arrow-python.pc.in -> build/lib.linux-x86_64-cpython-312/pyarrow/src
creating build/lib.linux-x86_64-cpython-312/pyarrow/tensorflow
copying pyarrow/tensorflow/plasma_op.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/tensorflow
copying pyarrow/tests/bound_function_visit_strings.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
copying pyarrow/tests/pyarrow_cython_example.pyx -> build/lib.linux-x86_64-cpython-312/pyarrow/tests
creating build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow
creating build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/CMakeLists.txt -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/api.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/arrow_to_pandas.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/arrow_to_pandas.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/arrow_to_python_internal.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/benchmark.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/benchmark.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/common.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/common.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/csv.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/csv.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/datetime.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/datetime.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/decimal.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/decimal.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/deserialize.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/deserialize.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/extension_type.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/extension_type.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/filesystem.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/filesystem.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/flight.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/flight.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/gdb.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/gdb.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/helpers.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/helpers.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/inference.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/inference.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/init.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/init.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/io.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/io.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/ipc.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/ipc.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/iterators.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_convert.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_convert.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_internal.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_interop.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_to_arrow.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/numpy_to_arrow.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/parquet_encryption.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/parquet_encryption.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pch.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/platform.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow_api.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/pyarrow_lib.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_test.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_test.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_to_arrow.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/python_to_arrow.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/serialize.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/serialize.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/type_traits.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/udf.cc -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/udf.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
copying pyarrow/src/arrow/python/visibility.h -> build/lib.linux-x86_64-cpython-312/pyarrow/src/arrow/python
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/README.md -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.test1.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.test1.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/decimal.jsn.gz -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
copying pyarrow/tests/data/orc/decimal.orc -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/orc
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.all-named-index.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.column-metadata-handling.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
copying pyarrow/tests/data/parquet/v0.7.1.some-named-index.parquet -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/parquet
creating build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/feather
copying pyarrow/tests/data/feather/v0.17.0.version.2-compression.lz4.feather -> build/lib.linux-x86_64-cpython-312/pyarrow/tests/data/feather
running build_ext
creating /tmp/pip-install-2fn6embz/pyarrow_ad0ba6d0cd4345898a1cedc4dd36a4f6/build/cpp
-- Running CMake for PyArrow C++
cmake -DARROW_BUILD_DIR=build -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-2fn6embz/pyarrow_ad0ba6d0cd4345898a1cedc4dd36a4f6/build/dist -DPYTHON_EXECUTABLE=/usr/local/bin/python -DPython3_EXECUTABLE=/usr/local/bin/python -DPYARROW_CXXFLAGS= -DPYARROW_WITH_DATASET=off -DPYARROW_WITH_PARQUET_ENCRYPTION=off -DPYARROW_WITH_HDFS=off -DPYARROW_WITH_FLIGHT=off /tmp/pip-install-2fn6embz/pyarrow_ad0ba6d0cd4345898a1cedc4dd36a4f6/pyarrow/src
error: command 'cmake' failed: No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pyarrow
Failed to build pyarrow
ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
╭───────────────────── Traceback (most recent call last) ─────────────────────────────────╮ │
│ │ args = [ │ │
│ │ │ '/usr/local/bin/python', │ │
│ │ │ │ │
│ │ '/usr/local/lib/python3.12/site-packages/pip/__pip-run… │ │
│ │ │ 'install', │ │
│ │ │ '--ignore-installed', │ │
│ │ │ '--no-user', │ │
│ │ │ '--prefix', │ │
│ │ │ '/tmp/pip-build-env-21a4wy6w/overlay', │ │
│ │ │ '--no-warn-script-location', │ │
│ │ │ '--no-binary', │ │
│ │ │ ':none:', │ │
│ │ │ ... +9 │ │
│ │ ] │ │
│ │ extra_environ = { │ │
│ │ │ '_PIP_STANDALONE_CERT': │ │
│ │ '/usr/local/lib/python3.12/site-packages/pip/_vendor/c… │ │
│ │ } │ │
│ │ finder = <pip._internal.index.package_finder.PackageFinder │ │
│ │ object at 0x7f3aeb1fc0b0> │ │
│ │ format_control = 'only_binary' │ │
│ │ formats = set() │ │
│ │ index_urls = ['https://pypi.org/simple'] │ │
│ │ kind = 'build dependencies' │ │
│ │ pip_runnable = '/usr/local/lib/python3.12/site-packages/pip/__pip-run… │ │
│ │ prefix = <pip._internal.build_env._Prefix object at │ │
│ │ 0x7f3ae9b0b380> │ │
│ │ requirements = [ │ │
│ │ │ 'setuptools>=40.6.0', │ │
│ │ │ 'wheel', │ │
│ │ │ 'cython', │ │
│ │ │ 'pyarrow>=10.0.1,<10.1.0' │ │
│ │ ] │ │
│ │ spinner = <pip._internal.cli.spinners.NonInteractiveSpinner │ │
│ │ object at 0x7f3ae9b0b590> │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/local/lib/python3.12/site-packages/pip/_internal/utils/subprocess.py:22 │
│ 4 in call_subprocess │
│ │
│ 221 │ │ │ │ │ extra={"markup": True}, │
│ 222 │ │ │ │ ) │
│ 223 │ │ │ │
│ ❱ 224 │ │ │ raise error │
│ 225 │ │ elif on_returncode == "warn": │
│ 226 │ │ │ subprocess_logger.warning( │
│ 227 │ │ │ │ 'Command "%s" had error code %s in %s', │
│ │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ all_output = [ │ │
│ │ │ 'Collecting setuptools>=40.6.0\n', │ │
│ │ │ ' Obtaining dependency information for │ │
│ │ setuptools>=40.6.0 from https://files.pyt'+134, │ │
│ │ │ ' Using cached │ │
│ │ setuptools-68.2.2-py3-none-any.whl.metadata (6.3 │ │
│ │ kB)\n', │ │
│ │ │ 'Collecting wheel\n', │ │
│ │ │ ' Obtaining dependency information for wheel │ │
│ │ from https://files.pythonhosted.org'+116, │ │
│ │ │ ' Using cached │ │
│ │ wheel-0.41.3-py3-none-any.whl.metadata (2.2 │ │
│ │ kB)\n', │ │
│ │ │ 'Collecting cython\n', │ │
│ │ │ ' Obtaining dependency information for │ │
│ │ cython from https://files.pythonhosted.or'+159, │ │
│ │ │ ' Downloading │ │
│ │ Cython-3.0.5-cp312-cp312-manylinux_2_17_x86_64.m… │ │
│ │ │ 'Collecting pyarrow<10.1.0,>=10.0.1\n', │ │
│ │ │ ... +321 │ │
│ │ ] │ │
│ │ cmd = [ │ │
│ │ │ '/usr/local/bin/python', │ │
│ │ │ │ │
│ │ '/usr/local/lib/python3.12/site-packages/pip/__p… │ │
│ │ │ 'install', │ │
│ │ │ '--ignore-installed', │ │
│ │ │ '--no-user', │ │
│ │ │ '--prefix', │ │
│ │ │ '/tmp/pip-build-env-21a4wy6w/overlay', │ │
│ │ │ '--no-warn-script-location', │ │
│ │ │ '--no-binary', │ │
│ │ │ ':none:', │ │
│ │ │ ... +9 │ │
│ │ ] │ │
│ │ command_desc = 'pip subprocess to install build dependencies' │ │
│ │ cwd = None │ │
│ │ env = { │ │
│ │ │ 'HOSTNAME': '4ffeab1ce008', │ │
│ │ │ 'PYTHON_VERSION': '3.12.0', │ │
│ │ │ 'PWD': '/usr/src/schemachange', │ │
│ │ │ 'OPENBLAS_NUM_THREADS': '1', │ │
│ │ │ 'HOME': '/root', │ │
│ │ │ 'LANG': 'C.UTF-8', │ │
│ │ │ 'GPG_KEY': │ │
│ │ '7169605F62C751356D054A26A821E680E5FA6305', │ │
│ │ │ 'SHLVL': '1', │ │
│ │ │ 'PYTHON_PIP_VERSION': '23.2.1', │ │
│ │ │ 'PYTHON_GET_PIP_SHA256': │ │
│ │ '22b849a10f86f5ddf7ce148ca2a31214504ee6c83ef6268… │ │
│ │ │ ... +5 │ │
│ │ } │ │
│ │ error = <InstallationSubprocessError(reference='subproce… │ │
│ │ message='[green]pip subprocess to install build │ │
│ │ dependencies[/] did not run successfully.\nexit │ │
│ │ code: 1', context='See above for output.', │ │
│ │ note_stmt='This error originates from a │ │
│ │ subprocess, and is likely not a problem with │ │
│ │ pip.', hint_stmt=None)> │ │
│ │ extra_environ = { │ │
│ │ │ '_PIP_STANDALONE_CERT': │ │
│ │ '/usr/local/lib/python3.12/site-packages/pip/_ve… │ │
│ │ } │ │
│ │ extra_ok_returncodes = [] │ │
│ │ line = '' │ │
│ │ log_failed_cmd = True │ │
│ │ log_subprocess = <bound method VerboseLogger.verbose of │ │
│ │ <VerboseLogger pip.subprocessor (INFO)>> │ │
│ │ on_returncode = 'raise' │ │
│ │ output = 'Collecting setuptools>=40.6.0\n Obtaining │ │
│ │ dependency information for setuptools>'+30381 │ │
│ │ proc = <Popen: returncode: 1 args: │ │
│ │ ['/usr/local/bin/python', │ │
│ │ '/usr/local/lib/python...> │ │
│ │ proc_had_error = True │ │
│ │ show_stdout = False │ │
│ │ showing_subprocess = False │ │
│ │ spinner = <pip._internal.cli.spinners.NonInteractiveSpinner │ │
│ │ object at 0x7f3ae9b0b590> │ │
│ │ stdout_only = False │ │
│ │ unset_environ = [] │ │
│ │ use_spinner = True │ │
│ │ used_level = 15 │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
InstallationSubprocessError: pip subprocess to install build dependencies exited
with 1
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
@binitbhaskar, It does not appear to be a bug with schemachange.
Schemachange is supported on python 3.8, 3.9 and 3.10. Your docker container seems to be running Python 3.12.
Can you use python 3.10 instead of python 3.12 in your gitlab pipeline?