ImportError: cannot import name 'packaging' from 'pkg_resources'
Sieltek opened this issue · 4 comments
🐛 Describe the bug
Problem seems to arise when loading a model using install_py_dep_per_model=true which will upgrade the setuptools version to latest 70.0 and thus breaking the handler loading.
Solution might be to replace the deprecated import in base handler from pkg_resources import packaging by import packaging
Error logs
2024-06-05T10:24:37,046 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - Backend worker process died.
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 108, in load
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module, function_name = self._load_handler_file(handler)
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 153, in _load_handler_file
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name)
2024-06-05T10:24:37,047 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/usr/lib64/python3.11/importlib/init.py", line 126, in import_module
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1206, in _gcd_import
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1178, in _find_and_load
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1149, in _find_and_load_unlocked
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 690, in _load_unlocked
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 940, in exec_module
2024-06-05T10:24:37,048 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/tmp/models/9a75af565f7e478ebcf169da7ae7cbc0/handler.py", line 1, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from handler_yolo import YoloHandler
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/tmp/models/9a75af565f7e478ebcf169da7ae7cbc0/handler_yolo.py", line 6, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from ts.torch_handler.object_detector import ObjectDetector
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/torch_handler/object_detector.py", line 8, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from .vision_handler import VisionHandler
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/torch_handler/vision_handler.py", line 16, in
2024-06-05T10:24:37,049 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from .base_handler import BaseHandler
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/torch_handler/base_handler.py", line 13, in
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - from pkg_resources import packaging
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ImportError: cannot import name 'packaging' from 'pkg_resources' (/tmp/models/9a75af565f7e478ebcf169da7ae7cbc0/pkg_resources/init.py)
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG -
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - During handling of the above exception, another exception occurred:
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG -
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 263, in
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - worker.run_server()
2024-06-05T10:24:37,050 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 231, in run_server
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - self.handle_connection(cl_socket)
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 194, in handle_connection
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - service, result, code = self.load_model(msg)
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_service_worker.py", line 131, in load_model
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - service = model_loader.load(
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 110, in load
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module = self._load_default_handler(handler)
2024-06-05T10:24:37,051 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/home/torch/.local/lib/python3.11/site-packages/ts/model_loader.py", line 159, in _load_default_handler
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - module = importlib.import_module(module_name, "ts.torch_handler")
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "/usr/lib64/python3.11/importlib/init.py", line 126, in import_module
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - return _bootstrap._gcd_import(name[level:], package, level)
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1206, in _gcd_import
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1178, in _find_and_load
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1128, in _find_and_load_unlocked
2024-06-05T10:24:37,052 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 241, in _call_with_frames_removed
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1206, in _gcd_import
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1178, in _find_and_load
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - File "", line 1142, in _find_and_load_unlocked
2024-06-05T10:24:37,053 [INFO ] W-9000-crochet_yolov8s_512_240207_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'ts.torch_handler.handler'
2024-06-05T10:24:37,061 [INFO ] epollEventLoopGroup-5-1 org.pytorch.serve.wlm.WorkerThread - 9000 Worker disconnected. WORKER_STARTED
2024-06-05T10:24:37,062 [DEBUG] W-9000-crochet_yolov8s_512_240207_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2024-06-05T10:24:37,062 [DEBUG] W-9000-crochet_yolov8s_512_240207_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died., responseTimeout:120sec
Installation instructions
python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org pip==24.0 setuptools==69.5.1
python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host download.pytorch.org --no-cache-dir numpy==1.24.3 cython==0.29.34 wheel==0.40.0 pillow==9.3.0 psutil==5.9.5 requests==2.31.0 captum==0.6.0 packaging==23.1 pynvml==11.4.1 pyyaml==6.0
python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host download.pytorch.org --no-cache-dir torch==2.3.0+cpu torchvision==0.18.0+cpu torchtext==0.18.0 torchaudio==2.3.0+cpu --extra-index-url https://download.pytorch.org/whl/cpu
python3.11 -m pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host download.pytorch.org --no-cache-dir torchserve==0.11.0 torch-model-archiver==0.11.0 torch-workflow-archiver==0.2.13
mkdir /model-server
mkdir /model-server/model-store
mkdir /model-server/wf-store
Model Packaging
torch-model-archiver --model-name my_model_name --handler handler.py --extra-files index_to_name.json --serialized-file best.pt -r requirements.txt -v 1.0 --force
crash:
requirements.txt:
numpy
workaround:
requirements.txt:
numpy
setuptools==69.5.1
loader failing at
serve/ts/torch_handler/base_handler.py
l.13 from pkg_resources import packaging
config.properties
echo "
inference_address=http://0.0.0.0:8080
management_address=http://0.0.0.0:8081
metrics_address=http://0.0.0.0:8082
grpc_inference_port=7070
grpc_management_port=7071
cors_allowed_origin=*
cors_allowed_methods=GET, POST, PUT, DELETE, OPTIONS
default_workers_per_model=2
job_queue_size=1000
model_store=/model-server/model-store
workflow_store=/model-server/wf-store
load_models=all
prefer_direct_buffer=true
install_py_dep_per_model=true
enable_metrics_api=true
prefer_direct_buffer=true
disable_system_metrics=false
metrics_mode=prometheus
cpu_launcher_enable=true
cpu_launcher_args=--use_logical_core
" > /model-server/config.properties
Versions
Environment headers
Torchserve branch:
torchserve==0.11.0
torch-model-archiver==0.11.0
Python version: 3.11 (64-bit runtime)
Python executable: /usr/bin/python3
Versions of relevant python libraries:
captum==0.6.0
numpy==1.24.3
psutil==5.9.5
requests==2.31.0
torch==2.3.0+cpu
torch-model-archiver==0.11.0
torch-workflow-archiver==0.2.13
torchaudio==2.3.0+cpu
torchserve==0.11.0
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
wheel==0.40.0
torch==2.3.0+cpu
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
torchaudio==2.3.0+cpu
Java Version:
OS: N/A
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-18)
Clang version: N/A
CMake version: N/A
Environment:
library_path (LD_/DYLD_):
Repro instructions
torchserve --start --ts-config /model-server/config.properties
Possible Solution
Specify setuptool version in model.mar requirement.txt
workaround:
requirements.txt:
numpy
setuptools==69.5.1
python -m pip install setuptools==69.5.1
Exporting export PIP_NO_DEPS=true in the environment is also a workaround
If running in a Docker environment, you can use the following command:
docker run -e PIP_NO_CACHE_DIR=true -e PIP_NO_DEPS=true ...This sets PIP_NO_CACHE_DIR=true and PIP_NO_DEPS=true as environment variables within the Docker container.
BUT If you choose to run with PIP_NO_CACHE_DIR=true and PIP_NO_DEPS=true in environment, you need to manage the dependencies manually.
Also, if you get this or similar when trying to host Whisper, notice the comments in the overview regarding setuptools-rust.