Permission denied on `json-schemas`

[rgstephens]

Hi @masci, hope you are open to contributions to this repo. It's a great example project for learning Haystack.

I'd like to submit a PR on the build.sh script to parameterize some things that are hard coded and I have a prototype that I'm working on to learn Haystack which I may ask you to add in the coming weeks.

For now, I'm running into this permission error. Here's the command and output.

DATA_IMAGE_NAME=stephens/elasticsearch-custom ./build.sh

When I review the haystack container I see:

[2022-12-31 19:28:41 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2022-12-31 19:28:41 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2022-12-31 19:28:41 +0000] [1] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2022-12-31 19:28:41 +0000] [8] [INFO] Booting worker with pid: 8
[2022-12-31 19:28:43 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/opt/venv/lib/python3.10/site-packages/uvicorn/workers.py", line 66, in init_process
    super(UvicornWorker, self).init_process()
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/opt/venv/lib/python3.10/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/venv/lib/python3.10/site-packages/rest_api/application.py", line 13, in <module>
    app = get_app()
  File "/opt/venv/lib/python3.10/site-packages/rest_api/utils.py", line 28, in get_app
    from rest_api.controller import file_upload, search, feedback, document, health
  File "/opt/venv/lib/python3.10/site-packages/rest_api/controller/file_upload.py", line 20, in <module>
    indexing_pipeline: Pipeline = get_pipelines().get("indexing_pipeline", None)
  File "/opt/venv/lib/python3.10/site-packages/rest_api/utils.py", line 59, in get_pipelines
    pipelines = setup_pipelines()
  File "/opt/venv/lib/python3.10/site-packages/rest_api/pipeline/__init__.py", line 28, in setup_pipelines
    query_pipeline = Pipeline.load_from_yaml(Path(config.PIPELINE_YAML_PATH), pipeline_name=config.QUERY_PIPELINE_NAME)
  File "/opt/venv/lib/python3.10/site-packages/haystack/pipelines/base.py", line 1908, in load_from_yaml
    return cls.load_from_config(
  File "/opt/venv/lib/python3.10/site-packages/haystack/pipelines/base.py", line 1969, in load_from_config
    validate_config(pipeline_config, strict_version_check=strict_version_check)
  File "/opt/venv/lib/python3.10/site-packages/haystack/pipelines/config.py", line 245, in validate_config
    validate_schema(pipeline_config=pipeline_config, strict_version_check=strict_version_check, extras=extras)
  File "/opt/venv/lib/python3.10/site-packages/haystack/pipelines/config.py", line 302, in validate_schema
    schema = load_schema()
  File "/opt/venv/lib/python3.10/site-packages/haystack/nodes/_json_schema.py", line 423, in load_schema
    raise e
  File "/opt/venv/lib/python3.10/site-packages/haystack/nodes/_json_schema.py", line 417, in load_schema
    update_json_schema(main_only=True)
  File "/opt/venv/lib/python3.10/site-packages/haystack/nodes/_json_schema.py", line 437, in update_json_schema
    os.makedirs(destination_path, exist_ok=True)
  File "/usr/local/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/venv/lib/python3.10/site-packages/haystack/json-schemas'
[2022-12-31 19:28:43 +0000] [8] [INFO] Worker exiting (pid: 8)
[2022-12-31 19:28:45 +0000] [1] [INFO] Shutting down: Master
[2022-12-31 19:28:45 +0000] [1] [INFO] Reason: Worker failed to boot.

[rgstephens]

If I go back to a slightly older image, deepset/haystack:cpu-v1.12.2, the problem is resolved.

The cpu-main image that gives me the permission issue is digest cb0e25d57f1c.

[masci]

The fix for the permission error was merged as part of deepset-ai/haystack#3701 so cpu-main should be good again

[rgstephens]

Thanks!