Docker build/package issues on macos

[zacroker]

Are you certain it's a bug?

• Yes, it looks like a bug

Are you using the latest plugin release?

• Yes, I'm using the latest plugin release

Is there an existing issue for this?

• I have searched existing issues, it hasn't been reported yet

Issue description

Issue 1

Description

• Default builder does not allow installation of kafka and related packages
• When running sls package, the error is The folder you are executing pip from can no longer be found
  • The error obscures the actual issue, which is that there is something wrong with librdkafka

Solution

• Use a dockerfile to build the layer with the correct dependency

FROM public.ecr.aws/sam/build-python3.9

RUN yum update -y \
    && yum install -y which \
    && git clone https://github.com/edenhill/librdkafka.git \
    && cd librdkafka \
    && ./configure \
    && make \
    && make install \
    && cd ../ \
    && rm -rf librdkafka

Issue 2

Description

• Default builder does not allow installation of django_celery_beat
• Error emitted is OSError: [Errno 18] Invalid cross-device link: '/tmp/pip-target-1byxfmwv/lib/python/django_celery_beat' -> '/var/task/django_celery_beat'
• Not really sure of a workaround at this stage

Service configuration (serverless.yml) content

N/A

Command name and used flags

sls package --verbose --stage preview-1493

Command output

Packaging helios for stage preview-1493 (ap-southeast-2)

Packaging
Disabling function: 
...

Generating requirements.txt from poetry.lock
Parsed requirements.txt from pyproject.toml in /Users/zaccroker/develop/helios/.serverless/requirements.txt
Installing requirements from "/Users/zaccroker/Library/Caches/serverless-python-requirements/ecd98f46af22a3dce30457f1566bc6c7f95d2cab4b8374df3819cc5cece134f1_x86_64_slspyc/requirements.txt"
Docker Image: sls-py-reqs-custom
Using download cache directory /Users/zaccroker/Library/Caches/serverless-python-requirements/downloadCacheslspyc
Running docker run --rm -v /Users/zaccroker/Library/Caches/serverless-python-requirements/ecd98f46af22a3dce30457f1566bc6c7f95d2cab4b8374df3819cc5cece134f1_x86_64_slspyc\:/var/task\:z -v /Users/zaccroker/Library/Caches/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z -u 0 sls-py-reqs-custom python3.9 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache...
Environment: darwin, node 18.18.0, framework 3.35.2 (local), plugin 7.0.3, SDK 4.4.0
Credentials: Serverless Framework Provider: "arn:aws:iam::xxx:role/xxx" (https://app.serverless.com/clipsalsolar/apps/helios/helios/preview-1493/ap-southeast-2/providers)
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Running "docker run --rm -v /Users/zaccroker/Library/Caches/serverless-python-requirements/ecd98f46af22a3dce30457f1566bc6c7f95d2cab4b8374df3819cc5cece134f1_x86_64_slspyc:/var/task:z -v /Users/zaccroker/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 sls-py-reqs-custom python3.9 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache" failed with: "ERROR: Exception:
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/shutil.py", line 825, in move
    os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/tmp/pip-target-1byxfmwv/lib/python/django_celery_beat' -> '/var/task/django_celery_beat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "/var/lang/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/var/lang/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 510, in run
    self._handle_target_dir(
  File "/var/lang/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 569, in _handle_target_dir
    shutil.move(os.path.join(lib_dir, item), target_item_dir)
  File "/var/lang/lib/python3.9/shutil.py", line 841, in move
    copytree(src, real_dst, copy_function=copy_function,
  File "/var/lang/lib/python3.9/shutil.py", line 568, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/var/lang/lib/python3.9/shutil.py", line 522, in _copytree
    raise Error(errors)
shutil.Error: [('/tmp/pip-target-1byxfmwv/lib/python/django_celery_beat/locale/zh_hans', '/var/task/django_celery_beat/locale/zh_hans', "[Errno 17] File exists: '/var/task/django_celery_beat/locale/zh_hans'")]"

Environment information

Framework Core: 3.35.2 (local)
Plugin: 7.0.3
SDK: 4.4.0

serverless-python-requirements 6.0.0