[BUG] build fails on apple silicon
natb1 opened this issue · 10 comments
I'm working through the build on a silicon Mac. Has anybody else tried this? The build is failing due to platform issues.
=> ERROR [internal] load metadata for docker.io/library/openjdk:17-alpine 0.2s
------
> [internal] load metadata for docker.io/library/openjdk:17-alpine:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: no match for platform in manifest sha256:4b6abae565492dbe9e7a894137c966a7485154238902f2f25e9dbd9784383d81: not found
ERROR: Service 'files-manager' failed to build : Build failed
I have a PR for to fix this first issue. The build takes a while though so I will continue to update if I run into any additional issues.
Now I am seeing this when building the "backend" container:
#8 360.3 Traceback (most recent call last):
#8 360.3 File "<string>", line 36, in <module>
#8 360.3 File "<pip-setuptools-caller>", line 34, in <module>
#8 360.3 File "/tmp/pip-install-3_p1yfpl/pandas_ab3aa9c5f6ee4af681e0a94b2683c0d9/setup.py", line 749, in <module>
#8 360.3 **setuptools_kwargs)
#8 360.3 File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
#8 360.3 return distutils.core.setup(**attrs)
#8 360.3 File "/usr/local/lib/python3.7/distutils/core.py", line 148, in setup
#8 360.3 dist.run_commands()
#8 360.3 File "/usr/local/lib/python3.7/distutils/dist.py", line 966, in run_commands
#8 360.3 self.run_command(cmd)
#8 360.3 File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
#8 360.3 cmd_obj.run()
#8 360.3 File "/usr/local/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
#8 360.3 return orig.install.run(self)
#8 360.3 File "/usr/local/lib/python3.7/distutils/command/install.py", line 545, in run
#8 360.3 self.run_command('build')
#8 360.3 File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
#8 360.3 self.distribution.run_command(command)
#8 360.3 File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
#8 360.3 cmd_obj.run()
#8 360.3 File "/usr/local/lib/python3.7/distutils/command/build.py", line 135, in run
#8 360.3 self.run_command(cmd_name)
#8 360.3 File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
#8 360.3 self.distribution.run_command(command)
#8 360.3 File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
#8 360.3 cmd_obj.run()
#8 360.3 File "/usr/local/lib/python3.7/distutils/command/build_ext.py", line 340, in run
#8 360.3 self.build_extensions()
#8 360.3 File "/tmp/pip-install-3_p1yfpl/pandas_ab3aa9c5f6ee4af681e0a94b2683c0d9/setup.py", line 377, in build_extensions
#8 360.3 self.check_cython_extensions(self.extensions)
#8 360.3 File "/tmp/pip-install-3_p1yfpl/pandas_ab3aa9c5f6ee4af681e0a94b2683c0d9/setup.py", line 374, in check_cython_extensions
#8 360.3 """.format(src=src))
#8 360.3 Exception: Cython-generated file 'pandas/_libs/algos.c' not found.
#8 360.3 Cython is required to compile pandas from a development branch.
#8 360.3 Please install Cython or download a release package of pandas.
#8 360.3
#8 360.3 [end of output]
#8 360.3
#8 360.3 note: This error originates from a subprocess, and is likely not a problem with pip.
#8 360.3 error: legacy-install-failure
#8 360.3
#8 360.3 × Encountered error while trying to install package.
#8 360.3 ╰─> pandas
Not sure if it's a platform issue, just posting here in case anybody has seen it before.
^ this is now also fixed in PR, working through additional build issues
@natb1 Thanks for raising this issue up.
Docker images built with Apple Silicon (or another ARM64 based architecture) can create issues when deploying the images to a Linux or Windows based AMD64 environment. Therefore, you need a way to build AMD64 based images on the ARM64 architecture, whether it's using Docker build (for individual images) or docker-compose build (e.g. for multi-image apps running in a docker compose network).
For building single docker images: Set your environment variable using the command line or modifying your .bashrc or .zshenv file as suggested in the accepted answer.
export DOCKER_DEFAULT_PLATFORM=linux/amd64
Alternatively, in the Dockerfile, include the following flag in the FROM command (for a multi-stage Dockerfile build, the flag is only needed for the first stage):
FROM --platform=linux/amd64 python:3.7
For building images as part of a docker-compose build, include the platform: linux/amd64 for each service.
backend:
platform: linux/amd64
build: ./perseus-api
container_name: backend
environment:
PERSEUS_ENV: Docker
ports:
- "5000:5000"
depends_on:
- shareddb
- files-manager
Reference: forcing-docker-to-use-linux-amd64-platform
So we need not only to update docker-compose.yaml file, but we need to update all services in it. And also need to update Dockerfile in all services.
Yes, this is what the PR I submitted does. The build works for me if you just update the compose. I don't think the Dockerfiles need to be updated.
Sometimes we need to build without compose, using just docker build or podman build.
Also all components should have the same platform: linux/amd64.
So generally your PR will work but we need to expand it and test.
Ok, thank you. I also found an issue with the port used by the backend. 5000 is a standard port used by OSX. So I changed it to 5001.
5001 reserved to User service.
@chMatvey can we change backend port to anything different from 5000? Do we need to update anything except Dockerfile?
oops, just moved to 5004
The issue was fixed