Cannot build snap - core22 etc and bypass the gnome plug incompatibilities
abulka opened this issue · 8 comments
abulka commented
Following up from the discussion https://forum.snapcraft.io/t/failed-to-release-snap/33879/7 on your workarounds to get wxpython snaps working with core22 etc and bypass the gnome plug incompatibilities. You said:
It looks like the python app you’re packing is using wxWidgets. The app I packed also uses wxWidgets, python and the gnome extension on core22. Feel free to take a look https://github.com/goldstar611/chirp/blob/master/snap/snapcraft.yaml
So, trying to use your project as a working example, which I can later copy into my project. I forked your project and tried to build it, but got an error straight away.
snapcraft --use-lxd
Launching instance...
'override-pull' in part 'chirp' failed with code 127.
Review the scriptlet and make sure it's correct.
Failed to execute pack in instance.
Full execution log: '/home/andy/.local/state/snapcraft/log/snapcraft-20230310-172025.226395.log'
the tail of the full log is
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libxext6
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libdatrie1
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libglx-mesa0
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libglvnd0
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.038 Extracting stage package: libwayland-cursor0
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.077 :: + sudo apt install -y curl
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.080 :: /bin/bash: line 52: sudo: command not found
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.273 'override-pull' in part 'chirp' failed with code 127.
2023-03-10 17:32:08.191 :: Review the scriptlet and make sure it's correct.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Traceback (most recent call last):
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 171, in run
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 aex.execute(action, stdout=stream, stderr=stream)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 304, in execute
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 self._executor.execute(actions, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 128, in execute
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 self._run_action(act, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 193, in _run_action
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 handler.run_action(action, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 168, in run_action
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 state = handler(step_info, stdout=stdout, stderr=stderr)
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 194, in _run_pull
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 self._run_step(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 481, in _run_step
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 step_handler.run_scriptlet(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 275, in run_scriptlet
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 raise errors.ScriptletRunError(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 craft_parts.errors.ScriptletRunError: 'override-pull' in part 'chirp' failed with code 127.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Review the scriptlet and make sure it's correct.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 The above exception was the direct cause of the following exception:
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Traceback (most recent call last):
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 294, in _run_command
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 _run_lifecycle_and_pack(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 337, in _run_lifecycle_and_pack
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 lifecycle.run(
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 186, in run
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 raise errors.PartsLifecycleError(str(err)) from err
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 snapcraft.errors.PartsLifecycleError: 'override-pull' in part 'chirp' failed with code 127.
2023-03-10 17:32:08.191 :: 2023-03-10 22:32:05.318 Review the scriptlet and make sure it's correct.
2023-03-10 17:32:09.927 Failed to execute pack in instance.
2023-03-10 17:32:09.934 Detailed information: Run the same command again with --debug to shell into the environment if you wish to introspect this failure.
2023-03-10 17:32:09.952 Traceback (most recent call last):
2023-03-10 17:32:09.952 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 557, in _run_in_provider
2023-03-10 17:32:09.952 instance.execute_run(cmd, check=True, cwd=output_dir)
2023-03-10 17:32:09.952 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_providers/lxd/lxd_instance.py", line 289, in execute_run
2023-03-10 17:32:09.952 return self.lxc.exec(
2023-03-10 17:32:09.952 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_providers/lxd/lxc.py", line 410, in exec
2023-03-10 17:32:09.952 return runner(final_cmd, **kwargs) # pylint: disable=subprocess-run-check
2023-03-10 17:32:09.952 File "/snap/snapcraft/8931/usr/lib/python3.8/subprocess.py", line 516, in run
2023-03-10 17:32:09.952 raise CalledProcessError(retcode, process.args,
2023-03-10 17:32:09.952 subprocess.CalledProcessError: Command '['lxc', '--project', 'snapcraft', 'exec', 'local:snapcraft-chirp-snap-on-amd64-for-amd64-11828049', '--cwd', '/root/project', '--', 'env', 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin', 'SNAPCRAFT_MANAGED_MODE=1', 'snapcraft', 'pack', '--verbosity=brief', '--build-for', 'amd64']' returned non-zero exit status 1.
2023-03-10 17:32:09.952 Full execution log: '/home/andy/.local/state/snapcraft/log/snapcraft-20230310-172025.226395.log'
goldstar611 commented
/bin/bash: line 52: sudo: command not found
[Edit] sorry for the quick message. You'll need to get sudo added to the snap or just remove that line if you don't want curl.
abulka commented
Thanks - though I wonder how you manage to build the snap yourself, if the snap doesn't have sudo. I managed to fix it by adding
override-pull: |
apt update <----
apt install -y sudo <----
sudo apt install -y curl
....to chirp's snapcraft.yaml.
P.S. I've used your chirp snapcraft.yaml as a template for my own project but getting an error - the build script is attempting to create a virtual environment using the venv module, but it does not have permission to write to the /root/parts/build-the-python-stuff-please/install directory.
2023-03-11 19:00:24.373 :: to prefix=/root/parts/build-the-python-stuff-please/install/usr
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:20.300 remove directory /root/parts/build-the-python-stuff-please/build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.125 Executing PosixPath('/root/parts/build-the-python-stuff-please/run/build.sh')
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.151 :: + python3 -m venv /root/parts/build-the-python-stuff-please/install
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.625 :: Error: [Errno 13] Permission denied: '/root/parts/build-the-python-stuff-please/install/bin/python3'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.635 Failed to run the build script for part 'build-the-python-stuff-please'.
build-the-python-stuff-please:
plugin: python
source: .
build-environment:
# WORKAROUND: The python plugin is broken with gnome extension
- PATH: ${CRAFT_PART_INSTALL}/bin:${PATH}
- PYTHONPATH: ""
stage-packages:
- python3-requests
- python3-serial
- python3-six
- python3-suds
- python3-future
- python3-yattag
- python3-wxgtk4.0
stage:
# WORKAROUND: Skip venv from python plugin
- -bin/activate
- -bin/activate.csh
- -bin/activate.fish
- -bin/Activate.ps1
- -bin/python
- -bin/python3
- -bin/python3.10
- -bin/pip
- -bin/pip3
- -bin/pip3.10
- -pyvenv.cfg
python-requirements:
- /root/project/.requirement-extras/requirements-linux-22.txt
organize:
usr/lib/$CRAFT_ARCH_TRIPLET/pulseaudio/libpulsecommon-15.99.so: usr/lib/$CRAFT_ARCH_TRIPLET/libpulsecommon-15.99.so
# override-build: |
# install -Dm644 "snap/gui/pynsource.desktop" -t "$CRAFT_PART_INSTALL/usr/share/applications/"
# craftctl default Any ideas appreciated. Perhaps this is a side effect of your workarounds removing venv (as per your comment). But then why does my project want to use venv and yours does not?
goldstar611 commented
CHIRP has an override-pull section but it never actually calls craftctl default (this goes hand in hand with the source: NOT_USED.zip) Is that venv error coming from the pull step?
abulka commented
Unlike yours, my snapcraft.yaml doesn't have an override-pull section as I want to build from source: . The venv error I think is thus coming from the build step:
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.556 Executed: overlay gnome/sdk
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.556 Executing parts lifecycle: build build-the-python-stuff-please
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.557 Executing action
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.557 execute action build-the-python-stuff-please:Action(part_name='build-the-python-stuff-please', step=Step.BUILD, action_type=ActionType.RUN, reason=None, project_vars=None, properties=ActionProperties(changed_files=None, changed_dirs=None))
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:06.557 load state file: /root/parts/build-the-python-stuff-please/state/pull
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:12.486 fix artifacts: unpack_dir='/root/parts/build-the-python-stuff-please/install'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:12.496 fix symlink: path='/root/parts/build-the-python-stuff-please/install/usr/share/X11/rgb.txt', unpack_dir='/root/parts/build-the-python-stuff-please/install', root='/root/parts/build-the-python-stuff-please/install/usr/share/X11'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:12.526 fix symlink: target='/root/parts/build-the-python-stuff-please/install/etc/X11/rgb.txt'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:13.056 For pkg-config file /root/parts/build-the-python-stuff-please/install/usr/share/pkgconfig/shared-mime-info.pc, prefix was changed from prefix=/usr
2023-03-11 19:00:24.373 :: to prefix=/root/parts/build-the-python-stuff-please/install/usr
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:20.300 remove directory /root/parts/build-the-python-stuff-please/build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.125 Executing PosixPath('/root/parts/build-the-python-stuff-please/run/build.sh')
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.151 :: + python3 -m venv /root/parts/build-the-python-stuff-please/install
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.625 :: Error: [Errno 13] Permission denied: '/root/parts/build-the-python-stuff-please/install/bin/python3'
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.635 Failed to run the build script for part 'build-the-python-stuff-please'.
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.671 Traceback (most recent call last):
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.671 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 144, in _builtin_build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 subprocess.run(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/usr/lib/python3.8/subprocess.py", line 516, in run
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 raise CalledProcessError(retcode, process.args,
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 subprocess.CalledProcessError: Command '['/root/parts/build-the-python-stuff-please/run/build.sh']' returned non-zero exit status 1.
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 The above exception was the direct cause of the following exception:
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 Traceback (most recent call last):
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 171, in run
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 aex.execute(action, stdout=stream, stderr=stream)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 304, in execute
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 self._executor.execute(actions, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 128, in execute
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 self._run_action(act, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/executor.py", line 193, in _run_action
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 handler.run_action(action, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 168, in run_action
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 state = handler(step_info, stdout=stdout, stderr=stderr)
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 305, in _run_build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 self._run_step(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/part_handler.py", line 489, in _run_step
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 return step_handler.run_builtin()
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 106, in run_builtin
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 return handler()
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/craft_parts/executor/step_handler.py", line 152, in _builtin_build
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 raise errors.PluginBuildError(part_name=self._part.name) from process_error
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 craft_parts.errors.PluginBuildError: Failed to run the build script for part 'build-the-python-stuff-please'.
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 The above exception was the direct cause of the following exception:
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 Traceback (most recent call last):
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 294, in _run_command
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 _run_lifecycle_and_pack(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/lifecycle.py", line 337, in _run_lifecycle_and_pack
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 lifecycle.run(
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 File "/snap/snapcraft/8931/lib/python3.8/site-packages/snapcraft/parts/parts.py", line 186, in run
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 raise errors.PartsLifecycleError(str(err)) from err
2023-03-11 19:00:24.373 :: 2023-03-12 00:00:21.672 snapcraft.errors.PartsLifecycleError: Failed to run the build script for part 'build-the-python-stuff-please'.
2023-03-11 19:00:25.974 Failed to execute pack in instance.
goldstar611 commented
Could you send me a PR to https://github.com/goldstar611/atestsnap with your snapcraft.yaml file?
I'd like to see if Canonical's build servers fail in the same way. One thing I noticed was that sometimes local snap development didnt fail in the same way as the remote builders.
abulka commented
I've done two PR's, the first is just my full snapcraft.yaml file. The other is the entire project I'm trying to build.
Interestingly, in the full project copy, building the snap didn't give me a python3 -m venv related error but gave me a snap "pynsource": path "bin/python3" does not exist error. Not sure why the error changed - am running locally in a ubuntu 22.04 VM. There is also a github action .github/workflows/build-snap.yml which has in the past given me the same errors as locally.
goldstar611 commented
Hey Andy, I'm going to close the issue here but I'm curious if you ever got a sufficient workaround to some of the snap issues you faced (cant release because of symlink error, and such).
abulka commented
Yes I did - the new version of snapcraft 7.3.1.post21+gitfa823013 from edge or later removes the need for all workarounds. Can't wait for it to be available in stable so that GitHub Actions and other build tools can use it.
Thanks for asking!