FileNotFoundError: [Errno 2] No such file or directory: b'/home/myself/.local/share/containers/storage/overlay/4becb549defe1cc8b8a16e21e7e767d1e8e3cb9e4aa1b70d75cf63a4d42f975b/merged//tmp/ansible-tmp-1662124559.739739-172714-199418447811172/AnsiballZ_stat.py'
tpo opened this issue · 2 comments
tpo commented
I've updated from Fedora 35 to Fedora 36 and building my container image stopped working.
Running with ANSIBLE_VERBOSITY=4 ansible-bender -v build and am getting this:
15:15:51.123 core.py INFO no bender data found in the playbook
15:15:51.150 utils.py INFO running command: "['podman', 'version']"
15:15:51.313 utils.py INFO running command: "['buildah', 'version']"
15:15:51.334 utils.py INFO running command: "['buildah', 'from', '--name', '$PREFIX-20220902-151551129713-cont-20220902151551334368', 'docker.io/library/python:3.9-bullseye']"
15:15:53.671 utils.py INFO running command: "['buildah', 'run', '$PREFIX-20220902-151551129713-cont-20220902151551334368', 'true']"
15:15:54.008 utils.py INFO running command: "['buildah', 'rm', '$PREFIX-20220902-151551129713-cont-20220902151551334368']"
15:15:54.151 utils.py INFO running command: "['buildah', 'inspect', '-t', 'image', 'docker.io/library/python:3.9-bullseye']"
15:15:54.258 utils.py INFO running command: "['podman', 'run', '--rm', 'docker.io/library/python:3.9-bullseye', 'true']"
15:15:54.817 utils.py INFO running command: "['buildah', 'inspect', '-t', 'image', 'docker.io/library/python:3.9-bullseye']"
15:15:54.885 utils.py INFO running command: "['buildah', 'from', '--name', '$PREFIX-20220902-151551129713-cont', 'b6be79db3b12498f278ce0fd154a99b8a84b10093117d6b4e3dc7c846d2b103f']"
15:15:55.007 core.py INFO creating inventory file /tmp/abt2_2fh7j/inventory
15:15:55.028 utils.py INFO running command: "['ansible-playbook', '--version']"
15:15:55.383 core.py INFO we are running rootless, prepending `buildah unshare`
15:15:55.383 utils.py INFO running command: "['buildah', 'unshare', '--', 'ansible-playbook', '-c', 'buildah', '-i', '/tmp/abt2_2fh7j/inventory', '.build-playbook-20220902-151555027157-fqqkyweozo.yaml']"
ansible-playbook [core 2.12.7]
config file = /tmp/abt2_2fh7j/ansible.cfg
configured module search path = ['/home/myself/.ansible/collections/ansible_collections/sourcepole/public/plugins']
[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing names
to new standard, use callbacks_enabled instead. This feature will be removed
ansible python module location = /usr/lib/python3.10/site-packages/ansible
ansible collection location = /home/myself/.ansible/collections:/usr/share/ansible/collections
from ansible-core in version 2.15. Deprecation warnings can be disabled by
executable location = /usr/bin/ansible-playbook
setting deprecation_warnings=False in ansible.cfg.
python version = 3.10.6 (main, Aug 2 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)]
jinja version = 3.0.3
libyaml = True
Using /tmp/abt2_2fh7j/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /tmp/abt2_2fh7j/inventory as it did not pass its verify_file() method
script declined parsing /tmp/abt2_2fh7j/inventory as it did not pass its verify_file() method
auto declined parsing /tmp/abt2_2fh7j/inventory as it did not pass its verify_file() method
Parsed /tmp/abt2_2fh7j/inventory inventory source with ini plugin
Loading collection sourcepole.public from /home/myself/.ansible/collections/ansible_collections/sourcepole/public
[DEPRECATION WARNING]: "include" is deprecated, use include_tasks/import_tasks
instead. This feature will be removed in version 2.16. Deprecation warnings can
be disabled by setting deprecation_warnings=False in ansible.cfg.
redirecting (type: modules) ansible.builtin.openssh_keypair to community.crypto.openssh_keypair
Loading collection community.crypto from /usr/lib/python3.10/site-packages/ansible_collections/community/crypto
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python3.10/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.
Loading callback plugin a_container_image_snapshoter of type hard-worker, v2.0 from /home/myself/.local/lib/python3.10/site-packages/ansible_bender/callback_plugins/snapshoter.py
PLAYBOOK: .build-playbook-20220902-151555027157-fqqkyweozo.yaml ****************
Positional arguments: .build-playbook-20220902-151555027157-fqqkyweozo.yaml
verbosity: 4
connection: buildah
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/tmp/abt2_2fh7j/inventory',)
forks: 5
1 plays in .build-playbook-20220902-151555027157-fqqkyweozo.yaml
PLAY [Build drop-unreachable-postgres-dbs image] *******************************
META: ran handlers
TASK [some task running...] ***
[...]
15:15:59.478 api.py INFO layer None for content ae0cf3ee1bf18b3bcb6129a180cf280b50213fcdc1b38c7136b20ca6d8fb38509f89bb3279ea7026a25dddfb881204c012e375222b49d1db4385641433d4aded does not exist
15:15:59.478 utils.py INFO running command: "['buildah', 'rm', '$PREFIX-20220902-151551129713-cont']"
15:15:59.561 utils.py INFO running command: "['buildah', 'from', '--name', '$PREFIX-20220902-151551129713-cont', 'sha256:9ffc19d41952f24ea1c2196187d7790e8bf81d033f615d8ea879d698db44af15']"
redirecting (type: connection) ansible.builtin.buildah to containers.podman.buildah
Loading collection containers.podman from /home/myself/.ansible/collections/ansible_collections/containers/podman
Using buildah connection from collection
<$PREFIX-20220902-151551129713-cont> RUN [b'buildah', b'mount', b'$PREFIX-20220902-151551129713-cont']
STDOUT /home/myself/.local/share/containers/storage/overlay/4becb549defe1cc8b8a16e21e7e767d1e8e3cb9e4aa1b70d75cf63a4d42f975b/merged
STDERR
RC CODE 0
MOUNTPOINT b'/home/myself/.local/share/containers/storage/overlay/4becb549defe1cc8b8a16e21e7e767d1e8e3cb9e4aa1b70d75cf63a4d42f975b/merged/' RC 0 STDERR b''
<$PREFIX-20220902-151551129713-cont> RUN [b'buildah', b'run', b'$PREFIX-20220902-151551129713-cont', b'/bin/sh', b'-c', b'( umask 77 && mkdir -p "` echo /tmp `"&& mkdir "` echo /tmp/ansible-tmp-1662124559.739739-172714-199418447811172 `" && echo ansible-tmp-1662124559.739739-172714-199418447811172="` echo /tmp/ansible-tmp-1662124559.739739-172714-199418447811172 `" ) && sleep 0']
STDOUT ansible-tmp-1662124559.739739-172714-199418447811172=/tmp/ansible-tmp-1662124559.739739-172714-199418447811172
STDERR
RC CODE 0
STDOUT b''
STDERR b''
Using module file /usr/lib/python3.10/site-packages/ansible/modules/stat.py
<$PREFIX-20220902-151551129713-cont> PUT /home/myself/.ansible/tmp/ansible-local-1723215q6r34bg/tmp1p33k746 TO /tmp/ansible-tmp-1662124559.739739-172714-199418447811172/AnsiballZ_stat.py
<$PREFIX-20220902-151551129713-cont> RUN [b'buildah', b'run', b'$PREFIX-20220902-151551129713-cont', b'/bin/sh', b'-c', b'rm -f -r /tmp/ansible-tmp-1662124559.739739-172714-199418447811172/ > /dev/null 2>&1 && sleep 0']
STDOUT
STDERR
RC CODE 0
STDOUT b''
STDERR b''
<$PREFIX-20220902-151551129713-cont> RUN [b'buildah', b'umount', b'$PREFIX-20220902-151551129713-cont']
STDOUT
STDERR
RC CODE 0
RC 0 STDOUT b'' STDERR b''
The full traceback is:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/ansible/executor/task_executor.py", line 158, in run
res = self._execute()
File "/usr/lib/python3.10/site-packages/ansible/executor/task_executor.py", line 605, in _execute
result = self._handler.run(task_vars=variables)
File "/usr/lib/python3.10/site-packages/ansible/plugins/action/copy.py", line 519, in run
module_return = self._copy_file(source_full, source_rel, content, content_tempfile, dest, task_vars, follow)
File "/usr/lib/python3.10/site-packages/ansible/plugins/action/copy.py", line 263, in _copy_file
dest_status = self._execute_remote_stat(dest_file, all_vars=task_vars, follow=follow, checksum=force)
File "/usr/lib/python3.10/site-packages/ansible/plugins/action/__init__.py", line 774, in _execute_remote_stat
mystat = self._execute_module(module_name='ansible.legacy.stat', module_args=module_args, task_vars=all_vars,
File "/usr/lib/python3.10/site-packages/ansible/plugins/action/__init__.py", line 1021, in _execute_module
self._transfer_data(remote_module_path, module_data)
File "/usr/lib/python3.10/site-packages/ansible/plugins/action/__init__.py", line 500, in _transfer_data
self._transfer_file(afile, remote_path)
File "/usr/lib/python3.10/site-packages/ansible/plugins/action/__init__.py", line 477, in _transfer_file
self._connection.put_file(local_path, remote_path)
File "/home/myself/.ansible/collections/ansible_collections/containers/podman/plugins/connection/buildah.py", line 174, in put_file
shutil.copyfile(
File "/usr/lib64/python3.10/shutil.py", line 256, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: b'/home/myself/.local/share/containers/storage/overlay/4becb549defe1cc8b8a16e21e7e767d1e8e3cb9e4aa1b70d75cf63a4d42f975b/merged//tmp/ansible-tmp-1662124559.739739-172714-199418447811172/AnsiballZ_stat.py'
fatal: [$PREFIX-20220902-151551129713-cont]: FAILED! => {
"msg": "Unexpected failure during module execution.",
"stdout": ""
}
[...]
There was an error during execution: ansible-playbook execution failed: Command '['buildah', 'unshare', '--', 'ansible-playbook', '-c', 'buildah', '-i', '/tmp/abt2_2fh7j/inventory', '.build-playbook-20220902-151555027157-fqqkyweozo.yaml']' returned non-zero exit status 2.
You see I have left out some things above and also replaced PII with pseudonyms.
Other than that the buildah command that causes the stacktrace is actually not logged. That sucks and it'd be nice if that could be fixed. To see what is failing I have inserted the os.system line in .ansible/collections/ansible_collections/containers/podman/plugins/connection/buildah.py:
def put_file(self, in_path, out_path):
display.vvv("HELLO", host=self._container_id)
""" Place a local file located in 'in_path' inside container at 'out_path' """
super(Connection, self).put_file(in_path, out_path)
display.vvv("PUT %s TO %s" % (in_path, out_path), host=self._container_id)
if not self._mount_point or self.user:
rc, stdout, stderr = self._buildah(
"copy", [in_path, out_path])
if rc != 0:
raise AnsibleError(
"Failed to copy file from %s to %s in container %s\n%s" % (
in_path, out_path, self._container_id, stderr)
)
else:
real_out_path = self._mount_point + to_bytes(out_path, errors='surrogate_or_strict')
display.vvv("LALA ========== ", host=self._container_id)
os.system("echo 'PUT %s TO %s' > /tmp/jaja; ls -l %s >> /tmp/jaja" % (in_path, out_path, in_path))
This is in /tmp/jaja:
PUT /home/myself/.ansible/tmp/ansible-local-1723215q6r34bg/tmp1p33k746 TO /tmp/ansible-tmp-1662124559.739739-172714-199418447811172/AnsiballZ_stat.py
-rw-------. 1 root root 131419 2. Sep 15:16 /home/myslef/.ansible/tmp/ansible-local-1723215q6r34bg/tmp1p33k746
So in conclusion: I have no clue where shutil.copyfile gets the target path /home/myself/.local/share/containers/storage/overlay/4becb549defe1cc8b8a16e21e7e767d1e8e3cb9e4aa1b70d75cf63a4d42f975b/merged/ from that gets prefixed to the target file path /tmp/ansible-tmp-1662124559.739739-172714-199418447811172/AnsiballZ_stat.py. I assume the prefix path is some "random" hash and gets reconstructed every time anew. No idea how to proceed from here.
tpo commented
I am closing this bug report. After a new login session ansible-bender seems to behaving differently and the problem mentioned above does not occur any more.
However, in order to be able to execute ansible tasks reliably I had to set ansible_bender.cache_tasks: false in the vars: section in the playbook. Otherwise each run would fail in a different way.
Thanks a lot for ansible-bender!