KeyError 'id' when trying to change the image on an extant container.
the-maldridge opened this issue · 2 comments
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)*
/kind bug
Description
When attempting to change the image of an extant container the play crashes with the following error:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'id'
fatal: [netbox-phx1-0000]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 107, in <module>\n File \"<stdin>\", line 99, in _ansiballz_main\n File \"<stdin>\", line 47, in invoke_module\n File \"/usr/lib/python3.9/runpy.py\", line 210, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.9/runpy.py\", line 97, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.9/runpy.py\", line 87, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/modules/podman_container.py\", line 1093, in <module>\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/modules/podman_container.py\", line 1088, in main\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1734, in execute\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1623, in make_started\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1403, in different\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1324, in is_different\n File \"/tmp/ansible_containers.podman.podman_container_payload_5vnyw92k/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 963, in diffparam_image\nKeyError: 'id'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
Steps to reproduce the issue:
I'm not actually sure since this worked for over a year, then spontaneously stopped working some time in the last 60 days. Nothing beyond routine Apt updates has been applied in that time.
Describe the results you received:
I received a module crash, and the containers on the host appear to have been left unchanged.
Describe the results you expected:
I expect in check mode to get a delta of what will be changed, and in normal mode to have the containers changed on the host to match the defined state.
Additional information you deem important (e.g. issue happens only occasionally):
This happens reliably, but without any clear indication of what has changed.
Version of the containers.podman collection:
Either git commit if installed from git: git show --summary
Or version from ansible-galaxy if installed from galaxy: ansible-galaxy collection list | grep containers.podman
$ ansible-galaxy collection list | grep containers.podman
containers.podman 1.10.1
containers.podman 1.10.3
Output of ansible --version:
$ ansible --version
ansible [core 2.14.5]
config file = /home/maldridge/.ansible.cfg
configured module search path = ['/home/maldridge/src/configMgmt/ansible/library']
ansible python module location = /home/maldridge/bzenv/lib/python3.11/site-packages/ansible
ansible collection location = /home/maldridge/.ansible/collections:/usr/share/ansible/collections
executable location = /home/maldridge/bzenv/bin/ansible
python version = 3.11.4 (main, Jun 8 2023, 02:02:15) [GCC 12.2.0] (/home/maldridge/bzenv/bin/python3)
jinja version = 3.1.2
libyaml = True
Output of podman version:
$ podman version
Version: 3.0.1
API Version: 3.0.0
Go Version: go1.15.15
Built: Thu Jan 1 00:00:00 1970
OS/Arch: linux/amd64
Output of podman info --debug:
host:
arch: amd64
buildahVersion: 1.19.6
cgroupManager: cgroupfs
cgroupVersion: v2
conmon:
package: 'conmon: /usr/bin/conmon'
path: /usr/bin/conmon
version: 'conmon version 2.0.25, commit: unknown'
cpus: 32
distribution:
distribution: debian
version: "11"
eventLogger: journald
hostname: netbox-phx1-0000
idMappings:
gidmap:
- container_id: 0
host_id: 1641
size: 1
uidmap:
- container_id: 0
host_id: 1641
size: 1
kernel: 5.10.0-25-amd64
linkmode: dynamic
memFree: 60207534080
memTotal: 67351461888
ociRuntime:
name: crun
package: 'crun: /usr/bin/crun'
path: /usr/bin/crun
version: |-
crun version 0.17
commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
os: linux
remoteSocket:
exists: true
path: /run/user/1641/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
selinuxEnabled: false
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 0
swapTotal: 0
uptime: 192h 6m 51.04s (Approximately 8.00 days)
registries: {}
store:
configFile: /home/maldridge/.config/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: vfs
graphOptions: {}
graphRoot: /home/maldridge/.local/share/containers/storage
graphStatus: {}
imageStore:
number: 0
runRoot: /run/user/1641/containers
volumePath: /home/maldridge/.local/share/containers/storage/volumes
version:
APIVersion: 3.0.0
Built: 0
BuiltTime: Thu Jan 1 00:00:00 1970
GitCommit: ""
GoVersion: go1.15.15
OsArch: linux/amd64
Version: 3.0.1Package info (e.g. output of rpm -q podman or apt list podman):
podman/oldstable,now 3.0.1+dfsg1-3+deb11u4 amd64 [installed]
Playbook you run with ansible (e.g. content of playbook.yaml):
I am unable to post the entire playbook at this time. This is the failing task:
- name: Provision Netbox Core Service
containers.podman.podman_container: &netbox
name: netbox-core
pod: netbox
image: "{{ netbox_image }}"
state: started
restart_policy: always
env:
<output elided>Command line and output of ansible run with high verbosity
Please NOTE: if you submit a bug about idempotency, run the playbook with --diff option, like:
ansible-playbook -i inventory --diff -vv playbook.yml
Excerpt of the failing task provided. I am unable to post the entire playbook at this time.
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'id'
fatal: [netbox-phx1-0000]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 107, in <module>\n File \"<stdin>\", line 99, in _ansiballz_main\n File \"<stdin>\", line 47, in invoke_module\n File \"/usr/lib/python3.9/runpy.py\", line 210, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.9/runpy.py\", line 97, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.9/runpy.py\", line 87, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/modules/podman_container.py\", line 1093, in <module>\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/modules/podman_container.py\", line 1088, in main\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1734, in execute\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1623, in make_started\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1403, in different\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1324, in is_different\n File \"/tmp/ansible_containers.podman.podman_container_payload_0mhsjyw_/ansible_containers.podman.podman_container_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 963, in diffparam_image\nKeyError: 'id'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
Additional environment details (AWS, VirtualBox, physical, etc.):
Physical server, running debian 11. Fully updated prior to verifying behavior for this bug report.
What happens if you do podman image inspect {{ netbox_image }} in cli?
After some discussion in IRC it was determined that the underlying storage layer was broken, and was repaired by doing a podman system reset.