redfish_command fails with "UnboundLocalError: local variable 'systems_service' referenced before assignment"
vcrhonek opened this issue · 5 comments
Hello,
I'm trying to use Redfish Ansible module with virtual Redfish BMC from sushy-tools ([1]) to manage power of libvirt-managed VMs on localhost (because of testing Redfish related Ansible role). I'm able to power on the machine, but powering it off fails with following error:
$ ansible -vvv -m redfish_command -a "baseuri=localhost:8000 user=foo password=bar category=Systems command=PowerForceOff" localhost
ansible 2.7.5
config file = /home/vcrhonek/.ansible.cfg
configured module search path = [u'/home/vcrhonek/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.15 (default, Oct 15 2018, 15:24:06) [GCC 8.1.1 20180712 (Red Hat 8.1.1-5)]
Using /home/vcrhonek/.ansible.cfg as config file
/etc/ansible/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/etc/ansible/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /etc/ansible/hosts inventory source with ini plugin
META: ran handlers
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: vcrhonek
<127.0.0.1> EXEC /bin/sh -c 'echo ~vcrhonek && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "echo /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093" && echo ansible-tmp-1547473703.69-132881721848093="echo /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/remote_management/redfish/redfish_command.py
<127.0.0.1> PUT /home/vcrhonek/.ansible/tmp/ansible-local-26240Scgi5j/tmpBI3AOS TO /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/ /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py", line 113, in
_ansiballz_main()
File "/home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py", line 105, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py", line 48, in invoke_module
imp.load_module('main', mod, module, MOD_DESC)
File "/tmp/ansible_redfish_command_payload_aBLJy8/main.py", line 258, in
File "/tmp/ansible_redfish_command_payload_aBLJy8/main.py", line 219, in main
File "/tmp/ansible_redfish_command_payload_aBLJy8/ansible_redfish_command_payload.zip/ansible/module_utils/redfish_utils.py", line 133, in _find_systems_resource
UnboundLocalError: local variable 'systems_service' referenced before assignment
localhost | FAILED! => {
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File "/home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py", line 113, in \n _ansiballz_main()\n File "/home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py", line 105, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File "/home/vcrhonek/.ansible/tmp/ansible-tmp-1547473703.69-132881721848093/AnsiballZ_redfish_command.py", line 48, in invoke_module\n imp.load_module('main', mod, module, MOD_DESC)\n File "/tmp/ansible_redfish_command_payload_aBLJy8/main.py", line 258, in \n File "/tmp/ansible_redfish_command_payload_aBLJy8/main.py", line 219, in main\n File "/tmp/ansible_redfish_command_payload_aBLJy8/ansible_redfish_command_payload.zip/ansible/module_utils/redfish_utils.py", line 133, in _find_systems_resource\nUnboundLocalError: local variable 'systems_service' referenced before assignment\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
Any idea what's wrong? (I know that in the end it would need "ForceOff" instead "PowerForceOff" to work correctly, but that could be easily added to the module.)
It also seems that there's no way to select particular VM (if there's more than one). For example in case of "PowerOn" the command 'ansible -m redfish_command -a "baseuri=localhost:8000 user=foo password=bar category=Systems command=PowerOn" localhost' just powers on one of the VMs found in 'https://localhost:8000/redfish/v1/Systems/'. Am I correct? If so, why?
@vcrhonek Please refresh the code you are using, there was a commit last week that addresses the UnboundLocalError: local variable 'systems_service' referenced before assignment error.
This code was not tested against BMC emulators, so it's hard to say what is going on. There is currently a PR in progress to address cases when there is more than one system, although I don't know if that would apply here. Is each VM considered a different Redfish system?
@vcrhonek Please refresh the code you are using, there was a commit last week that addresses the
UnboundLocalError: local variable 'systems_service' referenced before assignmenterror.
Thanks! I can confirm that updating redfish_utils.py in module_utils directory fixes the error.
On the other hand, the command still doesn't succeed:
"msg": "ComputerSystem's Members array is either empty or missing"
But it seems that as soon as the machine is powered on, it disappears from Members list at https://localhost:8000/redfish/v1/Systems/, so this is in my opinion valid error and probably bug in the emulator.
This code was not tested against BMC emulators, so it's hard to say what is going on. There is currently a PR in progress to address cases when there is more than one system, although I don't know if that would apply here. Is each VM considered a different Redfish system?
Yes, I think so. I have two VMs at the moment (named 'fedora27' and 'rhel7.4'). I'm able for example to turn on/off different machines directly with:
$ curl -k -d '{"ResetType":"On"}' -H "Content-Type: application/json" -X POST https://localhost:8000/redfish/v1/Systems/rhel7.4/Actions/ComputerSystem.Reset
$ curl -k -d '{"ResetType":"On"}' -H "Content-Type: application/json" -X POST https://localhost:8000/redfish/v1/Systems/fedora27/Actions/ComputerSystem.Reset
$ curl -k -d '{"ResetType":"ForceOff"}' -H "Content-Type: application/json" -X POST https://localhost:8000/redfish/v1/Systems/fedora27/Actions/ComputerSystem.Reset
$ curl -k -d '{"ResetType":"ForceOff"}' -H "Content-Type: application/json" -X POST https://localhost:8000/redfish/v1/Systems/rhel7.4/Actions/ComputerSystem.Reset
But using redfish_command, the first PowerOn command turns on fedora27 machine, if I issue the same command again, then rhel7.4 is turned on (at least now - I'm not sure whether this order is deterministic). That's why I wonder why there's no mechanism in redfish_command to distinguish somehow between systems.
OK, I've investigated it further and this is what I've discovered, just for the record:
a) The emulator uses listDefinedDomains() libvirt API call, which returns only guest that are not running. It should use listAllDomains() call. For more info see [1] and [2].
b) With that fixed, Redfish Ansible module is able to power on/off the first guest in the list returned from libvirt (at least most of the time, I think I've noticed power on of the second machine once, but I'm not sure with that).
Closing the issue and looking forward to finished PR to address cases when there is more than one system:)
Thanks!
[1] https://www.redhat.com/archives/libvir-list/2016-June/msg00013.html
[2] https://libvirt.org/docs/libvirt-appdev-guide-python/en-US/html/libvirt_application_development_guide_using_python-Guest_Domains-Listing_Domains.html
Thank you for reporting and documenting @vcrhonek. For your reference, ansible/ansible#47108 is the bug for multiple systems. There is a PR open for it (scroll to the bottom to find it), though it's in the early stages and requires rework.
Thanks for the link!