Ansible Automation Platform "Missing required library \"pan-os-python\"." error
david-johnstone opened this issue · 14 comments
Describe the bug
Currently running Ansible Automation Platform AAP 2.2 which uses podman containers to execute templates and still receiving the same error as #324 despite having pan-os-python being installed in the container. Using latest collection from Galaxy paloaltonetworks-panos-3.0.0
Requesting reopening: #324
Error:
{
"pypi": "https://pypi.org/project/pan-python",
"syspath": [
"/tmp/ansible_panos_object_facts_payload_164rchcs/ansible_panos_object_facts_payload.zip",
"/usr/lib64/python36.zip",
"/usr/lib64/python3.6",
"/usr/lib64/python3.6/lib-dynload",
"/usr/lib64/python3.6/site-packages",
"/usr/lib/python3.6/site-packages"
],
"msg": "Missing required library "pan-python".",
Expected behavior
Library should be identified as being installed when calling panos_object_facts
Current behavior
When calling the panos_object_facts function from paloaltonetworks.panos, the collection doesn't identify pan-python despite it being installed in the container.
Possible solution
Unsure.
Steps to reproduce
Files required to create Podman container:
bindep.txt
gcc
python3-libselinux [platform:redhat]
python3 [platform:dpkg]
execution-environment.yaml
version: 2
dependencies:
galaxy: requirements.yml
python: requirements.txt
system: bindep.txt
additional_build_steps:
prepend: |
RUN pip3 install --upgrade pip setuptools
append:
- RUN ls -la /etc
requirements.txt
psutil
pandevice==0.14.0
certifi
chardet==3.0.4
idna==2.8
pan-os-python
pan-python
requests==2.22.0
urllib3==1.25.11
xmltodict==0.12.0
openpyxl
selinux
requirements.yml
collections:
paloaltonetworks.panos
community.general
Build command:
ansible-builder build --tag /ee_panorama_1_03 -v3
Error from Ansible Automation Platform Console:
{
"pypi": "https://pypi.org/project/pan-python",
"syspath": [
"/tmp/ansible_panos_object_facts_payload_164rchcs/ansible_panos_object_facts_payload.zip",
"/usr/lib64/python36.zip",
"/usr/lib64/python3.6",
"/usr/lib64/python3.6/lib-dynload",
"/usr/lib64/python3.6/site-packages",
"/usr/lib/python3.6/site-packages"
],
"msg": "Missing required library "pan-python".",
Container pan-os-python Details
Downloading https://galaxy.ansible.com/download/paloaltonetworks-panos-3.0.0.tar.gz to /home/runner/.ansible/tmp/ansible-local-1v3483z7d/tmpwwp4fwrg/paloaltonetworks-panos-3.0.0-xk3y1ej6
Installing 'paloaltonetworks.panos:3.0.0' to '/usr/share/ansible/collections/ansible_collections/paloaltonetworks/panos'
paloaltonetworks.panos:3.0.0 was installed successfully
Task Object
panos_object_facts
Any assistance would be greatly appreciated as we are currently migrating from Tower to AAP.
Screenshots
Context
Your Environment
- Collection: paloaltonetworks.panos
- Python: 3.6
- Ansible: Ansible Automation Platform 2.2 ansible>=2.12
- Latest versions used.
Hello,
Same issue here, any update?
Regards
None yet. We have a piece of automation that has stalled as a result of this bug.
There are two python libraries, both are required for the Ansible collection:
- pan-python
- pan-os-python
This issue's title is saying you're missing "pan-os-python", but the error message is actually saying "pan-python". So there's definitely some confusion here.
#324 was closed out because Ansible sometimes likes to look in places users don't expect, and python as a runtime environment is wonky as all get-out sometimes. In your error, due to the enhancements from #324, we know what the paths are that Ansible is looking in for both pan-python and pan-os-python. At the very least, pan-python is not in any of the paths mentioned in syspath in the error output.
Make sure you've installed both pan-python and pan-os-python in one of the paths mentioned in your syspath, probably "/usr/lib64/python3.6/site-packages" based on what I see.
One final thing to note: Ansible is moving towards a minimum python version of 3.8. We've had to update our CI runtime environments because some other the newer Ansible runtimes bomb out if you're not running 3.8. I would recommend to start moving in that direction when you finish dealing with the library location errors.
@shinmog Also running AAP 2.2.
I am having the exact same issue. But I'm running Python 3.8 and have all the required python modules installed. I don't even get proper error code now when invoking panos_op. I just get "Missing required libraries." I have dug through controller logs and the API and debug level 3 to try to get more information but I get nothing. I verified that all the packages are installed in the various syspath folders of Python3.8 in my execution environment container. I have racked my brains trying to get it to work, but I'm completely confused at this point as to why it isn't working.
Update: You can get around this bug by using a URI module to run the commands you want, like so:
- name: Gather system info
uri:
validate_certs: no
url: 'https://{{ ansible_host }}/api/?type=op&cmd=<show><system><info></info></system></show>&key={{ api_key }}'
return_content: yes
register: res1
ignore_errors: yes
The generic error of "Missing required libraries" only happens when you're missing a module specific library. For panos_op, that library is xmltodict. Unfortunately, module specific python library dependencies are checked before the generic ones (pan-os-python and pan-python), so you don't see what the python paths are that Ansible is looking for.
I'd recommend using a module different from panos_op, see if you get a library error or not, and if you do, you'll have the paths where Ansible is checking to ensure that dependencies are installed where Ansible is looking.
As an example, panos_address_object or something (gathered_filter was added in v2.11.0):
- name: 'Get foo objects'
panos_address_object:
provider: '{{ provider_dict }}'
state: 'gathered'
gathered_filter: 'name contains "foo"'@shinmog So, I appreciate your help, but I think it's possible some type of RHEL8 update broke the paloaltonetworks.panos module on AAP 2.2.
I got the paths for you from a simple module just panos_facts, which worked a few months ago without issue with no configuration changes. It all of the sudden stopped working in the last week or so, which just so happens to be after the patching window.
{
"pypi": "https://pypi.org/project/pan-python",
"syspath": [
"/tmp/ansible_panos_facts_payload_iekbe07f/ansible_panos_facts_payload.zip",
"/usr/lib64/python36.zip",
"/usr/lib64/python3.6",
"/usr/lib64/python3.6/lib-dynload",
"/usr/lib64/python3.6/site-packages",
"/usr/lib/python3.6/site-packages"
],
"msg": "Missing required library \"pan-python\".",
As you can see here, for some reason it's looking in Python 3.6, I have no idea why. It worked perfectly fine a month ago. I didn't set it to do that anywhere in the container. The syspath of Python on my execution environment contains absolutely no 3.6 references, it's all pointing to a newer version. As you can see from the screenshot I posted above. I even built another container to test in and directly installed the panos collection into the container instead of calling it from the project root in collections/requirements.yml. Neither resolved it.
I have been troubleshooting this for 3 days and cannot get it resolved. I have talked to other people who have no issues on regular community CLI ansible with the panos module, it's only an AAP problem.
I have reached out to a Redhat Architect about this, contacted the devs here, and reached out to RedHat support and still do not have a resolution.
So the only fix I've found, is to set the interpreter manually at the template or inventory level:
---
ansible_python_interpreter: /usr/bin/python3
For awareness for anyone else. I also tried adding this to the ansible.cfg for the container build file:
[defaults]
interpreter_python=/usr/bin/python3
I couldn't get the ansible.cfg file to circumvent 3.6 for whatever reason, even after specifying it in the ansible.cfg, so YMMV, but the top yaml variable works in getting around this issue.
that ansible.cfg file must be in the root of your github repo.
Indeed, I tested that and it's working.
You could also take that one-liner ansible.cfg and copy it into the /etc/ansible/ansible.cfg directory on your container, and it would probably also work, but I haven't tested that yet.
Since you're running AAP 2.2 the same as @Dixie9000 it's likely you have the same issue. If that's the case, then his fix will work for you as well.
I've added some documentation about "ansible_python_interpreter" to the docs for others, this will be on the main documentation page to help others if they have the same issue.
Furthermore, panos_op now fails gracefully if xmltodict is not present.
This will be present in the next collection release.
I might be wrong here to comment but the mentioned error I am able to resolved on both of the below versions. This may be quick workaround
#ansible --version
ansible [core 2.14.1]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.11.1 [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] (/usr/local/bin/python3.11)
jinja version = 3.1.2
libyaml = True
#ansible --version
ansible [core 2.13.7]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.8.12 [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)]
jinja version = 3.1.2
libyaml = True
#Solution:
1- You need to install pan-python & pan-os-python using pip for the current version of python which you are seeing ansible version output of yours then finally you have to run the below command & error should be gone.
pip3.6 install pan-python
pip3.6 install pan-os-python