Add --pre flag for ansible-galaxy client

[pabelanger]

SUMMARY

We publish semver collection (vyos.vyos:0.0.1-dev58) to ansible galaxy, however by default it seems ansible-galaxy client will not be able to find it. You must specific the version number directly to fetch it.

https://galaxy.ansible.com/vyos/vyos

EDIT:

This is working as expect, but it would be nice to Include:

--pre Include pre-release and development versions. By default, ansible-galaxy only finds stable versions.

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible-galaxy

ANSIBLE VERSION

ansible 2.9.1

CONFIGURATION

STEPS TO REPRODUCE

$ ansible-galaxy collection install vyos.vyos -p /tmp/11123
[WARNING]: The specified collections path '/tmp/11123' is not part of the configured Ansible collections paths '/home/pabelanger/.ansible/collections:/usr/share/ansible/collections'. The
installed collection won't be picked up in an Ansible run.

Process install dependency map
ERROR! Cannot meet requirement * for dependency vyos.vyos from source 'https://galaxy.ansible.com/api/'. Available versions before last requirement added: 
Requirements from:
	base - 'vyos.vyos:*'
(venv) [pabelanger@localhost openstack]$ ansible-galaxy collection install vyos.vyos:0.0.1-dev57 -p /tmp/11123
[WARNING]: The specified collections path '/tmp/11123' is not part of the configured Ansible collections paths '/home/pabelanger/.ansible/collections:/usr/share/ansible/collections'. The
installed collection won't be picked up in an Ansible run.

Process install dependency map
Starting collection install process
Installing 'vyos.vyos:0.0.1-dev57' to '/tmp/11123/ansible_collections/vyos/vyos'

EXPECTED RESULTS

vyos.vyos collection to be found and installed

ACTUAL RESULTS

[ansibot]

Files identified in the description:

• lib/ansible/galaxy

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[pabelanger]

Talking with people, I think what we want to add is the ability to pass --pre-release flag to ansible-galaxy, to have galaxy client return the latest version uploaded. This would work much like pip

[sivel]

For background and clarity, for the 2.9 release, we decided to not tackle this problem. Largely this is due to collections using semver (semver allows for pre-release versioning), and to keep from introducing another dependency, we handle release versions using distutils.version.StrictVersion, and reject all versions that do not adhere to StrictVersion (pre-releases are included in this).

As a result we have no way to sort pre-releases. Potentially we could switch to LooseVersion if --pre is specified.

[pabelanger]

thanks for info! Yah, I think LooseVersion on --pre makes sense to me.

[jborean93]

@pabelanger The trouble with using LooseVersion is that it doesn't strictly order things the way that semantic versioning would expect. So if we were to do that then we would be going against what people would expect with pre release versioning. The 3 options I see are;

1. Add a dep to Ansible for a semantic versioning library so we can order pre releases properly
2. Add an optional dep for the above and only allow --pre if that is present and importable
3. Implement our own semantic versioning parser

The first option is probably never going to happen, we are trying to reduce our dependencies as much as possible. The 2nd would lead to poor user experience as Python deps can be quite complex in some environment. The last option may be doable but quite a big ask, prerelease versioning rules for semantic versioning is quite complex.

[pabelanger]

I would suggest we look to another project, for example openstack, which has a good solution to this for python packages. They use version logic with in pbr (https://opendev.org/openstack/pbr/) for semantic versioning, I suspect we can leverage a lot of that logic if needed.

How that falls into 1,2 or 3 would still need to be decided.

FWIW: We are using pbr today to generate our semantic versioning numbers, works well.

[jborean93]

That would be option 3 as per https://opendev.org/openstack/pbr/src/branch/master/pbr/version.py. It remains to be seen if the ability to sort pre-release versions instead of an explicit opt in is what we want to have.

[ktdreyer]

I found this ticket from ansible/galaxy#2137

@jborean93 wrote above:

Implement our own semantic versioning parser

On first glance, I don't think this would be impossible, particularly if we use the code from python-semanticversion as inspiration. Here's the important regular expression (from semantic_version/base.py)

version_re = re.compile(r'^(\d+)\.(\d+)\.(\d+)(?:-([0-9a-zA-Z.-]+))?(?:\+([0-9a-zA-Z.-]+))?$')

We could build a really small SemanticVersion class that implements just enough of the features we need on the client. I have posted a work-in-progress patch at #65676 that does this for my own use-case.

[abadger]

Informational:

Currently the galaxy server will show the prereleases on the galaxy UI despite /usr/bin/ansible-galaxy not serving those releases.