Changes impacting Collection Contributors and maintainers

Question

Changes impacting Collection Contributors and maintainers

gundalow opened this issue 4 years ago · 119 comments

NO LONGER IN USE

Please subscribe to ansible-collections/news-for-maintainers

Details of what's changed

In accordance with the Community decision, we have created the news-for-maintainers repository for announcements of changes impacting collection maintainers (see the examples) instead of this issue.

To keep yourself well-informed and, therefore, things in your collection working, please subscribe to the repository by using the Watch button in the upper right corner on the repository's home page.
If you do not want to get notifications about related discussions, please subscribe only to Issues.
Please read the brief guidelines on how the repository should be used.
Please avoid unnecessary discussions in issues, use the Discussions feature. Every comment posted will notify a lot of folks!

Answer 1 · 2020-04-10T15:11:03.000Z

Remove `tests/sanity/requirements.txt` from collections

The file tests/sanity/requirements.txt was put into collections by mistake, it should be removed.

See #43 for details

Answer 2 · 2020-04-10T15:11:07.000Z

Update collection tests from RHEL 7.6 to 7.8

Update CI to run against RHEL 7.8/

See #44 for details

Answer 3 · 2020-04-14T12:53:18.000Z

Changelog process for collections

Feedback needed

We have a continuing need for changelogs (or release notes) to cover:

ansible/ansible - currently has a working solution using changelog fragments
collections - two options mentioned so far are reno and towncrier
Ansible Community Distribution (ACD) - a bundled package that includes the trimmed down ansible/ansible and a set of curated collections that will mimic the current user experience of Ansible with all batteries included (2.9 level of content). ACD has no proposed solution yet, but whatever is used by collections has to provide fragments that can be drawn up into an ACD changelog/release note.

There are various proposals outlined #18 Please comment and vote for preferred solution.

Answer 4 · 2020-04-30T14:36:09.000Z

Update collection tests from RHEL 8.1 to 8.2

Update CI to run against RHEL 8.2.

See #54 for details.

Answer 5 · 2020-05-05T15:57:02.000Z

Remove ANSIBLE_METADATA

Use meta/routing.yml for deprecated and removed status.

Do collection owners want a DOCUMENTATION field to cover preview vs stable status for a module??

See #57 for details.

Answer 6 · 2020-05-07T07:22:15.000Z

ansible-test (validate-modules) now checks for correct module deprecation

All, we've just merged ansible/ansible#68646
This means that deprecated modules and plugins

MUST NOT have a leading _
Deprecations are defined by meta/routing.yml (see community.general for an example)

Call to action

Please do a CI run and if anything fails please update as per ansible-collections/community.general#195

Answer 7 · 2020-05-11T06:59:24.000Z

Update collection tests to test on Fedora 32

Update CI to run against Fedora 32.

See #59 for details.

Answer 8 · 2020-05-13T11:32:34.000Z

Dropping `ANSIBLE_METADATA`

In Ansible 2.9 and earlier we had the concept of ANSIBLE_METADATA, to express support level and deprecation. With the move to Collections, and support being defined if a Collection is in Galaxy vs Automation Hub, ANSIBLE_METADATA is no longer needed.See #57 for background

Call to action

Please bulk remove the ANSIBLE_METADATA section from all modules in your collection and reference this issue in the PR so we can track what's been done.

community.general ansible-collections/community.general#339
community.network ansible-collections/community.network#35
community.aws ansible-collections/community.aws#70?
your collection...

Answer 9 · 2020-05-13T21:46:05.000Z

Sanity tests for removed_in_version and deprecated_aliases

The devel version of ansible-test now validates removed_in_version and deprecated_aliases (ansible/ansible#66920). For collections, these are assumed to be collection version numbers, i.e. in semver format. Version numbers not in that format - for example verison numbers taken from ansible/ansible during the migration - are not valid semver versions and thus fail validation.

Call to action

Please ensure that deprecation version numbers are valid collection version numbers.
Alternatively, if you do not have a deprecation / versioning plan yet (and thus no idea how to rewrite Ansible versions to collection versions), temporarily update tests/sanity/ignore-2.10.txt as in ansible-collections/community.general#337.

Answer 10 · 2020-05-14T06:04:38.000Z

Next Virtual Contributors Summit

We're going to set the date for our next Contributor Summit soon, please let us know what works best for you! https://doodle.com/poll/d8ttii3zpv5a2ic5

Answer 11 · 2020-05-15T04:35:52.000Z

ansible-test default test container updated

The devel version of ansible-test has been upgraded to use the latest version of the default test container.

This may result in new test failures.

There are several issues that are likely to occur:

Syntax errors on Python 3.9.
PEP8 sanity test failures.
Missing requirements in unit or integration tests.

Call to action

If you are using the --docker or --docker default options for testing your collection, re-run all tests to check for new issues.
If you find that Python requirements are missing, add them to the appropriate test requirements file in the collection. Different ansible-test commands use different files:
- tests/unit/requirements.txt
- tests/integration/requirements.txt
- tests/integration/integration.requirements.txt
- tests/integration/windows-integration.requirements.txt
- tests/integration/network-integration.requirements.txt

Answer 12 · 2020-05-18T22:21:45.000Z

Remove "smoke tests" from migrated collections

Before migration there were test groups configured for ansible/ansible on Shippable that used "smoke tests" as an optimization when all tests were triggered. These groups were used for tests such as Windows, AWS, etc.

These smoke tests groups do not make sense for collections, and their presence could result in too few (or possibly no) tests running on those collections when certain changes are made.

Call to action

If you are the maintainer of a collection which was migrated from ansible/ansible and are using copies of the original Shippable scripts (tests/utils/shippable/*.sh) you should remove smoke test configuration from your collection:

Remove CI groups in tests/integration/targets/*/aliases which contain smoketest.
Update scripts in tests/utils/shippable/*.sh to remove smoketest logic. Look for the string smoketest as well as for use of the --changed-all-target option for ansible-test.

Answer 13 · 2020-06-07T14:36:31.000Z

Next Ansible PR review day (Collections)

The next PR day will be Wednesday 17th June 2020 from 1400UTC

See ansible/community#407 (comment) for further info. Hope to see you there!

Answer 14 · 2020-06-10T03:29:50.000Z

Specifying test requirements for collections

The default test container for ansible-test has been reduced from a compressed size of about 1.6 GB to 700 MB. This was partially accomplished by removing pre-installed Python packages that are not required by ansible-test.

Some collection unit and integration tests may have been depending on those pre-installed requirements. As a result, some tests may now fail or be skipped.

Call to action

If you are the maintainer of a collection and are using ansible-test to run unit or integration tests, make sure to declare Python test requirements in your collections:

tests/integration/requirements.txt
tests/unit/requirements.txt

Separate requirements can be used for each type of integration test if required:

tests/integration/integration.requirements.txt
tests/integration/windows-integration.requirements.txt
tests/integration/network-integration.requirements.txt

When any of these files are present, ansible-test will use pip to install them if the --docker, --venv or --requirements options are used.

Answer 15 · 2020-06-16T18:27:23.000Z

Freeze and bump of devel version to 2.11.0.dev0 requires tests/sanity/ignore-2.11.txt

When today (AFAIK) ansible-base's stable-2.10 branch will be forked from devel, and devel's version bumped to 2.11.0.dev0, ansible-test sanity will look for ignore entries in tests/sanity/ignore-2.11.txt and no longer in tests/sanity/ignore-2.10.txt. (The ansible-test version from stable-2.10 will use that file.)

So in case you have entries in tests/sanity/ignore-2.10.txt and test with devel, sanity tests will start failing until you copy tests/sanity/ignore-2.10.txt to tests/sanity/ignore-2.11.txt.

Answer 16 · 2020-06-16T20:16:50.000Z

Move changelog and porting guide entries from ansible/ansible to your collection

The ansible/ansible existing 2.10 porting guide has entries that really belong with the individual collections. Similarly, changelog fragments are being removed from ansible/ansible if they are not for code within the ansible/ansible repo.

Call to Action

Please move any changelogs or porting guide entries from ansible/ansible to your collection if you haven't already.

See https://github.com/ansible-community/antsibull-changelog for how to use the antsibull-changelog tool to generate your collection level changelog.rst file (which will include a porting guide section based on the new fragment types described here https://github.com/ansible-community/antsibull-changelog/blob/master/docs/changelogs.rst). This is the recommended approach so that your changelog/porting guide information will feed up cleanly to a 2.10 ansible (aka ACD) changelog and porting guide set of documents.
See ansible-collections/community.crypto#67 as an example of moving porting guide entries.
See https://github.com/ansible/ansible/pulls?q=is%3Apr+reap+in%3Atitle+is%3Aclosed and ansible/ansible#68498 for changelog fragments that have already been removed from ansible/ansible.

Answer 17 · 2020-06-16T20:38:35.000Z

I cannot edit the above comment, however please see ansible/ansible#68498 as well for removed changelog fragments (it was the first and largest)

Answer 18 · 2020-06-17T20:54:03.000Z

Deprecation in collections (and ansible-base)

Here's a summary of how deprecation by version or by date works on various levels. The collection name used in ansible-base (i.e. ansible/ansible) is ansible.builtin.

Module/plugin deprecation:
- deprecated entry in DOCUMENTATION with:
  - why, alternative (required)
  - either removed_in: <version> or removed_at_date: <date>
  - (removed_from_collection is automatically inserted if not present)
- deprecation entry in meta/runtime.yml with:
  - warning_text (optional)
  - either removal_version: <version> or removal_date: <date>
Plugin option deprecation in DOCUMENTATION (for option itself, for ini entry, for vars entry, for env entry)
- why (required)
- alternative (optional)
- either version: <version> or date: <date>
- (collection_name is automatically inserted if not present)
Module argument spec option deprecation:
- either removed_in_version="<version>" or removed_at_date="<date>"
- removed_from_collection="<namespace.name>" (required)
Module argument spec alias deprecation:
- name="<alias>" (required)
- collection_name="<namespace.name>" (required)
- either version="<version>" or date="<date>"
Display.deprecated() and AnsibleModule.deprecate() calls
- first positional argument msg (required)
- either version="<version>" or date="<date>"
- collection_name="<namespace.name>" (required)

Answer 19 · 2020-06-17T20:54:40.000Z

Also, stable-2.10 has been branched.

Answer 20 · 2020-06-19T23:03:24.000Z

2 things I've come across in the Windows collections that is probably a good idea for others to check

Requirements files

The tests/unit/requirements.txt file generated from migration had a lot of packages that are not needed for every collection
Would be a good idea to go through the collection to make sure it only contains the ones that are needed
- this reduces test time
- also reduces the chance of version incompatibilities when trying to install on older Python versions

check_matrix.py

The tests/utils/shippable/check_matrix.py if copied from ansible/ansible would have the wrong repo to check in the code
- This makes the script useless as it won't perform the check
- The line here needs to be edited to the new repo location just like community.general has done

Answer 21 · 2020-06-23T20:40:29.000Z

updating M() and :seealso: module references in DOCUMENTATION blocks

the M() and :seealso: sections in each module DOCUMENTATION block need to use FQCN.

Call to action

Please update M() and :seealso: to include FQCN.

Examples

For modules in a collection: M(community.general.apt_rpm)
For modules in `ansible-base: M(ansible.builtin.yum)

Answer 22 · 2020-07-01T12:35:38.000Z

AnsibleFest call for papers is open

As you may have heard, we have decided to make AnsibleFest 2020 a virtual experience. We hope to use this opportunity to collaborate, connect, and chat with more automation fans than ever before. It is exciting to think about how many more people will be able to join in on the automation conversation.

AnsibleFest Virtual Experience will be a free, immersive, multi day event on 13-14 October, 2020, that will deliver empowering content from customer keynotes, breakout sessions, access to Ansible experts, and more.

This is a great opportunity to share your automation story with thousands of open source enthusiasts and technology experts.

On Monday 12th October (to be confirmed) we host another virtual Contributors Summit as part of AnsibleFest, more details on that closer to the time.

Call to action

We'd love to hear from the community, especially those like yourselves that have started their journey with collections.

Call for proposals is now open!

Submit your automation story here by 15th July 2020.

Examples

You can see previous videos and slides here

Answer 23 · 2020-07-22T19:55:17.000Z

`version_added` in collections

Please note that version_added in a collection always refers to the collection's version numbers (in particluar, they must be of the form x.y.z)! So please do not use version_added: "2.10" in collections, but instead put version_added: <version of your collection the feature was added> in there.

(If you absolutely want to refer to a version of another collection, you can add version_added_collection: namespace.name next to version_added. The name of ansible-base is ansible.builtin.)

Answer 24 · 2020-07-23T19:07:23.000Z

Collection module docs now on docs.ansible.com for review

Go to https://docs.ansible.com/ansible/2.10/collections/index.html
Verify your collection docs look valid.
See https://github.com/ansible-community/antsibull/issues for a list of known issues and enhancements we are already tracking. Before you open an issue, please drop into the #ansible-docs IRC channel to discuss/troubleshoot any problems you are seeing.

Known issues and Important Caveats

The docs reflect the version of each collection that is in the current Ansible 2.10 Alpha (alpha3). We will update after each alpha release to pick up whatever collection version is in that release.
Boolean default is showing 0. This is fixed and will be updated in the docs soon.
community.general module docs are not present yet. This should be fixed soon as well.

Answer 25 · 2020-07-28T16:29:26.000Z

`version_added` in collections is now better validated (modules only)

ansible/ansible#70869 fixed validation of version_added for modules in collections: now also the top-level version_added is checked, and the correct error codes are returned when version_added fails validation for (sub-)options and (sub-)return values.

Answer 26 · 2020-07-30T20:00:24.000Z

We've published the tentative release schedule for the ansible-2.10 package: https://groups.google.com/forum/#!topic/ansible-devel/srweNQ92pJw Important dates from that schedule:

2020-08-18: Ansible 2.10 Beta freeze.
2020-09-08: Ansible 2.10 Final freeze/rc1

Please see the full post to know what's expected by those dates.

In the future this information will live on docs.ansible.com.

Answer 27 · 2020-07-30T20:14:39.000Z

The 2020-08-18 date is of particular importance. This means that by that day, you MUST have released 1.0.0 for it to appear in Ansible 2.10. If you only have a 0.x.y release by then, we will not automatically include a newer version since according to semver, anything can happen and no stability guarantees are made. So please make sure that you released 1.0.0 by then.

Answer 28 · 2020-08-07T19:30:20.000Z

We've just introduced an Alpine3 distro-test-container for ansible/ansible, and included it our CI matrix. This container may be of use in collections as well. One important note is that we tried to not install many packages that replace functionality in BusyBox, so we get a better test matrix against BusyBox.

https://quay.io/repository/ansible/alpine3-test-container

Answer 29 · 2020-08-16T11:31:41.000Z

Since I've seen that in quite a few collections now: if you use ansible-test sanity --docker -v --color --python 3.6 for running sanity tests (and nothing else), you must remove the --python 3.6 part! The parameter causes that several tests are not run (like compile / import tests for Python versions that are not 3.6). Only use the --python parameter to ansible-test sanity --docker if you know exactly what it does, and you are sure that you are calling ansible-test sanity multiple times to really cover all tests!

Answer 30 · 2020-08-27T18:10:59.000Z

Minor change to the release schedule. We're going to leave feature freeze for ansible-2.10.0 development until ansible-2.10.0beta1 is released next week. If you have features you still want to get in to your collection in time for 2.10.0, you still have time to upload them to galaxy.ansible.com. If this causes you issues, please let me know. (abadger1999 on irc.freenode.net, tkuratom redhat.com for email)

Details: https://groups.google.com/forum/?oldui=1#!topic/ansible-devel/uhHDkaWGDys

Answer 31 · 2020-09-05T01:40:38.000Z

Some more changes to the pre-release schedule. These are due to the one week slip of ansible-base-2.10.1: https://groups.google.com/forum/#!topic/ansible-devel/618bSHJ7K64

The new dates would be:

08-Sep-2020: ansible-2.10.0b2
15-Sep-2020: ansible-2.10.0rc1 (moved from 10-Sep)
17-Sep-2020: IRC meeting where any blockers should be proposed and discussed.
- Add blockers to the agenda: ansible/community#539
- If at all possible, if you propose a blocker attend the meeting itself at 18:00UTC in the #ansible-community channel on irc.freenode.net.
22-Sep-2020: If no blockers are found, ansible-2.10.0 final will release as previously scheduled.

Answer 32 · 2020-09-10T15:50:34.000Z

Some more information on the IRC meeting to decide on blockers for the Ansible-2.10.0 release:

If you find something major enough to block the Ansible release, please add a comment on the Community Meeting Agenda: ansible/community#539 asking us to consider whether the issue is a blocker. If the issue has a ticket open elsewhere, linking to the ticket will be appreciated.
If at all possible, attend the IRC meeting on Thursday, September 17th, 2020, 18:00 UTC to state your case as to why the issue should block the release. The meeting takes place in the #ansible-community channel on irc.freenode.net. If you are new to irc, you can use their online webchat to access freenode: https://webchat.freenode.net/ You'll need to register to participate in our channels: https://freenode.net/kb/answer/registration I highly recommend registering and testing that you can speak in the #ansible-community channel prior to the meeting.
We'll decide at the meeting whether to slip the Ansible-2.10.0 final release for any of the listed blockers.

Important: If the Meeting Agenda does not contain any comments listing potential blockers when the meeting starts on Thursday, we'll end the meeting early. We will wait for five minutes or so to see if anyone proposes a blocker at the start of the meeting itself but otherwise we'll cut the meeting short. So please, if you have a candidate blocker; get your issue listed on the Community Agenda ASAP.

Answer 33 · 2020-09-12T06:25:03.000Z

Changes to `ansible-galaxy login` for role/collection publishers

If you use the ansible-galaxy CLI to publish roles or collections to Galaxy, you may care about this. The GitHub API that underlies the ansible-galaxy login command is being removed in November. Without it, users doing role or collection publish operations via the Galaxy CLI will need to find their Galaxy token interactively in a browser and pass it to the CLI with --token (as the ability to generate a token from the CLI will no longer function). We're not sure what people do today or how much this option is used out in the world, so please read and weigh in at ansible/ansible#71628 in the next few days if you have an opinion.

Answer 34 · 2020-09-25T20:49:22.000Z

macOS 10.15 testing image now available in CI

It is now possible to test against macOS 10.15 using anisble-test >=2.10.0. The existing OS X 10.11 images are still available.

ansible-test integration --remote macos/10.15 [test]
ansible-test shell --remote macos/10.15

The macOS testing is now being done on newer hosts, so test execution for macOS 10.15 and OS X 10.11 should be much faster now.

Answer 35 · 2020-09-28T06:11:38.000Z

If your collection's integration tests use a copy of prepare_http_tests, you want to look at https://github.com/ansible/ansible/pull/71841/files for how to adjust it to run with macOS 10.15. That PR might also be useful for other changes needed.

Answer 36 · 2020-09-29T20:38:47.000Z

`ansible-test sanity` validates more semantic versioning properties for collections

ansible/ansible#71679 added more validation of version_added and removal versions (for deprecations) in collections so that the versions mentiones there satisfy semantic versioning:

features can not be added in patch releases (i.e. version_added versions must be minor or major releases);
API cannot be changed backwards incompatibility in minor and patch releases (i.e. removal versions for deprecations must be major releases).

Answer 37 · 2020-11-04T19:50:41.000Z

Please make sure to not include unwanted artifacts in your collection release on galaxy!

Common problems in the past have been:

venvs
tests/output/ directory

Please download the tarball of the latest release of your collection to check whether it includes something that shouldn't be there, if you're not sure.

Answer 38 · 2020-11-19T13:26:36.000Z

Ansible-3.0.0 Schedule and Preview of 4.0.0 Schedule

In yesterday's public Ansible Community Meeting the community signed off on the Ansible-3.0.0 release schedule.

Background

Package ansible-core (previously known as ansible-base): The Engine plus the builtin plugins
Package ansible 2.10 The ~70 collections that hold the modules in ansible-2.9. Depends on ansible-core

The rest of this document is about the ansible package

TL;DR

ansible package will move to semver (MAJOR.MINOR.PATCH)
The next major ansible release will be 3.0.0, no ansible 2.11 release
See below for proposed release dates

For full details click here

Answer 39 · 2020-12-15T19:20:28.000Z

New sanity check for deprecation / tombstone dates / versions in meta/runtime.yml

ansible/ansible#72625 added validation for deprecation and tombstone dates and versions in meta/runtime.yml. This should stop you from accidentally forgetting a deprecated redirect scheduled for removal, or adding an invalid removal version (removals must must happen in major releases according to semver).

Answer 40 · 2021-01-08T05:15:39.000Z

ansible-test no longer includes virtualenv on FreeBSD or RHEL instances for Python 3.x

ansible-test integration has been updated to no longer pre-install the virtualenv package on Python 3.x for FreeBSD or RHEL instances provisioned with the --remote option.

On Python 3.x, it is recommended that you do not install the virtualenv package as part of your tests. Instead, when running tests on Python 3.x, use the Python built-in venv module.

Below are recommendations on how to handle potential failures in tests as a result of this change:

If you use virtualenv directly in shell scripts, switch to using source virtualenv.sh or source virtualenv-isolated.sh instead. The former uses --system-site-packages while the latter does not. These scripts are provided by ansible-test. Starting with Ansible 2.11 (devel branch) they support both venv and virtualenv~~, although backporting venv support has been suggested for Ansible 2.9 and 2.10. Until that occurs, installing virtualenv for Ansible 2.9 and 2.10 will serve as a temporary work-around~~. For Ansible 2.10 and earlier you will need to set the ANSIBLE_TEST_PREFER_VENV environment variable to opt-in to using venv.
If you use the virtualenv parameter for the pip Ansible module, then be sure to specify the virtualenv_command parameter as well. See the pip integration test for an example.
If you are using virtualenv in Ansible command/shell/raw tasks, consider using the pip Ansible module instead as explained above. Alternatively, use the command search shown in that example to determine the appropriate command to invoke.

NOTE: The upcoming macOS 11 image for ansible-test will not include virtualenv. Future updates to the Linux test containers used by ansible-test for Python 3.x may also exclude virtualenv.

Answer 41 · 2021-01-11T21:57:57.000Z

macOS 11.1 now available for testing

macOS 11.1 with Python 3.9.1 is now available in CI (--remote macos/11.1). As mentioned previously, it does not contain virtualenv.

If you are testing Ansible 2.10 or later, including devel, please switch to testing with macOS 11.1.

For Ansible 2.9, continue to use osx/10.11. We do not plan to fix tests for stable-2.9 to pass using the macos/11.1 image.

Answer 42 · 2021-01-21T23:35:22.000Z

freebsd/11.4 and freebsd/12.2 now available for testing on `devel` branch

FreeBSD 11.4 and FreeBSD 12.2 are now both available for testing in CI (--remote freebsd/11.4 and --remote freebsd/12.2 respectively), for those using ansible-test from the devel branch.

Answer 43 · 2021-01-22T20:36:16.000Z

Ubuntu 20.04 now available for testing on `devel` branch

Ubuntu 20.04 (--docker ubuntu2004) is now available for use with ansible-test on the devel branch.

Note that due to packaging issues with python3-virtualenv in Ubuntu 20.04, users may find that some tests fail if the tests use virtualenv. In this case, users are referred to this announcement which provides useful information on how to resolve these failures.

Answer 44 · 2021-01-28T05:13:58.000Z

`rstcheck` sanity test no longer used for collections

The rstcheck sanity test in ansible-test (devel branch) is now limited to running against ansible/ansible and is no longer used for collections. See ansible/ansible#73391 for additional details.

Answer 45 · 2021-01-29T20:36:44.000Z

Replace Fedora 30 and 31 in CI with Fedora 32 and 33

Fedora 30 and 31 are unsupported and have reached end-of-life.

If your collection is tested against either of these releases, update your testing to use Fedora 32 and 33 instead.

Support for these older releases will be dropped from the devel version of ansible-test in the future (timeframe TBD).

Answer 46 · 2021-02-06T12:46:01.000Z

Security: avoid information leaks (that are easy avoidable) in your modules by using `no_log=True`!

I created a WIP PR for ansible-test's validate-modules (based on a branch by @gundalow) which looks for module options that potentially should have had no_log=True set to prevent secrets from showing up in log files: ansible/ansible#73508 Please run this PR on your collections (install ansible-core from that branch, run ansible-test sanity --docker -v --test validate-modules) to see if it spots something that should be hidden. If you have suggestions of both how to broaden the option names to search for, and how to prevent more false-positives, please mention them in the PR!

Please note that I know that the PR produces a certain number of false positives. This is annoying, but I think it's more important to catch more information leaks. Also, false positives can be marked with no_log=False to indicate "I know the name looks like it is a secret, but it is definitely not". This shuts off the warning for a specific option without needing an ignore.txt entry (which would disable the warning for all options in the module).

If you're wondering whether something like that is needed for plugins: not really, since plugins have no mechanism to mark options as secret - mostly since there's no automatic code which could accidental log them. The plugin itself is responsible for making sure they are not logged (or leaked in another way). AnsibleModule usually logs the module call with all parameters; for example, on one of my servers, grepping for ansible in the syslog quickly found:

Feb 05 16:02:56 hostname.com.example.com ansible-ansible.legacy.stat[9280]: Invoked with path=/path/to/file follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True

If a module option is marked with no_log=True, it will not be dumped here. If you have an option secret_api_key which is not marked as no_log=True, the secret API key provided by the user ends up in the log (if the user doesn't mark the whole task with no_log: true). So it's really important that modules use no_log=True for everything that shouldn't be logged.

Answer 47 · 2021-02-09T18:56:22.000Z

Hey, the azure pipelines container has been updated to include an additional dependency and will speed up runs. This change is purely for performance.

Please change the entry

resources:
  containers:
    - container: default
      image: quay.io/ansible/azure-pipelines-test-container:1.7.1

to

resources:
  containers:
    - container: default
      image: quay.io/ansible/azure-pipelines-test-container:1.8.0

in your .azure-pipelines/azure-pipelines.yml.

Answer 48 · 2021-02-10T16:56:59.000Z

Action Required by March 1st for Collections Using Azure Pipelines

See the report below for updates that may be required for collections tested on Azure Pipelines.

The entries marked Remove must be completed by March 1st to avoid test failures.

Azure Pipelines Test Matrix Updates

Projects using Azure Pipelines to test against the ansible-core devel branch have been checked to verify their test matrix is up-to-date.

A report on each project's status, as well as the required and recommended actions are explained below.

Using the Checklist

Each project is given a status as follows:

Skipped - The project was skipped because it does not use platforms that were evaluated. No action is necessary.
Current - The project uses platforms that were evaluated, and all are current. No action is necessary.
Update - The project uses platforms that were evaluated, and one or more changes are indicated.

The types of changes are as follows:

Add - The platform is already tested by the project, but the version is not. Add the version to the matrix.
Remove - The platform and version are deprecated and will be removed from ansible-test on March 1st. Remove the version from the matrix.
Consider - The platform is not yet tested by the project. Consider adding the version to the matrix.

IMPORTANT: These changes should only be made for the portion of the test matrix tested against the devel branch of ansible-core.

Checklist

Answer 49 · 2021-02-10T18:48:51.000Z

Action Required by March 1st for Collections Using `ansible-test` with `--docker`

If your collection is tested using https://dev.azure.com/ansible/ see the previous announcement instead.

If your collection is tested with ansible-test using the --docker option, continue reading.

IMPORTANT: These changes only affect tests against the Ansible devel and upcoming stable-2.11 branches.

On March 1st, the following containers will no longer be supported by the --docker option:

fedora30
fedora31
ubuntu1604

If your collection is tested against any of these containers, please update your tests to use the following instead:

fedora32
fedora33
ubuntu1804
ubuntu2004

These changes must be completed by March 1st for any tests run against the Ansible devel branch to avoid test failures. Changes should not be made for tests run against existing Ansible stable-X branches.

Answer 50 · 2021-02-11T00:00:19.000Z

Updated `pylint` Sanity Test

The pylint sanity test for ansible-test has been updated to pylint 2.6.0 in the devel branch. This is the latest version of pylint and it brings with it several changes:

Support for Python 3.9.
New and updated rules.
One known regression affecting reporting of "bad" variable names.

As a result of this update you may begin seeing new sanity test failures. These issues should be resolved, or as a temporary measure, ignores should be added.

If you encounter any of the following issues, please open an issue in the https://github.com/ansible/ansible/ repo so they can be addressed:

Tracebacks, crashes or hangs running the sanity tests.
Significant false positives reported by pylint's added or updated rules.

Answer 51 · 2021-02-12T22:50:19.000Z

Plugin Import Tests

The import sanity test has been extended to cover plugins. Previously it only covered modules.

There are several scenarios which are likely to cause errors to show up in collections as a result of this change:

Regressions in Ansible - If an import available to plugins in a previous version of Ansible is removed or renamed, this can cause import errors for collections that make use of that import. If this occurs, it is most likely a regression in Ansible itself and a bug report should be opened against ansible/ansible to report the issue.
Missing Collection Dependencies - If a collection depends on other collections, those dependencies must be installed in the same collection root as the collection under test. If you see import errors for collection dependencies, make sure they are installed in the correct location.
Unhandled ImportError - Just like modules, plugins must now gracefully handle ImportError for all imports which are not provided by the Python standard library, collection dependencies, Ansible's dependencies, or Ansible itself. For example, a collection plugin that imports kerberos must do so by wrapping it in a try/except ImportError handler, just as would be done if it were a module.

(See https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/import.html for more details and examples of how to handle it correctly.)

In Bullhorn 20

Answer 52 · 2021-02-18T20:50:02.000Z

`pylint` Sanity Test Update

The pylint sanity test in ansible-core's devel branch no longer ignores used-before-assignment errors (ansible/ansible#73639). These errors are usually not false positives, so if your collection's sanity tests start failing because of this, better look carefully and not just ignore these errors.

In Bullhorn 20

Answer 53 · 2021-03-05T06:40:07.000Z

Vendored `ipaddress` in ansible.netcommon 2.0.0 has been removed

In case your collection uses the vendored ipaddress module from ansible.netcommon, either directly (import ansible_collections.ansible.netcommon.plugins.module_utils.compat.ipaddress) or indirectly (import ansible.module_utils.compat.ipaddress, see ansible-core's module_utils routing): it has been removed from ansible.netcommon 2.0.0 (released 3 days ago) and these imports will no longer work!

a) If you need it for code that only supports Python 3.5 or newer, you can replace it with import ipaddress (https://docs.python.org/3/library/ipaddress.html);
b) Otherwise, you can (i) rewrite your code to not use it, (ii) vendor it in your collection (example), or (ii) require the user to install it manually for Python 2.6/2.7 (pip install ipaddress).

In Bullhorn 21

Answer 54 · 2021-03-12T16:52:17.000Z

validate-modules change in `devel`: flags module options that might reveal secret data

There is a new test in validate-modules (ansible/ansible#73508) which reports module options that look like they could contain secret data (by the option name), but do not have no_log set. Examples are things like password, key or token.

Since the test aims at catching as many real cases as possible, it unfortunately also produces a lot of false positives. DO NOT use ignore-2.11.txt entries to ignore these! Instead, add no_log=False to the module option's argument spec entry to flag this specific option as "contains no secret data". Ignoring the error for a file will also disable this check for new module options, and this this a really bad idea!

(See also #45 (comment))

Answer 55 · 2021-03-14T11:32:50.000Z

An update to the previous post, since I've now seen that in multiple places: if you have to add no_log=True somewhere, this is usually a security vulnerability and should be treated as such! That means: it should be properly documented in the changelog (if you're using antsibull-changelog: section security_fixes), and a new release which fixes them should be released soonish.

If the affected modules have been migrated from other collections, and/or from Ansible 2.9, you should backport these security fixes to the other collections and to Ansible's stable-2.9 branch.

(If you are just marking false-positives with no_log=False, that's not a vulnerability and doesn't need special attention.)

Answer 56 · 2021-03-25T20:05:33.000Z

A new version of the Azure Pipelines test container image is available (1.9.0). The following collections should update to using the new image. Please check them off once they have been updated to use the new image.

Answer 57 · 2021-04-06T04:56:18.000Z

stable-2.11 has been branched

The ansible/ansible repository now has a stable-2.11 branch, and the version of the devel branch has been bumped to 2.12.0.dev0. This means that if you have an tests/sanity/ignore-2.11.txt file in your collection, you need to copy it to tests/sanity/ignore-2.12.txt. Also if your collection is included in Ansible, please make sure that CI also tests against the new stable-2.11 branch. Ansible 4.0.0 will be based on ansible-core 2.11, and thus your collection must work with stable-2.11.

(We still recommend to also test against the devel branch to be informed early about breaking changes. But testing against stable-2.11 is required by the inclusion criteria, now that stable-2.11 exists.)

Answer 58 · 2021-04-15T18:33:56.000Z

Goodbye include: long live import_X/include_X
ansible/ansible#71262

Answer 59 · 2021-04-15T18:36:44.000Z

All core lookups were moved to support the plugin config system ansible/ansible#74108
In the future we will deprecate 'custom inline string parsing', for exampe:

lookup('ini', 'key filename=ansible.ini section=whateves')

becomes

lookup('ini', 'key', filename='ansible.ini', section='whateves')

This uses the 'python function argument parsing' and allows for better escaping and less confusion with k=v pairs.

Answer 60 · 2021-04-15T18:42:09.000Z

@bcoca can you expand on what you mean by deprecation? Does that apply only to the lookups in core? Or will internal changes somehow force or attempt to force that no lookups can perform "inline string parsing"?

Answer 61 · 2021-04-15T18:45:14.000Z

@briantist in the end the plugin can do what it wants. But to be consistent with the documentation/configuration declaration it should not do it's own parsing, so it can deliver predictable results as well as keep common escaping and other input rules.

Also I'm trying to eliminate the miriad of inline string parsers, each with it's own conventions, rules and limitations:

https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/plugins/lookup/ini.py#L73
https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/plugins/lookup/first_found.py#L135
https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/plugins/lookup/csvfile.py#L147

And the 'common' one that most didn't use:
https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/parsing/splitter.py#L49

Answer 62 · 2021-04-15T18:51:35.000Z

@briantist in the end the plugin can do what it wants. But to be consistent with the documentation/configuration declaration it should not do it's own parsing, so it can deliver predictable results as well as keep common escaping and other input rules.

Thanks. I ask as someone who migrated a lookup to use config manager and argument parsing, but kept the string parsing for compatibility. Unfortunately I think most users are still using (and prefer) the old style, much to my chagrin. I expect a lot of pushback if I attempt to deprecate it.

But what I like is that this does give me some support for not allowing this in future lookup plugins within the collection. I had been wrestling internally whether or not to support string style for familiarity for end users, even though I'd have preferred not to.

I know deprecation notices in lookups still aren't working (for anyone curious: ansible/ansible#73240 ), but if they were, would this deprecation be detected automatically? It seems like it couldn't be since it's not really done in a common function (as you've shown in your edit).

Edit: I didn't even know there was a common function..

Answer 63 · 2021-04-19T07:24:36.000Z

stable-2.11 has been branched

The ansible/ansible repository now has a stable-2.11 branch, and the version of the devel branch has been bumped to 2.12.0.dev0. This means that if you have an tests/sanity/ignore-2.11.txt file in your collection, you need to copy it to tests/sanity/ignore-2.12.txt. Also if your collection is included in Ansible, please make sure that CI also tests against the new stable-2.11 branch. Ansible 4.0.0 will be based on ansible-core 2.11, and thus your collection must work with stable-2.11.

(We still recommend to also test against the devel branch to be informed early about breaking changes. But testing against stable-2.11 is required by the inclusion criteria, now that stable-2.11 exists.)

ansible-collections/dellemc.enterprise_sonic#39

Answer 64 · 2021-04-19T23:38:33.000Z

Changes to ansible-test units behavior

If you maintain a collection that uses ansible-test to run unit tests, the following changes made to the devel version of ansible-test may impact your collection:

Unit tests for controller-side code are now run only under Python 3.8 and later. This matches the minimum Python version required by Ansible itself for controller-side code.
Unit tests for modules must reside under tests/unit/plugins/modules/ to be detected as module unit tests. If they reside in another location, they will be detected as tests for controller-side code, which will prevent them from running under Python 3.7 and earlier.
Unit tests for module_utils must reside under tests/unit/plugins/module_utils/ to be detected as module_utils unit tests. If they reside in another location, they will be detected as tests for controller-side code, which will prevent them from running under Python 3.7 and earlier.
Unit tests detected as modules or module_utils tests are no longer able to import from ansible.*, except for ansible.module_utils.*.

Answer 65 · 2021-04-20T00:13:08.000Z

@mattclay

Unit tests detected as modules or module_utils tests are no longer able to import from ansible.*, except for ansible.module_utils.*.

I have a module_utils unit test that's now failing to import from ansible.plugins import AnsiblePlugin. It was passing yesterday/earlier today.

Fail: https://github.com/ansible-collections/community.hashi_vault/runs/2385869734?check_suite_focus=true#step:5:461
Line: https://github.com/ansible-collections/community.hashi_vault/blob/main/tests/unit/plugins/module_utils/test_hashi_vault_option_adapter.py#L10

This module_util is intended to be used by both modules and plugins. The tests need to test the plugin functionality to test that (the util itself doesn't import from ansible.*).

What's the purpose of making this intentionally not work?

Answer 66 · 2021-04-20T00:45:01.000Z

@briantist Importing ansible.plugins.* under Python 3.7 and earlier is no longer supported, since starting with Ansible 2.12 the controller requires Python 3.8 or later. Although doing so does not result in import failures or syntax errors yet, that will be changing very soon as we start taking advantage of newer Python features in the controller code. Since modules and module_utils are expected to work on Python versions older than what the controller supports, this means that unit tests for those cannot depend on code that only works newer Python versions.

If there are module or module_utils unit tests that depend on ansible.plugins.* or other ansible code outside of ansible.module_utils, then there are a few options available:

Update the unit tests to not depend on the code which will be dropping support for Python 3.7 and earlier.
Keep the dependency, and relocate the unit tests so they are executed as controller tests. This will of course limit unit testing of the code to Python versions supported by the controller (3.8+).

Answer 67 · 2021-04-20T00:54:48.000Z

@mattclay I see, thank you for the explanation.

If I were to

relocate the unit tests so they are executed as controller tests

What would be an appropriate place for that? Is test/unit/plugins/plugin_utils/ recognized as controller-only? That seems like it would be the ideal location.

Answer 68 · 2021-04-20T01:05:53.000Z

@briantist There is no standard location for controller-only module/module_utils unit tests. However, if the tests are located any place other than tests/unit/plugins/modules/ or tests/unit/plugins/modules_utils/ they will be considered controller-only. Keep in mind that ansible-test change detection (--changed option) will no longer trigger tests for modules or module_utils if they have been placed in a non-standard location.

It looks like community.hashi_vault supports Python 2.7 and later. Are you planning on continuing to support Python 2.7 and later, while only running unit tests on Python 3.8 and later?

Answer 69 · 2021-04-20T02:00:40.000Z

It looks like community.hashi_vault supports Python 2.7 and later. Are you planning on continuing to support Python 2.7 and later, while only running unit tests on Python 3.8 and later?

Uncertain. May only support Py 2.7 and later on the Ansible core versions that support it (it would be a little weird to have supported python 2.7+ and suddenly yank it, while still supporting earlier Ansible versions..), so maybe continuing to run the units on earlier py versions, when coupled with earlier core versions... I'll have to see if/how I can do that effectively.

Answer 70 · 2021-04-20T04:36:11.000Z

@briantist dropping support for a Python version means you have to release a new major version. As long as you run unit tests also with older Ansible/ansible-base/-core versions (and support these versions with your collection), I wouldn't just drop support because devel no longer supports them :)

Answer 71 · 2021-04-20T13:58:25.000Z

I think we're agreeing @felixfontein ! I don't want to drop support. But I now have to figure out how to split/arrange my unit tests for module utils that are intended to be used both controller and remote side, so that I can run the tests successfully. I'll work on it.. in the meantime my tests are going to be failing.

Answer 72 · 2021-04-21T20:29:24.000Z

freebsd/13.0 now available for testing on `devel` branch

FreeBSD 13.0 is now available for testing in CI (--remote freebsd/13.0) for collections testing with ansible-test from devel.

freebsd/11.4 will be dropped from ansible-test later in the 2.12 development cycle, so collections testing against devel are encouraged to target FreeBSD 12 (currently freebsd/12.2) and 13 (currently freebsd/13.0) for their tests, dropping 11.

Answer 73 · 2021-04-23T07:26:58.000Z

ansible-test Sanity Tests Require Python 3.8+

The devel version of ansible-test now requires Python 3.8 or later for sanity tests (except compile and import).

If you run sanity tests using --docker (or --docker default) no changes are necessary. However, if you use the --python option or your own environment to run using older Python versions, sanity tests will be skipped.

Answer 74 · 2021-04-27T19:16:42.000Z

ansible-test Collection Configuration

The devel version of ansible-test now supports an optional collection-specific configuration file in tests/config.yml.

Currently there is one configuration option available to declare the Python versions a collection supports for testing.

For example, if a collection requires Python 3.6 or later to run modules, the following configuration could be used:

modules:
  python_requires: '>= 3.6'

For collections which consist of modules that depend on ansible-connection, the following config is recommended:

modules:
  python_requires: controller

See the sample configuration file for additional details.

Answer 75 · 2021-04-29T15:44:47.000Z

Codecov bash uploader now hosted in S3

Due to the recent security incident, we now have our own copy of the bash uploader hosted in S3. Update any reference to https://codecov.io/bash to use the new URL.

Answer 76 · 2021-04-30T16:44:01.000Z

Update testing to use Fedora 34

Fedora 34 (--docker fedora34) has been added to ansible-test on the devel branch. Update collections to test against Fedora 33 and 34.

Fedora 32 will be going EOL soon, and will be removed from ansible-test on devel later in the ansible-core 2.12 development cycle.

Answer 77 · 2021-05-04T13:12:14.000Z

Reminder about the ansible release schedule:

The last Ansible 3 release is planned for May 11, 2021
Ansible 4 release candidate 1 is also planned for May 11, 2021
If there are no blocker bugs discovered, Ansible 4 final will be made on may 18, 2021

Answer 78 · 2021-05-10T15:34:40.000Z

Testing with Python 3.10

Python 3.10 is now supported by the devel branch version of ansible-test. The default container image has been updated to include Python 3.10.0b1.

When testing a collection with ansible-test sanity or ansible-test units, be aware of the following:

If running without the --python option, tests will automatically be run using Python 3.10. You may see new test failures under this version of Python that will require resolution.
If running with the --python option, be sure to update your test configuration to include Python 3.10.

Periodically throughout the Ansible 2.12 development cycle, the default container will be updated to include newer pre-release versions of Python 3.10, until the final version has been released. The final version of Python 3.10 is currently expected on October 4th -- the same day as Ansible 2.12 RC1.

Answer 79 · 2021-06-15T20:09:11.000Z

galaxy.ansible.com maintenance 2021-06-19 0800UTC

All, Just a heads up that there will be some maintenance happening on ansible.galaxy.com this Saturday at 0800 UTC lasting up to an hour. We don't anticipate any downtime, though there is always the risk.

If you are interested in the future of Galaxy we have a short survey to help us prioritise the feature work, please let us know your thoughts via short survey (background)

Thanks

Answer 80 · 2021-06-16T06:20:02.000Z

That should say Saturday 19th for galaxy.ansible.com maintenance, updated.

Answer 81 · 2021-07-29T12:32:14.000Z

Be aware that Ansible Galaxy unexpectedly made a change to require mandatory requires_ansible in meta/runtime.yml file for uploading collection[1][2]. And after you add it you'll probably will hit ansible-test failure in sanity checks if you use it with --venv option, since it can't check this field because of failing requirement[3].
Worth to mention you must have meta/runtime.yml now for your collection to be uploaded.

[1] ansible/galaxy-importer#122
[2] ansible/galaxy-importer#124
[3] ansible/ansible#75353

(Added to Bullhorn 31)

Answer 82 · 2021-07-30T00:14:44.000Z

ansible-test default test container updated

The default container image has been updated to include Python 3.10.0b4

Azure Pipelines Test Matrix Updates

Projects using Azure Pipelines to test against the ansible-core devel branch have been checked to verify their test matrix is up-to-date.

A report on each project's status, as well as the required and recommended actions are explained below.

Using the Checklist

Each project is given a status as follows:

Skipped - The project was skipped because it does not use platforms that were evaluated. No action is necessary.
Current - The project uses platforms that were evaluated, and all are current. No action is necessary.
Update - The project uses platforms that were evaluated, and one or more changes are indicated.

The types of changes are as follows:

Add - The platform is already tested by the project, but the version is not. Add the version to the matrix.
Remove - The platform and version are deprecated and will be removed from ansible-test in the future. Remove the version from the matrix.
Consider - The platform is not yet tested by the project. Consider adding the version to the matrix.

IMPORTANT: These changes should only be made for the portion of the test matrix tested against the devel branch of ansible-core.

Checklist

Answer 83 · 2021-07-30T06:20:46.000Z

^ Related issue #179

Answer 84 · 2021-07-30T15:26:09.000Z

The PR adding RHEL 8.4 to ansible-test has been merged. If you were seeing failures as a result of adding/trying to add RHEL 8.4, they should be resolved now.

Answer 85 · 2021-08-05T15:16:51.000Z

Ansible-test's validate-module sanity check in ansible-core's devel branch has a new feature which checks that _info and _facts modules have supports_check_mode=True (ansible/ansible#75324). (See Ansible's conventions which mandate that _info and _facts modules should do that.)

Answer 86 · 2021-08-09T09:37:17.000Z

Please make sure to not use Fix with this issue following in your PRs. If you (or the person/bot merging) has sufficient rights, this issue will be closed. Thanks :)

Answer 87 · 2021-08-11T10:17:09.000Z

ansible.posix:main has been fixed in ansible-collections/ansible.posix#237 like below:

Answer 88 · 2021-08-12T04:09:02.000Z

ansible-test pylint version updated

The pylint version in ansible-test has been updated to 2.9.3.

As a result, new pylint errors may appear when testing collections against the devel version of ansible-test.

Answer 89 · 2021-08-31T09:05:43.000Z

We are happy to announce that the registration (free) for the Ansible Contributor Summit is open.

Which day should you attend?

Day 1 (Tuesday, September 28)

If you are wondering what it means to "contribute" to Ansible, or if you have been using Ansible for a while and are thinking of ways to get more involved with the project and community, then Day 1 is for you!

Among other topics, there will be:

Ask a maintainer session (Interactive session covering what it means to be/become a collection maintainer, their roles, responsibilities, how open source projects work/operate, and more).
Group instructor-led class in setting up a development environment (The main purpose of the session is to help you set up a minimalistic but complete Ansible collection development environment).
Interactive self-passed instruqt scenarios that help you learn different aspects of development.

Day 2 (Friday, October 1)

If you have already been contributing or participating in our previous Contributor Summits, you are still welcome to attend Day 1, however we think that Day 2 will be more interesting for you.

Among other topics, there will be:

Welcome and announcements.
Matrix, the new communication format.
Different presentations by Core, Cloud, Network, and other teams.

Hackathon (Tuesday to Friday)

In addition, we will be running a Hackathon throughout the week.

This is for anyone with an interest in contributing to Ansible (with a focus on Collections), including those who have never contributed and are unsure of how to get started, existing contributors and maintainers, etc. all are welcome!
Resources (people, labs, docs, etc) will be available to help people through the entire process.

More details will be available soon. Please register here so that you will be informed of the updates!

See you at the summit!

Answer 90 · 2021-09-21T01:39:15.000Z

ansible-test split controller/target testing

The devel version of ansible-test (part of the upcoming ansible-core 2.12 release) now supports integration testing with separate hosts and Python versions for the controller and target.

If you maintain a collection which runs tests using Python 3.7 or earlier, please pay special attention to the following, as your tests will be required to run in a "split" configuration involving two separate Python interpreters, possibly on separate hosts.

Collection authors may need to update their tests to work correctly with split testing. Most issues will be due to tests assuming that the controller and target are the same host, such as:

Using modules to access the controller without delegate_to: localhost.
Using delegate_to: localhost (or other forms of localhost access) when intending to connect to the target.
Using lookup plugins to access the target.
Assuming files or environment variables from one host are available on the other.
Installing module requirements on the controller.
Installing plugin requirements on the target.
Assuming the controller and target use the same Python version or path to the Python interpreter.

There are also other things that are likely to cause problems with tests, such as:

Using the pip CLI instead of the Ansible pip module. If the pip CLI must be used, use one of the following:
- controller (script): python -m pip
- controller (playbook): {{ ansible_playbook_python }} -m pip
- target (playbook): {{ ansible_python_interpreter }} -m pip.__main__
Not using the {plugin_type}_{plugin_name} naming convention for integration test targets for controller plugins.
Installing cryptography or pyopenssl on the target without constraining the versions installed.

Answer 91 · 2021-09-21T19:01:51.000Z

ansible-core Python 3.8 dependency

ansible-core devel (part of the upcoming ansible-core 2.12 release) requires Python3.8+ on the controller now. In addition to a simple version check that provides a friendly-ish error, we've removed a number of places with obvious Py2 compat, and made a few Python3.8+ code changes.

Answer 92 · 2021-09-21T23:16:51.000Z

ansible-test container updates

The base and default containers for ansible-test will be updated several times this week ahead of code freeze:

Wednesday - The containers will be updated to reflect the new requirements used by ansible-test. If your collection has been relying on pre-installed requirements in the default container, instead of declaring/installing them, then tests may fail as a result of the change.
Thursday or Friday - The underlying base image is expected to be updated from Ubuntu 18.04 to Ubuntu 20.04. No additional changes to requirements are planned. Tests that install OS packages may be affected by this change. (postponed for ansible-core 2.13)

Answer 93 · 2021-09-22T00:09:32.000Z

Dropping Python 2.6 Support

On Monday, after creation of the stable-2.12 branch, support for Python 2.6 will be removed from the ansible devel branch.

Answer 94 · 2021-09-23T07:37:09.000Z

ansible-test pip constraints update

On Thursday, pip constraints not used by ansible-test will be removed from ansible-test in the devel branch.

Collections which depend on these constraints to install their requirements will need to provide them instead of depending on ansible-test to do so. The constraints which will be removed can be found here: ansible/ansible#75763

Answer 95 · 2021-09-24T16:59:51.000Z

stable-2.12 branch point

The core team made a decision earlier this week to change the point when we create the stable-2.12 branch to happen at Feature Freeze. This change was made to accommodate the core team no longer performing releases (another team at Red Hat now performs all of the core upstream and downstream releases), and due to a bug in GitHub that prevents us from enabling and disabling branch protections.

With the change in schedule, we've also decided for this time only to delay bumping the version in devel to 2.13 at stable branching until Wednesday September 29.

We're planning some revised guidance for devel testing in CI in the near future, and once that has become more well defined we will provide that information.

Answer 96 · 2021-09-24T17:18:05.000Z

Preparing sanity test ignores for 2.13 development

If you maintain a collection that is tested against the devel branch with ansible-test sanity, be sure to make a copy of your tests/sanity/ignore-2.12.txt file as tests/sanity/ignore-2.13.txt before September 29th to avoid test failures. Keep the two files in sync until the version in devel changes, at which point only the 2.13 file should be updated for devel.

Answer 97 · 2021-09-27T08:05:26.000Z

Reminder - Day 1 of the Ansible Contributor Summit and the Hackathon start tomorrow (Tuesday, 28 September 2021)!

For more info and the latest updates, please see the Ansible Contributor Summit 2021.09 HackMD.

Answer 98 · 2021-09-30T15:35:15.000Z

Reminder - Day 2 of the Ansible Contributor Summit and the last day of the Hackathon start tomorrow (Friday, 1 October 2021)!

For more info and the latest updates, please see the Ansible Contributor Summit 2021.09 HackMD.

Answer 99 · 2021-09-30T15:42:46.000Z

ansible's devel branch's version got bumped to 2.13.0.dev0. So if you didn't copy tests/sanity/ignore-2.12.txt to tests/sanity/ignore-2.13.txt (in case that file had contents), you'll notice soon if you're testing against devel :)

Answer 100 · 2021-09-30T19:44:50.000Z

Removal of Python 2.6 Support

Removal of Python 2.6 support from the devel branch is expected to be completed today. This was previously announced for Monday, but was delayed until the version bump in devel occurred.

Collection maintainers may need to make the following changes once the removal is complete:

If you have any sanity test ignores for Python 2.6 in your ignore-2.13.txt file, you will need to remove them.
If you have Python 2.6 or CentOS 6 in your CI test matrix, you will need to remove them.

Details of what's changed

Remove tests/sanity/requirements.txt from collections

Update collection tests from RHEL 7.6 to 7.8

Changelog process for collections

Update collection tests from RHEL 8.1 to 8.2

Remove ANSIBLE_METADATA

ansible-test (validate-modules) now checks for correct module deprecation

Call to action

Update collection tests to test on Fedora 32

Dropping ANSIBLE_METADATA

Call to action

Sanity tests for removed_in_version and deprecated_aliases

Call to action

Next Virtual Contributors Summit

ansible-test default test container updated

Call to action

Remove "smoke tests" from migrated collections

Call to action

Next Ansible PR review day (Collections)

Specifying test requirements for collections

Call to action

Freeze and bump of devel version to 2.11.0.dev0 requires tests/sanity/ignore-2.11.txt

Move changelog and porting guide entries from ansible/ansible to your collection

Call to Action

Deprecation in collections (and ansible-base)

Requirements files

check_matrix.py

updating M() and :seealso: module references in DOCUMENTATION blocks

Call to action

Examples

AnsibleFest call for papers is open

Call to action

Examples

version_added in collections

Collection module docs now on docs.ansible.com for review

Known issues and Important Caveats

version_added in collections is now better validated (modules only)

Changes to ansible-galaxy login for role/collection publishers

macOS 10.15 testing image now available in CI

ansible-test sanity validates more semantic versioning properties for collections

Please make sure to not include unwanted artifacts in your collection release on galaxy!

Ansible-3.0.0 Schedule and Preview of 4.0.0 Schedule

Background

TL;DR

New sanity check for deprecation / tombstone dates / versions in meta/runtime.yml

ansible-test no longer includes virtualenv on FreeBSD or RHEL instances for Python 3.x

macOS 11.1 now available for testing

freebsd/11.4 and freebsd/12.2 now available for testing on devel branch

Ubuntu 20.04 now available for testing on devel branch

rstcheck sanity test no longer used for collections

Replace Fedora 30 and 31 in CI with Fedora 32 and 33

Security: avoid information leaks (that are easy avoidable) in your modules by using no_log=True!

Action Required by March 1st for Collections Using Azure Pipelines

Azure Pipelines Test Matrix Updates

Using the Checklist

Checklist

Action Required by March 1st for Collections Using ansible-test with --docker

Updated pylint Sanity Test

Plugin Import Tests

pylint Sanity Test Update

Vendored ipaddress in ansible.netcommon 2.0.0 has been removed

validate-modules change in devel: flags module options that might reveal secret data

stable-2.11 has been branched

stable-2.11 has been branched

Changes to ansible-test units behavior

freebsd/13.0 now available for testing on devel branch

ansible-test Sanity Tests Require Python 3.8+

ansible-test Collection Configuration

Codecov bash uploader now hosted in S3

Update testing to use Fedora 34

Testing with Python 3.10

galaxy.ansible.com maintenance 2021-06-19 0800UTC

ansible-test default test container updated

Azure Pipelines Test Matrix Updates

Using the Checklist

Checklist

ansible-test pylint version updated

Day 1 (Tuesday, September 28)

Day 2 (Friday, October 1)

Hackathon (Tuesday to Friday)

Remove `tests/sanity/requirements.txt` from collections

Dropping `ANSIBLE_METADATA`

`version_added` in collections

`version_added` in collections is now better validated (modules only)

Changes to `ansible-galaxy login` for role/collection publishers

`ansible-test sanity` validates more semantic versioning properties for collections

freebsd/11.4 and freebsd/12.2 now available for testing on `devel` branch

Ubuntu 20.04 now available for testing on `devel` branch

`rstcheck` sanity test no longer used for collections

Security: avoid information leaks (that are easy avoidable) in your modules by using `no_log=True`!

Action Required by March 1st for Collections Using `ansible-test` with `--docker`

Updated `pylint` Sanity Test

`pylint` Sanity Test Update

Vendored `ipaddress` in ansible.netcommon 2.0.0 has been removed

validate-modules change in `devel`: flags module options that might reveal secret data

freebsd/13.0 now available for testing on `devel` branch