ansible/ansible

s3_bucket error with Minio - GetBucketVersioning NotImplemented

Closed this issue · 6 comments

benbro commented
SUMMARY

When trying to create a minio bucket with s3_bucket I'm getting an error because minio doesn't support bucket versioning.
This is a regression. Worked in ansible 2.4.
Might be after the migration to boto3:
PR #37189

ISSUE TYPE
  • Bug Report
COMPONENT NAME

s3_bucket

ANSIBLE VERSION
ansible 2.6.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]
CONFIGURATION

I'm using python3:
inventories/production/hosts

[all:vars]
ansible_python_interpreter=/usr/bin/python3
OS / ENVIRONMENT

Ubuntu 18.04.

STEPS TO REPRODUCE

PR #42474 is required to test and prevent issue #42459

- name: python3-botocore package
  apt:
    name: python3-botocore
    state: latest

- name: python3-boto package
  apt:
    name: python3-boto
    state: latest

- name: create bucket
  s3_bucket:
    name: mybucket
    state: present
    aws_access_key: "{{key}}"
    aws_secret_key: "{{secret}}"
    s3_url: "{{url}}"
    ceph: true
EXPECTED RESULTS

No error

ACTUAL RESULTS
botocore.exceptions.ClientError: An error occurred (NotImplemented) when calling the GetBucketVersioning operation: A header you provided implies functionality that is not implemented.

I think that the module tries to ignore versioning if the option isn't set but fails to do that.

benbro commented

I'm getting similar errors for GetBucketRequestPayment and GetBucketTagging:

botocore.exceptions.ClientError: An error occurred (NotImplemented) when calling the GetBucketRequestPayment

botocore.exceptions.ClientError: An error occurred (NotImplemented) when calling the GetBucketTagging

A temp workaround for minio is to comment out the GetBucketVersioning, GetBucketRequestPayment and GetBucketTagging parts:

   '''
    # Versioning
    try:
        versioning_status = get_bucket_versioning(s3_client, name)
    except (ClientError, BotoCoreError) as e:
        module.fail_json_aws(e, msg="Failed to get bucket versioning")

    if versioning is not None:
        required_versioning = None
        if versioning and versioning_status.get('Status') != "Enabled":
            required_versioning = 'Enabled'
        elif not versioning and versioning_status.get('Status') == "Enabled":
            required_versioning = 'Suspended'

        if required_versioning:
            try:
                put_bucket_versioning(s3_client, name, required_versioning)
                changed = True
            except (BotoCoreError, ClientError) as e:
                module.fail_json_aws(e, msg="Failed to update bucket versioning")

            versioning_status = wait_versioning_is_applied(module, s3_client, name, required_versioning)

    # This output format is there to ensure compatibility with previous versions of the module
    versioning_return_value = {
        'Versioning': versioning_status.get('Status', 'Disabled'),
        'MfaDelete': versioning_status.get('MFADelete', 'Disabled'),
    }
    '''
    versioning_return_value = {}
   '''
    # Requester pays
    try:
        requester_pays_status = get_bucket_request_payment(s3_client, name)
    except (BotoCoreError, ClientError) as e:
        module.fail_json_aws(e, msg="Failed to get bucket request payment")

    payer = 'Requester' if requester_pays else 'BucketOwner'
    if requester_pays_status != payer:
        put_bucket_request_payment(s3_client, name, payer)
        requester_pays_status = wait_payer_is_applied(module, s3_client, name, payer, should_fail=False)
        if requester_pays_status is None:
            # We have seen that it happens quite a lot of times that the put request was not taken into
            # account, so we retry one more time
            put_bucket_request_payment(s3_client, name, payer)
            requester_pays_status = wait_payer_is_applied(module, s3_client, name, payer, should_fail=True)
        changed = True
    '''
    '''
    # Tags
    try:
        current_tags_dict = get_current_bucket_tags_dict(s3_client, name)
    except (ClientError, BotoCoreError) as e:
        module.fail_json_aws(e, msg="Failed to get bucket tags")

    if tags is not None:
        if current_tags_dict != tags:
            if tags:
                try:
                    put_bucket_tagging(s3_client, name, tags)
                except (BotoCoreError, ClientError) as e:
                    module.fail_json_aws(e, msg="Failed to update bucket tags")
            else:
                try:
                    delete_bucket_tagging(s3_client, name)
                except (BotoCoreError, ClientError) as e:
                    module.fail_json_aws(e, msg="Failed to delete bucket tags")
            wait_tags_are_applied(module, s3_client, name, tags)
            current_tags_dict = tags
            changed = True
    '''
    current_tags_dict = {}
morph027 commented

Hm, ran into the same today ....

  • 👍1
psi-4ward commented

Same here, my storage has no payer api

ansibot commented

Thank you very much for your interest in Ansible. Ansible has migrated much of the content into separate repositories to allow for more rapid, independent development. We are closing this issue/PR because this content has been moved to one or more collection repositories.

For further information, please see:
https://github.com/ansible/ansibullbot/blob/master/docs/collection_migration.md