Bucket policy failed to apply in the expected time

[4n70w4]

SUMMARY

I use https://min.io/ minio/minio:RELEASE.2019-04-09T01-22-30Z

TASK [Publlish policy] *********************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Bucket policy failed to apply in the expected time", "resource_actions": ["storage:GetBucketVersioning", "storage:PutBucketPolicy", "storage:GetBucketPolicy", "storage:ListBuckets", "storage:GetBucketRequestPayment"]}

But bucket is created and policy applied!

ISSUE TYPE

• Bug Report

COMPONENT NAME

s3_bucket

ANSIBLE VERSION

ansible 2.9.12
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/anton/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.17 (default, Jul 20 2020, 15:37:01) [GCC 7.5.0]

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

- name: Publlish policy
  amazon.aws.s3_bucket:
    name: "{{ minio_bucket }}"
    policy: "{{ lookup('template', 'policy.json.j2') }}"
    state: present
    s3_url: "{{ minio_endpoint }}"
    aws_access_key: "{{ minio_key }}"
    aws_secret_key: "{{ minio_secret }}"
    validate_certs: no
    region: us-east-1

But if I remove policy: "{{ lookup('template', 'policy.json.j2') }}" line task is ok.

EXPECTED RESULTS

Don't show fatal, show ok.

ACTUAL RESULTS

TASK [Publlish policy] *********************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Bucket policy failed to apply in the expected time", "resource_actions": ["storage:GetBucketVersioning", "storage:PutBucketPolicy", "storage:GetBucketPolicy", "storage:ListBuckets", "storage:GetBucketRequestPayment"]}

[4n70w4]

ansible/ansible#71396

[4n70w4]

Okay. My policy:

{
  "Id": "Policy1554905040487",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1554905036932",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::something/*",
      "Principal": "*"
    }
  ]
}

It works if I directly PUT this json to https://storage.my/something/?policy.

I remove Id and wrap Principal in object with AWS key.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1554905036932",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::something/*",
      "Principal": {
        "AWS":  "*"
      }
    }
  ]
}

And all ok via Ansible!

[tremble]

Thanks for taking the time to make your report.

I'm guessing that MiniO is performing some conversions behind the scenes which make it difficult to tell if the policies are equivalent.

What's key here is that we need to know
a) Exactly what policy do you pass Ansible
b) Exactly what policy does MinIO think is live if you query the APIs (AWS makes tweaks to the policy before saving and returning it, MinIO probably does too)

I don't have access to MinIO so I can't perform any testing myself.

[4n70w4]

Minio playground https://play.minio.io/minio/login

[tremble]

Full error based on s3_bucket from this collection.

fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "aws_access_key": "Q3AM3UQ867SPQQA43P2F",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "ceph": false,
            "debug_botocore_endpoint_logs": false,
            "ec2_url": null,
            "encryption": null,
            "encryption_key_id": null,
            "force": false,
            "name": "ansible-collections-amazon-aws-149",
            "policy": "{\"Id\": \"Policy1554905040487\", \"Version\": \"2012-10-17\", \"Statement\": [{\"Sid\": \"Stmt1554905036932\", \"Action\": [\"s3:GetObject\"], \"Effect\": \"Allow\", \"Resource\": \"arn:aws:s3:::ansible-collections-amazon-aws-149/*\", \"Principal\": \"*\"}]}",
            "profile": null,
            "purge_tags": true,
            "region": "us-east-1",
            "requester_pays": false,
            "s3_url": "https://play.minio.io/",
            "security_token": null,
            "state": "present",
            "tags": null,
            "validate_certs": false,
            "versioning": null
        }
    },
    "live_policy": {
        "ID": "Policy1554905040487",
        "Statement": [
            {
                "Action": [
                    "s3:GetObject"
                ],
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "*"
                    ]
                },
                "Resource": [
                    "arn:aws:s3:::ansible-collections-amazon-aws-149/*"
                ],
                "Sid": "Stmt1554905036932"
            }
        ],
        "Version": "2012-10-17"
    },
    "msg": "Bucket policy failed to apply in the expected time",
    "requested_policy": {
        "Id": "Policy1554905040487",
        "Statement": [
            {
                "Action": [
                    "s3:GetObject"
                ],
                "Effect": "Allow",
                "Principal": "*",
                "Resource": "arn:aws:s3:::ansible-collections-amazon-aws-149/*",
                "Sid": "Stmt1554905036932"
            }
        ],
        "Version": "2012-10-17"
    }
}

This confirms that minio is tweaking the policy behind the scenes.

[tremble]

Ok, some further testing - AWS doesn't make the same tweak, BUT the documentation explicitly states that the two principal definitions used are equivalent:

https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html#principal-anonymous

The following elements are equivalent:
"Principal": "*"
"Principal" : { "AWS" : "*" }

[ansibullbot]

cc @jillr @s-hertel @wimnat
click here for bot help