s3_bucket.destroy_bucket fails to delete unversioned items #1533
Labels
waiting_on_contributor
Needs help. Feel free to engage to get things unblocked
Comments
|
@evohnave Thank you for raising this. Would you be willing to open a Pull Request with a patch. |
alinabuzachis
added
waiting_on_contributor
|
@alinabuzachis I'm willing, but gotta tell you that I've promised to do this sort of thing to other open source projects and never gotten around to it. I will try to get to it tonight. |
|
Pull request #1538 submitted. |
softwarefactory-project-zuul bot
pushed a commit
that referenced
this issue
May 11, 2023
s3_bucket: fix VersionId==null when s3 object not versioned
SUMMARY
Boto3 1.26.129 (possibly earlier) returns "VersionId": "null" from s3_client.list_object_versions() when s3 objects are not versioned. Previously, "VersionId" was None when an s3 object was not versioned. This change broke s3_bucket.destroy_bucket() because the the VersionId was no longer popped (line 1166) when the s3 object was not versioned, and the subsequent attempts to delete the s3 object failed as the "VersionId" was absolutely not "null". Adding in `or fk.get("VersionId")=="null" will catch this new value for non-versioned s3 objects while allowing backwards compatibility with previous versions that return None for "VersionId".
Fixes #1533 s3_bucket.destroy_bucket fails to delete unversioned items
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
s3_bucket.py
ADDITIONAL INFORMATION
Ensure that there is an s3 bucket with objects in it
name: Remove buckets
s3_bucket:
name: "my_bucket_with_objects_with_no_versioning"
state: absent
force: yes
Reviewed-by: Mark Chappell
patchback bot
pushed a commit
that referenced
this issue
May 11, 2023
s3_bucket: fix VersionId==null when s3 object not versioned
SUMMARY
Boto3 1.26.129 (possibly earlier) returns "VersionId": "null" from s3_client.list_object_versions() when s3 objects are not versioned. Previously, "VersionId" was None when an s3 object was not versioned. This change broke s3_bucket.destroy_bucket() because the the VersionId was no longer popped (line 1166) when the s3 object was not versioned, and the subsequent attempts to delete the s3 object failed as the "VersionId" was absolutely not "null". Adding in `or fk.get("VersionId")=="null" will catch this new value for non-versioned s3 objects while allowing backwards compatibility with previous versions that return None for "VersionId".
Fixes #1533 s3_bucket.destroy_bucket fails to delete unversioned items
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
s3_bucket.py
ADDITIONAL INFORMATION
Ensure that there is an s3 bucket with objects in it
name: Remove buckets
s3_bucket:
name: "my_bucket_with_objects_with_no_versioning"
state: absent
force: yes
Reviewed-by: Mark Chappell
(cherry picked from commit 94bb14c)
softwarefactory-project-zuul bot
pushed a commit
that referenced
this issue
May 11, 2023
…1545) [PR #1538/94bb14c2 backport][stable-6] s3_bucket: fix VersionId==null when s3 object not versioned This is a backport of PR #1538 as merged into main (94bb14c). SUMMARY Boto3 1.26.129 (possibly earlier) returns "VersionId": "null" from s3_client.list_object_versions() when s3 objects are not versioned. Previously, "VersionId" was None when an s3 object was not versioned. This change broke s3_bucket.destroy_bucket() because the the VersionId was no longer popped (line 1166) when the s3 object was not versioned, and the subsequent attempts to delete the s3 object failed as the "VersionId" was absolutely not "null". Adding in `or fk.get("VersionId")=="null" will catch this new value for non-versioned s3 objects while allowing backwards compatibility with previous versions that return None for "VersionId". Fixes #1533 s3_bucket.destroy_bucket fails to delete unversioned items ISSUE TYPE Bugfix Pull Request COMPONENT NAME s3_bucket.py ADDITIONAL INFORMATION Ensure that there is an s3 bucket with objects in it name: Remove buckets s3_bucket: name: "my_bucket_with_objects_with_no_versioning" state: absent force: yes Reviewed-by: Mark Chappell
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
aws s3_client.list_object_versions() now (at least as of boto3 1.26.129) returns "VersionId": "null" for those objects that have no versions. Per the code in s3_bucket.destroy_bucket (lines 1161-1167) previously it did not return a "VersionId" if there was no versioning and so it popped those
Nonevalues. With these "null" values,s3_client.delete_objects(line 1170) is failing on "Permission Denied". Deletes without the "VersionId" keys work.Recommend the following change to those lines (comments omitted):
Issue Type
Bug Report
Component Name
s3_bucket.destroy_bucket
Ansible Version
$ ansible --versionnot applicable
Collection Versions
$ ansible-galaxy collection listAWS SDK versions
$ pip show boto boto3 botocoreWARNING: Package(s) not found: boto
Name: boto3
Version: 1.24.28
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: c:\users\administrator\anaconda3\lib\site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
Name: botocore
Version: 1.27.28
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: c:\users\administrator\anaconda3\lib\site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
Configuration
$ ansible-config dump --only-changednot applicable - transcribing from an isolated system.
OS / Environment
CENTOS 7
Steps to Reproduce
Expected Results
I expected all objects in the bucket to get deleted, but they did not since there was no versioning applied.
Actual Results
The bucket does not get deleted since the objects in it remained.
Code of Conduct
The text was updated successfully, but these errors were encountered: