Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add contentMD5 for putObject to support object locking #394

Merged
merged 2 commits into from
Jan 27, 2023

Conversation

tPl0ch
Copy link
Contributor

@tPl0ch tPl0ch commented Jan 26, 2023

Motivation

We are hitting an error when using this library because our bucket is configured with object locking:

Content-MD5 OR x-amz-checksum- HTTP header is required for Put Object requests with Object Lock parameters
(Service: S3, Status Code: 400, Request ID: XXX, Extended Request ID: XXX)

The AWS S3 API requires passing the Content-MD5 option for putObject requests that target buckets with object locking enabled.

From the putObject API description:

To ensure that data is not corrupted traversing the network, use the Content-MD5 header.
When you use this header, Amazon S3 checks the object against the provided MD5 value and,
if they do not match, returns an error. Additionally, you can calculate the MD5 while
putting an object to Amazon S3 and compare the returned ETag to the calculated MD5 value.

The Content-MD5 header is required for any request to upload an object with a retention
period configured using Amazon S3 Object Lock. For more information about Amazon S3 Object
Lock, see Amazon S3 Object Lock Overview in the Amazon S3 User Guide.

This MR adds a BC support for passing an Option[String] as contentMD5, which is a base64 encoded MD5 hash. The library users will be responsible to provide a correct contentMD5.

Urgency

This is extremely urgent for us since it blocks the go-live of an important project. So if you @regis-leray could have a look shortly and hopefully merge this change to the lib we could use a snapshot version to start our go-live. Thank you very much in advance!

@tPl0ch tPl0ch requested a review from a team as a code owner January 26, 2023 11:01
@regis-leray
Copy link
Member

@tPl0ch can you please fix the javadoc thank you

We are hitting an error when using this library because our bucket is configured with
[object locking](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html):

```
Content-MD5 OR x-amz-checksum- HTTP header is required for Put Object requests with Object Lock parameters
(Service: S3, Status Code: 400, Request ID: XXX, Extended Request ID: XXX)
```

The AWS S3 API requires passing the `Content-MD5` option for `putObject` requests that
target buckets with object locking enabled.

From the [`putObject` API description](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html):

> To ensure that data is not corrupted traversing the network, use the Content-MD5 header.
  When you use this header, Amazon S3 checks the object against the provided MD5 value and,
  if they do not match, returns an error. Additionally, you can calculate the MD5 while
  putting an object to Amazon S3 and compare the returned ETag to the calculated MD5 value.

> The Content-MD5 header is required for any request to upload an object with a retention
  period configured using Amazon S3 Object Lock. For more information about Amazon S3 Object
  Lock, see Amazon S3 Object Lock Overview in the Amazon S3 User Guide.

This MR adds a BC support for passing an `Option[String]` as contentMD5, which is a base64
encoded MD5 hash. The library users will be responsible to provide a correct contentMD5.
@tPl0ch tPl0ch force-pushed the feature-content-md5 branch from 248dd25 to 58d2ea2 Compare January 27, 2023 18:46
@tPl0ch
Copy link
Contributor Author

tPl0ch commented Jan 27, 2023

@regis-leray done. Thanks for the quick response, much appreciated! ❤️

@regis-leray regis-leray merged commit 05ab72c into zio:series/2.x Jan 27, 2023
@tPl0ch tPl0ch deleted the feature-content-md5 branch January 28, 2023 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants