GetObjectAttributes fails when requesting multiple attributes

[nipunn1313]

Describe the bug

Using GetObjectAttributes, if you pass .object_attributes more than once, it fails with a SignatureMismatch error.

If you pass exactly one object attribute, it works fine.

Expected Behavior

Expect successful run with GetObjectAttributesOutput

Current Behavior

Error { code: "SignatureDoesNotMatch", message: "The request signature we calculated does not match the signature you provided. Check your key and signing method.", request_id: "PJA6AWQ2ZER94C80", s3_extended_request_id: "m8ovRg+2pjaRryn3rKIdPg4UtjPHtaTujzn6H5QhjYOnMq4Ee0iUnCmmZkESCPD3JQqzQncFv2w=" }

Reproduction Steps

let result: Result<
    GetObjectAttributesOutput,
    aws_sdk_s3::types::SdkError<GetObjectAttributesError>,
> = self
    .client
    .get_object_attributes()
    .bucket(self.bucket.clone())
    .key(String::from(key))
    // There's some bug in aws rust sdk where you can't request more
    // than one object attribute at the same time. Either one individually
    // works ok.
    .object_attributes(aws_sdk_s3::model::ObjectAttributes::Checksum)
    .object_attributes(aws_sdk_s3::model::ObjectAttributes::ObjectSize)
    .send()
    .await;

Possible Solution

Maybe there's some bug in the signing from rust sdk? Seems to work ok from AWS CLI, so API seems ok.

Additional Information/Context

No response

Version

│   │   │   ├── aws-sdk-s3 v0.24.0
│   │   │   │   ├── aws-credential-types v0.54.1 (*)
│   │   │   │   ├── aws-endpoint v0.54.1 (*)
│   │   │   │   ├── aws-http v0.54.1 (*)
│   │   │   │   ├── aws-sig-auth v0.54.1 (*)
│   │   │   │   ├── aws-sigv4 v0.54.1 (*)
│   │   │   │   ├── aws-smithy-async v0.54.1 (*)
│   │   │   │   ├── aws-smithy-checksums v0.54.1
│   │   │   │   │   ├── aws-smithy-http v0.54.1 (*)
│   │   │   │   │   ├── aws-smithy-types v0.54.1 (*)
│   │   │   │   ├── aws-smithy-client v0.54.1 (*)
│   │   │   │   ├── aws-smithy-eventstream v0.54.1 (*)
│   │   │   │   ├── aws-smithy-http v0.54.1 (*)
│   │   │   │   ├── aws-smithy-http-tower v0.54.1 (*)
│   │   │   │   ├── aws-smithy-json v0.54.1 (*)
│   │   │   │   ├── aws-smithy-types v0.54.1 (*)
│   │   │   │   ├── aws-smithy-xml v0.54.1 (*)
│   │   │   │   ├── aws-types v0.54.1 (*)

Environment details (OS name and version, etc.)

MacOS 13.1

Logs

Here's some logs. I redacted private(ish) object keys that's likely to not be relevant.

 2023-01-28T04:03:13.304Z TRACE aws_smithy_http_tower::dispatch > dispatching request request=Request { method: GET, uri: https://test-convex-files.s3.us-east-1.amazonaws.com/{redacted object key}?attributes, version: HTTP/1.1, headers: {"x-amz-object-attributes": "Checksum", "x-amz-object-attributes": "ObjectSize", "user-agent": "aws-sdk-rust/0.54.1 os/macos lang/rust/1.66.0-nightly", "x-amz-user-agent": "aws-sdk-rust/0.54.1 api/s3/0.24.0 os/macos lang/rust/1.66.0-nightly", "x-amz-date": "20230128T040313Z", "authorization": Sensitive, "x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}, body: SdkBody { inner: Once(Some(b"")), retryable: true } }
 2023-01-28T04:03:13.621Z TRACE aws_smithy_http::middleware     > read HTTP response body http_response=Response { status: 403, version: HTTP/1.1, headers: {"x-amz-request-id": "PJA6AWQ2ZER94C80", "x-amz-id-2": "m8ovRg+2pjaRryn3rKIdPg4UtjPHtaTujzn6H5QhjYOnMq4Ee0iUnCmmZkESCPD3JQqzQncFv2w=", "content-type": "application/xml", "transfer-encoding": "chunked", "date": "Sat, 28 Jan 2023 04:03:13 GMT", "server": "AmazonS3"}, body: b"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAZ7FI5RP2XSZRGDJU</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256\n20230128T040313Z\n20230128/us-east-1/s3/aws4_request\nd8e2ece11034bb46dab2b3736c098880284d9f822e8b915be13f19f49c4e6eec</StringToSign><SignatureProvided>abcc3b7f77c4d04c2ca3d27b4af90bdbe4713dca6da8ecdb8c04ff6578bd840e</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 33 30 31 32 38 54 30 34 30 33 31 33 5a 0a 32 30 32 33 30 31 32 38 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 64 38 65 32 65 63 65 31 31 30 33 34 62 62 34 36 64 61 62 32 62 33 37 33 36 63 30 39 38 38 38 30 32 38 34 64 39 66 38 32 32 65 38 62 39 31 35 62 65 31 33 66 31 39 66 34 39 63 34 65 36 65 65 63</StringToSignBytes><CanonicalRequest>GET\n/{redacted object key}\nattributes=\nhost:test-convex-files.s3.us-east-1.amazonaws.com\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20230128T040313Z\nx-amz-object-attributes:Checksum,ObjectSize\nx-amz-user-agent:aws-sdk-rust/0.54.1 api/s3/0.24.0 os/macos lang/rust/1.66.0-nightly\n\nhost;x-amz-content-sha256;x-amz-date;x-amz-object-attributes;x-amz-user-agent\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest><CanonicalRequestBytes>47 45 54 0a 2f 63 61 72 6e 69 74 61 73 2d 63 39 66 63 62 66 63 65 2d 34 38 30 38 2d 34 35 61 32 2d 38 39 66 33 2d 34 33 33 39 36 35 33 36 61 33 64 35 2f 35 66 61 61 39 38 37 64 2d 38 38 66 63 2d 34 65 30 32 2d 39 38 37 33 2d 32 31 39 39 37 30 37 34 61 61 66 66 0a 61 74 74 72 69 62 75 74 65 73 3d 0a 68 6f 73 74 3a 74 65 73 74 2d 63 6f 6e 76 65 78 2d 66 69 6c 65 73 2e 73 33 2e 75 73 2d 65 61 73 74 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 32 33 30 31 32 38 54 30 34 30 33 31 33 5a 0a 78 2d 61 6d 7a 2d 6f 62 6a 65 63 74 2d 61 74 74 72 69 62 75 74 65 73 3a 43 68 65 63 6b 73 75 6d 2c 4f 62 6a 65 63 74 53 69 7a 65 0a 78 2d 61 6d 7a 2d 75 73 65 72 2d 61 67 65 6e 74 3a 61 77 73 2d 73 64 6b 2d 72 75 73 74 2f 30 2e 35 34 2e 31 20 61 70 69 2f 73 33 2f 30 2e 32 34 2e 30 20 6f 73 2f 6d 61 63 6f 73 20 6c 61 6e 67 2f 72 75 73 74 2f 31 2e 36 36 2e 30 2d 6e 69 67 68 74 6c 79 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 6f 62 6a 65 63 74 2d 61 74 74 72 69 62 75 74 65 73 3b 78 2d 61 6d 7a 2d 75 73 65 72 2d 61 67 65 6e 74 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35</CanonicalRequestBytes><RequestId>PJA6AWQ2ZER94C80</RequestId><HostId>m8ovRg+2pjaRryn3rKIdPg4UtjPHtaTujzn6H5QhjYOnMq4Ee0iUnCmmZkESCPD3JQqzQncFv2w=</HostId></Error>" }

[nipunn1313]

Did some more debugging and isolated the problem to here:

 2023-01-28T08:34:20.990Z TRACE aws_sigv4::http_request::sign > signing request request=SignableRequest { method: GET, uri: https://test-convex-files.s3.us-east-1.amazonaws.com/{redacted object key}?attributes, headers: {"x-amz-object-attributes": "Checksum", "x-amz-object-attributes": "ObjectSize", "user-agent": "aws-sdk-rust/0.54.1 os/macos lang/rust/1.66.0-nightly", "x-amz-user-agent": "aws-sdk-rust/0.54.1 api/s3/0.24.0 os/macos lang/rust/1.66.0-nightly"}, body: Bytes([]) } params=SigningParams { access_key: "redacted", secret_key: "redacted", security_token: None, region: "us-east-1", service_name: "s3", time: SystemTime { tv_sec: 1674894860, tv_nsec: 990574000 }, settings: SigningSettings { percent_encoding_mode: Single, payload_checksum_kind: XAmzSha256, signature_location: Headers, expires_in: None, excluded_headers: Some(["user-agent"]), uri_path_normalization_mode: Disabled } }
 2023-01-28T08:34:20.991Z TRACE aws_sigv4::http_request::sign > canonical_request=GET
{redacted object key}
attributes=
host:test-convex-files.s3.us-east-1.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20230128T083420Z
x-amz-object-attributes:Checksum
x-amz-object-attributes:Checksum
x-amz-user-agent:aws-sdk-rust/0.54.1 api/s3/0.24.0 os/macos lang/rust/1.66.0-nightly

^ as you can see the canonicalization of the request is incorrect in the case of repeated headers (per docs https://docs.aws.amazon.com/general/latest/gr/create-signed-request.html#create-canonical-request)

[nipunn1313]

Fixed with smithy-lang/smithy-rs#2261!

[ysaito1001]

Thank you for looking into the issue and creating a PR out of it! We'll review the submitted PR.

[ysaito1001]

Fix released in https://github.com/awslabs/aws-sdk-rust/releases/tag/release-2023-03-30.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.