no auth scheme matched auth scheme options error when using `no_credentials` with put object #878

messense · 2023-08-25T08:24:29Z

Describe the bug

when using no_credentials() aws-sdk-s3 throws
no auth scheme matched auth scheme options. This is a bug. Please file an issue., same as #864 but only occurred in write operations like put_object.

Expected Behavior

Disable credentials and signing.

Current Behavior

errors with "no auth scheme matched auth scheme options. This is a bug. Please file an issue."

Reproduction Steps

use aws_sdk_s3 as s3;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = ::aws_config::from_env().no_credentials().load().await;
    let client = s3::Client::new(&config);

    let res = client
        .put_object()
        .bucket("test")
        .key("test")
        .body(b"test".to_vec().into())
        .send()
        .await?;
    println!("{:?}", res);
    Ok(())
}

Possible Solution

No response

Additional Information/Context

No response

Version

├── aws-config v0.56.1
│   ├── aws-credential-types v0.56.1
│   │   ├── aws-smithy-async v0.56.1
│   │   ├── aws-smithy-types v0.56.1
│   ├── aws-http v0.56.1
│   │   ├── aws-credential-types v0.56.1 (*)
│   │   ├── aws-smithy-http v0.56.1
│   │   │   ├── aws-smithy-eventstream v0.56.1
│   │   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-types v0.56.1
│   │   │   ├── aws-credential-types v0.56.1 (*)
│   │   │   ├── aws-smithy-async v0.56.1 (*)
│   │   │   ├── aws-smithy-client v0.56.1
│   │   │   │   ├── aws-smithy-async v0.56.1 (*)
│   │   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   │   ├── aws-smithy-http-tower v0.56.1
│   │   │   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   ├── aws-smithy-types v0.56.1 (*)
│   ├── aws-sdk-sso v0.30.0
│   │   ├── aws-credential-types v0.56.1 (*)
│   │   ├── aws-http v0.56.1 (*)
│   │   ├── aws-runtime v0.56.1
│   │   │   ├── aws-credential-types v0.56.1 (*)
│   │   │   ├── aws-http v0.56.1 (*)
│   │   │   ├── aws-sigv4 v0.56.1
│   │   │   │   ├── aws-smithy-eventstream v0.56.1 (*)
│   │   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   ├── aws-smithy-async v0.56.1 (*)
│   │   │   ├── aws-smithy-eventstream v0.56.1 (*)
│   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   ├── aws-smithy-runtime-api v0.56.1
│   │   │   │   ├── aws-smithy-async v0.56.1 (*)
│   │   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   │   ├── aws-types v0.56.1 (*)
│   │   ├── aws-smithy-async v0.56.1 (*)
│   │   ├── aws-smithy-client v0.56.1 (*)
│   │   ├── aws-smithy-http v0.56.1 (*)
│   │   ├── aws-smithy-json v0.56.1
│   │   │   └── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-smithy-runtime v0.56.1
│   │   │   ├── aws-smithy-async v0.56.1 (*)
│   │   │   ├── aws-smithy-client v0.56.1 (*)
│   │   │   ├── aws-smithy-http v0.56.1 (*)
│   │   │   ├── aws-smithy-runtime-api v0.56.1 (*)
│   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-smithy-runtime-api v0.56.1 (*)
│   │   ├── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-types v0.56.1 (*)
│   ├── aws-sdk-sts v0.30.0
│   │   ├── aws-credential-types v0.56.1 (*)
│   │   ├── aws-http v0.56.1 (*)
│   │   ├── aws-runtime v0.56.1 (*)
│   │   ├── aws-smithy-async v0.56.1 (*)
│   │   ├── aws-smithy-client v0.56.1 (*)
│   │   ├── aws-smithy-http v0.56.1 (*)
│   │   ├── aws-smithy-json v0.56.1 (*)
│   │   ├── aws-smithy-query v0.56.1
│   │   │   ├── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-smithy-runtime v0.56.1 (*)
│   │   ├── aws-smithy-runtime-api v0.56.1 (*)
│   │   ├── aws-smithy-types v0.56.1 (*)
│   │   ├── aws-smithy-xml v0.56.1
│   │   ├── aws-types v0.56.1 (*)
│   ├── aws-smithy-async v0.56.1 (*)
│   ├── aws-smithy-client v0.56.1 (*)
│   ├── aws-smithy-http v0.56.1 (*)
│   ├── aws-smithy-http-tower v0.56.1 (*)
│   ├── aws-smithy-json v0.56.1 (*)
│   ├── aws-smithy-types v0.56.1 (*)
│   ├── aws-types v0.56.1 (*)
├── aws-sdk-s3 v0.30.0
│   ├── aws-credential-types v0.56.1 (*)
│   ├── aws-http v0.56.1 (*)
│   ├── aws-runtime v0.56.1 (*)
│   ├── aws-sigv4 v0.56.1 (*)
│   ├── aws-smithy-async v0.56.1 (*)
│   ├── aws-smithy-checksums v0.56.1
│   │   ├── aws-smithy-http v0.56.1 (*)
│   │   ├── aws-smithy-types v0.56.1 (*)
│   ├── aws-smithy-client v0.56.1 (*)
│   ├── aws-smithy-eventstream v0.56.1 (*)
│   ├── aws-smithy-http v0.56.1 (*)
│   ├── aws-smithy-json v0.56.1 (*)
│   ├── aws-smithy-runtime v0.56.1 (*)
│   ├── aws-smithy-runtime-api v0.56.1 (*)
│   ├── aws-smithy-types v0.56.1 (*)
│   ├── aws-smithy-xml v0.56.1 (*)
│   ├── aws-types v0.56.1 (*)

Environment details (OS name and version, etc.)

Ubuntu 22.04

Logs

2023-08-25T08:22:31.721380Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}: aws_config::fs_util: loaded home directory src="HOME"
2023-08-25T08:22:31.721446Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}:load_config_file{file=Default(Config)}: aws_config::profile::parser::source: performing home directory substitution home="/home/ubuntu" path="~/.aws/config"
2023-08-25T08:22:31.721477Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}:load_config_file{file=Default(Config)}: aws_config::profile::parser::source: home directory expanded before="~/.aws/config" after="/home/ubuntu/.aws/config"
2023-08-25T08:22:31.721510Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}:load_config_file{file=Default(Config)}: aws_config::profile::parser::source: config file loaded path=Some("/home/ubuntu/.aws/config") size=169
2023-08-25T08:22:31.721560Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}:load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: performing home directory substitution home="/home/ubuntu" path="~/.aws/credentials"
2023-08-25T08:22:31.721582Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}:load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: home directory expanded before="~/.aws/credentials" after="/home/ubuntu/.aws/credentials"
2023-08-25T08:22:31.721608Z DEBUG load_region{provider=ProfileFileRegionProvider { provider_config: ProviderConfig { env: Env(Real), fs: Fs(Real), sleep: Some(SharedAsyncSleep(TokioSleep)), region: None } }}:load_config_file{file=Default(Credentials)}: aws_config::profile::parser::source: config file loaded path=Some("/home/ubuntu/.aws/credentials") size=920
2023-08-25T08:22:31.732160Z DEBUG hyper_rustls::config: with_native_roots processed 331 valid and 0 invalid certs
2023-08-25T08:22:31.751405Z DEBUG aws_sdk_s3::config: using retry strategy with partition 's3'
2023-08-25T08:22:31.751705Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'serialization' phase
2023-08-25T08:22:31.751833Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime_api::client::interceptors::context: entering 'before transmit' phase
2023-08-25T08:22:31.751901Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::retries::strategy::standard: no client rate limiter configured, so no token is required for the initial request.
2023-08-25T08:22:31.751926Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::orchestrator: retry strategy has OKed initial request
2023-08-25T08:22:31.751956Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::orchestrator: beginning attempt #1
2023-08-25T08:22:31.765882Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator::endpoints: will use endpoint Endpoint { url: "https://test.s3.us-east-1.amazonaws.com", headers: {}, properties: {"authSchemes": Array([Object({"signingName": String("s3"), "name": String("sigv4"), "signingRegion": String("us-east-1"), "disableDoubleEncoding": Bool(true)})])} }
2023-08-25T08:22:31.766017Z DEBUG invoke{service=s3 operation=PutObject}:try_op:try_attempt: aws_smithy_runtime::client::orchestrator: encountered orchestrator error; halting
2023-08-25T08:22:31.766104Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::retries::strategy::standard: encountered unretryable error attempts=1 max_attempts=5
2023-08-25T08:22:31.766130Z DEBUG invoke{service=s3 operation=PutObject}:try_op: aws_smithy_runtime::client::orchestrator: a retry is either unnecessary or not possible, exiting attempt loop

The text was updated successfully, but these errors were encountered:

jdisanti · 2023-08-25T16:43:18Z

The S3 model doesn't state which operations work without credentials, so I did some guess work in the fix for #864 and chose ListObjects, ListObjectsV2, HeadObject, and GetObject to work without them. I need to look into this more.

messense · 2023-08-28T01:49:31Z

What I'm after is an equivalent of --no-sign-request option of aws-cli, so I want every operation to respect no_credentials().

jdisanti · 2023-08-28T22:45:19Z

Working on a fix. I'm going to open up all the S3 operations to no_credentials.

In #2907, I created an allow list of S3 operations to add `@optionalAuth` to, but this turns out to be too restrictive, as seen in awslabs/aws-sdk-rust#878. This PR restores the original middleware behavior of allowing optional auth for all S3 operations. ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

jdisanti · 2023-08-29T22:39:30Z

A fix for this will be included in the next minor release.

messense · 2023-09-20T07:14:39Z

A fix for this will be included in the next minor release.

Just tested the newly released aws-sdk-s3 0.31.0, the issue still occurs.

github-actions · 2023-09-20T13:26:09Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

jdisanti · 2023-09-20T16:30:19Z

My bad. I merged the fix into our main branch instead of the patch release branch, so it didn't go out with 0.31 😞

jdisanti · 2023-09-20T19:17:53Z

Just a heads up, we're exercising our release infrastructure, so the next couple weeks or so of releases aren't going to include this fix. It will go out after we cut a non-patch release of smithy-rs.

rcoh · 2023-11-02T19:58:15Z

This is going out in 0.35 (currently queued up to release)

github-actions · 2023-11-02T19:58:32Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

rcoh · 2023-11-07T16:34:29Z

Verified fixed in 0.35

messense added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Aug 25, 2023

jdisanti removed the needs-triage This issue or PR still needs to be triaged. label Aug 25, 2023

jdisanti self-assigned this Aug 28, 2023

jdisanti added the p2 This is a standard priority issue label Aug 28, 2023

jdisanti mentioned this issue Aug 28, 2023

Add @optionalAuth to all S3 operations smithy-lang/smithy-rs#2955

Merged

jdisanti added the pending-release This issue will be fixed by an approved PR that hasn't been released yet. label Aug 29, 2023

rcoh closed this as completed Sep 20, 2023

jdisanti reopened this Sep 20, 2023

rcoh closed this as completed Nov 2, 2023

rcoh removed the pending-release This issue will be fixed by an approved PR that hasn't been released yet. label Nov 7, 2023

NickDarvey mentioned this issue Jun 25, 2024

chore: upgrade aws-config for aarch64-pc-windows support lancedb/lance#2510

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

no auth scheme matched auth scheme options error when using `no_credentials` with put object #878

no auth scheme matched auth scheme options error when using `no_credentials` with put object #878

messense commented Aug 25, 2023 •

edited

Loading

jdisanti commented Aug 25, 2023

messense commented Aug 28, 2023

jdisanti commented Aug 28, 2023

jdisanti commented Aug 29, 2023

messense commented Sep 20, 2023

github-actions bot commented Sep 20, 2023

jdisanti commented Sep 20, 2023

jdisanti commented Sep 20, 2023

rcoh commented Nov 2, 2023

github-actions bot commented Nov 2, 2023

rcoh commented Nov 7, 2023

no auth scheme matched auth scheme options error when using no_credentials with put object #878

no auth scheme matched auth scheme options error when using no_credentials with put object #878

Comments

messense commented Aug 25, 2023 • edited Loading

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

Version

Environment details (OS name and version, etc.)

Logs

jdisanti commented Aug 25, 2023

messense commented Aug 28, 2023

jdisanti commented Aug 28, 2023

jdisanti commented Aug 29, 2023

messense commented Sep 20, 2023

github-actions bot commented Sep 20, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

jdisanti commented Sep 20, 2023

jdisanti commented Sep 20, 2023

rcoh commented Nov 2, 2023

github-actions bot commented Nov 2, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

rcoh commented Nov 7, 2023

no auth scheme matched auth scheme options error when using `no_credentials` with put object #878

no auth scheme matched auth scheme options error when using `no_credentials` with put object #878

messense commented Aug 25, 2023 •

edited

Loading