Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: improve AWS credential loading between S3 and DynamoDb code paths #2887

Merged
merged 4 commits into from
Sep 17, 2024
Merged

feat: improve AWS credential loading between S3 and DynamoDb code paths #2887

merged 4 commits into from
Sep 17, 2024

Commits on Sep 17, 2024

  1. feat: improve AWS credential loading between S3 and DynamoDb code paths 

    This change implements a number of improvements to the code paths for
loading credentials.  This is a prerequisite to fixing assume role
support delta-io#2879 but should also address a number of bugs I noticed:

* Python libraries can pass in keys via `storage_options` which are used
  for configuration of the AmazonS3 ObjectStore, but those credentials
  would not be used in the construction of the DynamoDB connection
* Using AWS credentials such as those from ~/.aws/profile or SSO would
  not be properly dropped into the AmazonS3 object store creation

There is some additional work that needs to come in to clean up how
various options overrides are managed still.

Sponsored-by: Scribd Inc.
    @rtyler
    rtyler committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7409ab5 View commit details
    Browse the repository at this point in the history

  2. fix: upgrade aws dependencies to incorporate newer bugfixes 

    the latest aws-config does a proper sequencing on the credential
resolution and avoids unnecessary queries to IMDSv2
    @rtyler
    rtyler committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    46f2793 View commit details
    Browse the repository at this point in the history

  3. fix: allow for assuming a role when bootstrapping AWS credential conf… 

    …iguration

Fixes delta-io#2879
    @rtyler
    rtyler committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    9e2d000 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into bug/iam-assume-role

    @rtyler
    rtyler authored Sep 17, 2024
    Configuration menu
    Copy the full SHA
    9846bac View commit details
    Browse the repository at this point in the history