AWS WebIdentityToken exposure in log files

[wkargul]

https://rustsec.org/advisories/RUSTSEC-2024-0358

When using AWS WebIdentityTokens with the object_store crate, in the event of
a failure and automatic retry, the underlying reqwest error, including the
full URL with the credentials, potentially in the parameters, is written to the
logs.

 object_store v0.9.1
      └── datafusion-proto-common v39.0.0
          └── datafusion-proto v39.0.0
              └── deltalake-core v0.18.1
                  ├── deltalake v0.18.1
                  │   └── raw-service v1.0.4
                  └── deltalake-aws v0.1.2
                      └── deltalake v0.18.1 (*)

[rtyler]

Can you try a local cargo update? The dependencies of deltalake-core 0.18.1 specify a newer object_store, so i'm not sure how the dependency tree shared above is possible 😕

[wkargul]

Tried cargo update -p object_store
and

error: There are multiple `object_store` packages in your project, and the specification `object_store` is ambiguous.
Please re-run this command with one of the following specifications:
  [email protected]
  [email protected]

[wkargul]

Using deltalake = { version = "0.18.1", features = ["datafusion", "s3"] } in my Cargo.toml

[rtyler]

I'm still unable to reproduce, with a new project and a cargo add --features datafusion,s3 I am only getting the latest object_store

plantain% cargo tree | grep -rin "object_store"
    │   │   │   │   ├── in v0.10.1
    │   │   │   │   │   ├── in v0.10.1 (*)
    │   │   │   │   ├── in v0.10.1 (*)
    │   │   │   ├── in v0.10.1 (*)
    │   │   │   │   ├── in v0.10.1 (*)
    │   │   │   ├── in v0.10.1 (*)
    │   │   ├── in v0.10.1 (*)
    │   ├── in v0.10.1 (*)

[wkargul]

I've removed target as well as tried with cargo clean - still have the error

[wkargul]

deltalake-core v0.18.1 depends on datafusion-proto v39.0.0 that uses datafusion-proto-common v39.0.0 that uses legacy object_store @rtyler

[rtyler]

Aha, I see that. That's effectively an error in the datafusion dependency chain which we don't have a way to remedy other than upgrades. our main branch has datafusion 40 already in it, and the next Rust crate releases will include it

[rtyler]

0.18.2 has the fix here i believe