Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deltalake 0.7.0 with s3 feature compliation error due to rusoto_dynamodb version conflict #1191

Closed
ylashin opened this issue Feb 28, 2023 · 0 comments · Fixed by #1202
Closed
Labels
bug Something isn't working

Comments

@ylashin
Copy link

ylashin commented Feb 28, 2023

Environment

Delta-rs: 0.7.0

Binding: Rust

Environment:

  • Cloud provider: AWS
  • OS: MacOS Venture
  • Other: rustc 1.67.1

Bug

What happened:
It sounds like duplicate of #973 which is declared fixed but seems not yet. The version numbers of the conflict are different this time.

Trying to compile a simple program that enables the s3 feature fails to compile due to a version mismatch in rusoto_dynamodb dependency

What you expected to happen:
Successful compliation

How to reproduce it:
main.rs:

#[tokio::main]
async fn main() {
    let table = deltalake::open_table_with_version("s3://foo", 0).await?;
    let files = table.get_files();
    println!("files = {:?}", files);
}

cargo.toml

[package]
name = "rsuty-delta-aws"
version = "0.1.0"
edition = "2021"

[dependencies]
deltalake = { version = "0.7.0", features = ["s3"] }
tokio = { version = "1.25.0", features = ["full"] }

Then compilation using cargo build produces the following:

error[E0308]: mismatched types
    --> /Users/yousry/.cargo/registry/src/github.com-1ecc6299db9ec823/deltalake-0.7.0/src/storage/s3.rs:527:17
     |
526  |             let lock_client = dynamodb_lock::DynamoDbLockClient::new(
     |                               -------------------------------------- arguments to this function are incorrect
527  |                 dynamodb_client,
     |                 ^^^^^^^^^^^^^^^ expected struct `rusoto_dynamodb::generated::DynamoDbClient`, found struct `DynamoDbClient`
     |
     = note: struct `DynamoDbClient` and struct `rusoto_dynamodb::generated::DynamoDbClient` have similar names, but are actually distinct types
note: struct `DynamoDbClient` is defined in crate `rusoto_dynamodb`
    --> /Users/yousry/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_dynamodb-0.47.0/src/generated.rs:6449:1
     |
6449 | pub struct DynamoDbClient {
     | ^^^^^^^^^^^^^^^^^^^^^^^^^
note: struct `rusoto_dynamodb::generated::DynamoDbClient` is defined in crate `rusoto_dynamodb`
    --> /Users/yousry/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_dynamodb-0.48.0/src/generated.rs:6449:1
     |
6449 | pub struct DynamoDbClient {
     | ^^^^^^^^^^^^^^^^^^^^^^^^^
     = note: perhaps two different versions of crate `rusoto_dynamodb` are being used?
note: associated function defined here
    --> /Users/yousry/.cargo/registry/src/github.com-1ecc6299db9ec823/dynamodb_lock-0.4.2/src/lib.rs:341:12
     |
341  |     pub fn new(client: DynamoDbClient, opts: DynamoDbOptions) -> Self {
     |            ^^^

For more information about this error, try `rustc --explain E0308`.
error: could not compile `deltalake` due to previous error
warning: build failed, waiting for other jobs to finish...

More details:
cargo tree can be used to track the issue but here are some screenshots.

deltalake itself references rusoto_dynamodb v 0.47.0

image

dynamodb_lock references rusoto_dynamodb v 0.48.0

image

For some reason, it seems that dynamodb_lock has been released while still using rusoto_dynamodb v 0.48.0.

image

I just wanted to share some references here and there although I am not really expert or across all those versioning changes.

@ylashin ylashin added the bug Something isn't working label Feb 28, 2023
wjones127 added a commit that referenced this issue Mar 1, 2023
# Description

cc @houqp @rtyler will need one of you to help with publishing a new
release.

# Related Issue(s)

For example:

- closes #1191

# Documentation

<!---
Share links to useful documentation
--->
chitralverma pushed a commit to chitralverma/delta-rs that referenced this issue Mar 17, 2023
# Description

cc @houqp @rtyler will need one of you to help with publishing a new
release.

# Related Issue(s)

For example:

- closes delta-io#1191

# Documentation

<!---
Share links to useful documentation
--->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant