Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement update operation #1390

Merged
merged 30 commits into from
Jun 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
c2e8324
mvp update operation
Blajda May 16, 2023
0bac94b
rebase on main and fix expression schemas
Blajda May 24, 2023
392e050
error update on non-deterministic predicate
Blajda May 28, 2023
1c48c32
Factor out find files
Blajda May 28, 2023
be4cbb9
remove duplicated test
Blajda May 28, 2023
cd00c6e
Obtain update and copy metrics for operation
Blajda May 29, 2023
ac6bf57
Add additional tests
Blajda Jun 2, 2023
68dffe0
Merge branch 'main' into update-op
Blajda Jun 5, 2023
c52aa09
Allow datafusion expressions or string expressions
Blajda Jun 6, 2023
1188985
Update tests
Blajda Jun 6, 2023
a036705
Update rust/src/operations/update.rs
Blajda Jun 6, 2023
5dc5bdf
Update rust/src/delta_datafusion.rs
Blajda Jun 6, 2023
8be224a
Update rust/src/operations/update.rs
Blajda Jun 6, 2023
1e54a71
Update rust/src/delta_datafusion.rs
Blajda Jun 6, 2023
3b685e6
return early if no files need to be updated. clean up find files impl
Blajda Jun 7, 2023
538848f
Merge branch 'main' into update-op
Blajda Jun 7, 2023
65a26cd
Update rust/src/operations/update.rs
Blajda Jun 7, 2023
ffb3631
Update rust/src/operations/update.rs
Blajda Jun 7, 2023
540c43b
cleanup unwraps
Blajda Jun 7, 2023
a001be0
update comment
Blajda Jun 7, 2023
eb0d867
update iter
Blajda Jun 8, 2023
f603e3d
Merge branch 'main' into update-op
Blajda Jun 8, 2023
fc30ae4
update cast_options docs
Blajda Jun 8, 2023
0f74bc7
cleanup unwraps
Blajda Jun 8, 2023
2f45a06
Update rust/src/delta_datafusion.rs
Blajda Jun 12, 2023
bc4377a
:WIP: resolve schema issues without projection
Blajda Jun 12, 2023
80b27cf
resolve schema issues with projection
Blajda Jun 13, 2023
9c7cd73
resolve merge conflicts with main
Blajda Jun 13, 2023
73137cc
implement comments
Blajda Jun 13, 2023
111c463
store time metrics in u64
Blajda Jun 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion rust/src/action/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,11 @@ pub enum DeltaOperation {
/// The condition the to be deleted data must match
predicate: Option<String>,
},
/// Update data matching predicate from delta table
Update {
/// The update predicate
predicate: Option<String>,
},

/// Represents a Delta `StreamingUpdate` operation.
#[serde(rename_all = "camelCase")]
Expand Down Expand Up @@ -632,6 +637,7 @@ impl DeltaOperation {
DeltaOperation::Create { .. } => "CREATE TABLE",
DeltaOperation::Write { .. } => "WRITE",
DeltaOperation::Delete { .. } => "DELETE",
DeltaOperation::Update { .. } => "UPDATE",
DeltaOperation::StreamingUpdate { .. } => "STREAMING UPDATE",
DeltaOperation::Optimize { .. } => "OPTIMIZE",
DeltaOperation::FileSystemCheck { .. } => "FSCK",
Expand Down Expand Up @@ -675,7 +681,8 @@ impl DeltaOperation {
| Self::FileSystemCheck {}
| Self::StreamingUpdate { .. }
| Self::Write { .. }
| Self::Delete { .. } => true,
| Self::Delete { .. }
| Self::Update { .. } => true,
}
}

Expand All @@ -695,6 +702,7 @@ impl DeltaOperation {
// TODO add more operations
Self::Write { predicate, .. } => predicate.clone(),
Self::Delete { predicate, .. } => predicate.clone(),
Self::Update { predicate, .. } => predicate.clone(),
_ => None,
}
}
Expand Down
Loading