-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(blockifier_reexecution): upload and download reexecution files #2229
base: main
Are you sure you want to change the base?
Conversation
Artifacts upload triggered. View details here |
7813040
to
6d3fbd4
Compare
Artifacts upload triggered. View details here |
Benchmark movements: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 3 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 9 unresolved discussions (waiting on @aner-starkware and @AvivYossef-starkware)
crates/blockifier_reexecution/Cargo.toml
line 18 at r2 (raw file):
clap = { workspace = true, features = ["cargo", "derive"] } flate2.workspace = true google-cloud-storage = "0.22.1"
pending approval
Code quote:
google-cloud-storage = "0.22.1"
crates/blockifier_reexecution/src/main.rs
line 245 at r2 (raw file):
Command::UploadFiles { block_numbers, directory_path } => { let directory_path = directory_path.unwrap_or("./crates/blockifier_reexecution/resources".to_string());
this string appears several times in this module; please make a named constant
Code quote:
"./crates/blockifier_reexecution/resources"
crates/blockifier_reexecution/src/main.rs
line 251 at r2 (raw file):
let files_prefix: String = fs::read_to_string(directory_path.clone() + "/offline_reexecution_files_prefix")
used twice; make it a constant?
Code quote:
"/offline_reexecution_files_prefix"
crates/blockifier_reexecution/src/main.rs
line 258 at r2 (raw file):
// Get the client with authentication. let config = ClientConfig::default().with_auth().await.unwrap();
this will fail if devs didn't log into google cloud?
when would this fail?
we may want a more informative error in an expect()
Code quote:
let config = ClientConfig::default().with_auth().await.unwrap();
crates/blockifier_reexecution/src/main.rs
line 271 at r2 (raw file):
assert!( client .get_object(&GetObjectRequest {
is there a different request type that just queries if the file exists? shame to download the entire thing before returning an error.
or, is this what get_object
(as opposed to download_object
) does?
Code quote:
GetObjectRequest
crates/blockifier_reexecution/src/main.rs
line 289 at r2 (raw file):
.upload_object( &UploadObjectRequest { bucket: "reexecution_artifacts".to_string(),
this bucket name should be a constant
Code quote:
"reexecution_artifacts"
crates/blockifier_reexecution/src/main.rs
line 305 at r2 (raw file):
} println!("All blocks uploaded successfully.");
can you add a link to the new directory / new objects on GCS here?
Code quote:
println!("All blocks uploaded successfully.");
crates/blockifier_reexecution/src/main.rs
line 334 at r2 (raw file):
bucket: "reexecution_artifacts".to_string(), object: files_prefix.clone() + &format!("block_{block_number}/reexecution_data.json"),
this string is duplicated many times, sometimes with a different prefix, but still; please put it in a variable / lambda function for single point of truth
Code quote:
"block_{block_number}/reexecution_data.json"
crates/blockifier_reexecution/src/main.rs
line 349 at r2 (raw file):
} println!("All blocks downloaded successfully.");
Suggestion:
println!("All blocks downloaded successfully to {directory_path}.");
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2229 +/- ##
===========================================
+ Coverage 40.10% 77.22% +37.11%
===========================================
Files 26 386 +360
Lines 1895 40467 +38572
Branches 1895 40467 +38572
===========================================
+ Hits 760 31249 +30489
- Misses 1100 6910 +5810
- Partials 35 2308 +2273 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
No description provided.