Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building multiple ort-sys with download-binaries may fail #322

Closed
qryxip opened this issue Nov 23, 2024 · 1 comment
Closed

Building multiple ort-sys with download-binaries may fail #322

qryxip opened this issue Nov 23, 2024 · 1 comment

Comments

@qryxip
Copy link
Contributor

qryxip commented Nov 23, 2024

When you use ort with download-binaries both in dependencies and in build-dependencies, in the following code, in-progress lib_dir may be used.

ort/ort-sys/build.rs

Lines 462 to 467 in 548bfed

let lib_dir = cache_dir.join(ORT_EXTRACT_DIR);
if !lib_dir.exists() {
let downloaded_file = fetch_file(prebuilt_url);
assert!(verify_file(&downloaded_file, prebuilt_hash), "hash of downloaded ONNX Runtime binary does not match!");
extract_tgz(&downloaded_file, &cache_dir);
}

Perhaps fetch-models has the same problem.

Solutions I come up with

  1. Lock lib_dir with fs4 or something

    FYI, Cargo calls flock(2) and FileLockExt directly.

  2. Extract the TGZ in a temporary directory and move it to lib_dir

  3. Detect if lib_dir is in-progress and just panic with a human readable message

Reproduction

Not yet, but I feel it happens for EPs.

@decahedron1
Copy link
Member

decahedron1 commented Nov 23, 2024

Should be fixed by eb51646 (& effd7a7)

qryxip pushed a commit to qryxip/ort that referenced this issue Nov 24, 2024
(cherry picked from commit eb51646)

Conflicts:
	ort-sys/build.rs
	ort-sys/src/internal/mod.rs
	src/session/builder/impl_commit.rs
qryxip added a commit to VOICEVOX/ort that referenced this issue Nov 25, 2024
Fixes the problem where building multiple ort-sys `download-binaries`
fail.

Cherry picked from eb51646, eb51646, and pykeio#323.

Refs: VOICEVOX/voicevox_core#859, pykeio#322
Refs: eb51646
Refs: effd7a7, pykeio#323
Co-authored-by: Carson M. <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants