Requesting tantivy release to support zstd dependency

[xrl]

Cut new release to support latest zstd

Mac Homebrew installs zstd 1.5.6 and it is not compatible with the tantivy available on crates.io. Pushing the latest tantivy with latest zstd would get builds working on OS X.

% brew info zstd                   
==> zstd: stable 1.5.6 (bottled), HEAD
Zstandard is a real-time compression algorithm
https://facebook.github.io/zstd/
/opt/homebrew/Cellar/zstd/1.5.6 (31 files, 2.1MB) *
  Poured from bottle using the formulae.brew.sh API on 2024-03-28 at 12:53:31
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/z/zstd.rb
License: BSD-3-Clause
==> Dependencies
Build: cmake ✘
Required: lz4 ✔, xz ✔
==> Options
--HEAD
        Install HEAD version
==> Analytics
install: 141,990 (30 days), 342,625 (90 days), 1,666,727 (365 days)
install-on-request: 7,894 (30 days), 11,864 (90 days), 117,268 (365 days)
build-error: 52 (30 days)

and when I try to install the latest tantivy cli from crates.io:

% cargo install tantivy-cli --force
    Updating crates.io index
  Installing tantivy-cli v0.21.0
    Updating crates.io index
[[[ SNIP ]]]
error[E0432]: unresolved import `zstd_sys::ZSTD_cParameter::ZSTD_c_experimentalParam6`
   --> /Users/xlange/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zstd-safe-6.0.6/src/lib.rs:609:13
    |
609 |             ZSTD_c_experimentalParam6 as ZSTD_c_targetCBlockSize,
    |             -------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |             |
    |             no `ZSTD_c_experimentalParam6` in `ZSTD_cParameter`
    |             help: a similar name exists in the module: `ZSTD_c_experimentalParam1`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `zstd-safe` (lib) due to 1 previous error
error: failed to compile `tantivy-cli v0.21.0`, intermediate artifacts can be found at `/var/folders/mc/3pj1vp8s239fn610rbv56f440000gp/T/cargo-installKaoKx6`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

but if I do a little legwork, clone tantivy-cli and manually bump the tantivy dependency:

tantivy-cli % git diff
diff --git a/Cargo.toml b/Cargo.toml
index 9641415..d6d8a40 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -27,7 +27,7 @@ urlencoded = "0.6"
 mount = "0.4"
 log = "0.4"
 env_logger = "0.10"
-tantivy = { git = "https://github.com/quickwit-oss/tantivy/", rev = "5b1bf1a"}
+tantivy = { git = "https://github.com/quickwit-oss/tantivy/", rev = "4e79e11007b6c9fc3d3965beefe33e1ffcf641d3"}
 crossbeam-channel = "0.5.8"

and that build goes through AOK:

% cargo build --release
[[[ SNIP ]]]
tantivy-cli % echo $?
0

[xrl]

The workaround would be to get Mac homebrew to install a previous zstd release. But everyone who uses tantivy would have to figure that out.

[shivam-880]

I am trying to use tantivy lib "0.21.1" instead of cli which is giving error:

error[E0432]: unresolved import `zstd_sys::ZSTD_cParameter::ZSTD_c_experimentalParam6`
   --> /Users/shivamkapoor/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zstd-safe-6.0.6/src/lib.rs:609:13
    |
609 |             ZSTD_c_experimentalParam6 as ZSTD_c_targetCBlockSize,
    |             -------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |             |
    |             no `ZSTD_c_experimentalParam6` in `ZSTD_cParameter`
    |             help: a similar name exists in the module: `ZSTD_c_experimentalParam1`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `zstd-safe` (lib) due to previous error

[xrl]

Right, the tantivy-cli was just an example. Yours is proof as well that what is in crates.io is not compatible with the latest zstd.

[shivam-880]

Updating Cargo.toml seemed to work for now!

# tantivy = { version = "0.21.1" }
tantivy = { git = "https://github.com/quickwit-oss/tantivy/", rev = "4e79e11007b6c9fc3d3965beefe33e1ffcf641d3" }

[xrl]

My next problem is that my project (https://github.com/rdkit-rs/cheminee) cannot be published to crates.io from my Mac because of this build issue. Cargo does not allow releasing crates with git dependencies.

[PSeitz]

A new release is planned for next week

[Ishiihara]

@PSeitz Thank you! Would you please share with us on where we are for the new release?

[PSeitz]

@PSeitz Thank you! Would you please share with us on where we are for the new release?

All the changes for the new release have been merged on main. I plan to release today or tomorrow.

[PSeitz]

tantivy 0.22 has been released!