Some tests in test_variants.py may produce segmentation faults

[musicinmybrain]

$ git clone https://github.com/milesgranger/cramjam.git
$ cd cramjam
$ uv venv _e
(_e) $ . _e/bin/activate
(_e) $ uv pip install -e .[dev]
(_e) $ python -m pytest -vs tests/test_variants.py
[…]
tests/test_variants.py::test_variants_different_dtypes[blosc2] Fatal Python error: Segmentation fault
(_e) $ python -m pytest -vs tests/test_variants.py
[…]
tests/test_variants.py::test_variants_different_dtypes[blosc2] Fatal Python error: Segmentation fault

In a build for Fedora Rawhide, I instead hit

tests/test_variants.py::test_variants_decompress_into[blosc2-bytearray-Buffer] Fatal Python error: Segmentation fault

This was on x86_64 only; other architectures succeeded. I’ve also had builds succeed, so it seems like this is “flaky” in practice.

This is probably related to cramjam/cramjam-cli#2 (comment), since I’ve only seen segmentation faults including blosc2 support.

[milesgranger]

Hrm, the common thing here is blosc, but in the CLI it was kinda expected since I know the cause, #160 but here I'm less certain. Especially since it's flaky.

Maybe some misuse of the blosc2 context/session thingy. I'm sorta tempted to take it out at this point until I have time to test/develop it better. However, being it's part of the experimental module I think this is okay now that I write this. By using that I think people are willfully opting into dragons. What do you think?

[musicinmybrain]

Hrm, the common thing here is blosc, but in the CLI it was kinda expected since I know the cause, #160 but here I'm less certain. Especially since it's flaky.

Maybe some misuse of the blosc2 context/session thingy. I'm sorta tempted to take it out at this point until I have time to test/develop it better. However, being it's part of the experimental module I think this is okay now that I write this. By using that I think people are willfully opting into dragons. What do you think?

The fact that blosc2 is in the experimental module is my justification for skipping the variant tests involving blosc2 and going ahead with shipping python-cramjam-2.9.0 and rust-libcramjam-0.6 (for which the C-API shared library doesn’t yet have any blosc2 functions) in Fedora. I don’t love the idea of shipping something with known memory-safety issues, but I agree that having it clearly marked experimental does help a lot.