Closing BGZI gives a 'munmap_chunk(): invalid pointer'

[xbello]

Using Nim 2.0.2, hts-nim 0.3.25.

(Also, depending of the source, the error is SIGSEGV: Illegal storage access. (Attempt to read from nil?)).

The minimal example I can do to make it fail is this:

var b: BGZI
doAssert(variants.open("tests/test_files/sample.tsv.gz")
b.close()

When compiled with ARC/ORC, it runs but fails with the error menctioned in the title 'munmap_chunk(): invalid pointer'. If compiled with the flag -mm:refc everything goes smoothly.

I think I've pinpointed the problem to hts_idx_set_meta

hts-nim/src/hts/csi.nim

Line 86 in 9efe3c6

return int(hts_idx_set_meta(idx, uint32(len(csi.meta)), csi.meta[0].addr, do_copy))

because if you return early in with return 0 just before the true return, the error goes away. It seems that there were problems here before (samtools/htslib#936).

Contents of "sample.tsv.gz":

chr1	10001	T	A
chr1	10002	A	C
chr12	10003	A	G

With a "sample.tsv.gz.csi" file besides it.

[brentp]

Hi Xabier, thanks for reporting. I pushed a fix for this.
There are still many things that need to change throughout hts-nim to be fully compatible with ORC/ARC. I'll keep checking those off as I have time, but PRs are also welcome.
Please let me know if you hit other issues.

[xbello]

I would love to help you more on this, I spend a lot of time with this issue trying to fix it and make a PR. But I couldn't understand what does the proc idx_set_meta with all that pointer code. Best I could do was to isolate the failure and create the issue.

Right now I'm overriding the BGZI.close with my own procedure that just skips the CSI sync step, which is less than ideal, but at least my code compiles with ORC and runs correctly (it only reads the BGZI files).

[brentp]

Thanks for the test files and diagnosing the exact location of the problem!
I can't remember how idx_set_meta works either. 😢