LeakSanitizer has encountered a fatal error

[sitsofe]

When statically compiling a closed source piece of software LeakSanitizer regularly encounters a fatal error:

# LSAN_OPTIONS=verbosity=1:log_threads=1 /scratch/disco.static --host=none
==17188==AddressSanitizer: failed to intercept '__isoc99_printf'
==17188==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==17188==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==17188==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==17188==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==17188==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==17188==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==17188==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==17188==AddressSanitizer: failed to intercept 'xdr_quad_t'
==17188==AddressSanitizer: failed to intercept 'xdr_u_quad_t'
==17188==AddressSanitizer: failed to intercept '__cxa_throw'
==17188==AddressSanitizer: libc interceptors initialized
|| `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
|| `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
|| `[0x00008fff7000, 0x02008fff6fff]` || ShadowGap  ||
|| `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
|| `[0x000000000000, 0x00007fff7fff]` || LowMem     ||
MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00 0x02008fff6fff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
malloc_context_size=30
SHADOW_SCALE: 3
SHADOW_GRANULARITY: 8
SHADOW_OFFSET: 0x7fff8000
==17188==Installed the sigaction for signal 11
==17188==Installed the sigaction for signal 7
==17188==Installed the sigaction for signal 8
==17188==T0: stack [0x7ffdb5572000,0x7ffdb5d72000) size 0x800000; local=0x7ffdb5d6f950
==17188==LeakSanitizer: Dynamic linker not found. TLS will not be handled correctly.
==17188==AddressSanitizer Init done
==17188==unpoisoning: 0x7ffdb5d6f150 5bc
==17188==unpoisoning: 0x7ffdb5d6f790 3c
==17188==unpoisoning: 0x7ffdb5d6e8c0 678
==17188==unpoisoning: 0x7ffdb5d6efc0 4
==17188==unpoisoning: 0x7ffdb5d6e240 8
==17188==unpoisoning: 0x7ffdb5d6e260 8
==17188==poisoning: 0x7ffdb5d6e260 8
==17188==poisoning: 0x7ffdb5d6e240 8
==17188==unpoisoning: 0x7ffdb5d6e220 80
==17188==unpoisoning: 0x7ffdb5d6e240 8
==17188==unpoisoning: 0x7ffdb5d6e260 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==unpoisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e280 8
==17188==poisoning: 0x7ffdb5d6e260 8
==17188==poisoning: 0x7ffdb5d6e240 8
==17188==unpoisoning: 0x7ffdb5d6e220 80
==17188==unpoisoning: 0x7ffdb5d6e7a0 8
==17188==poisoning: 0x7ffdb5d6e7a0 8
==17188==unpoisoning: 0x7ffdb5d6e780 40
==17188==poisoning: 0x7ffdb5d6efc0 4
==17188==poisoning: 0x7ffdb5d6e8c0 678
==17188==unpoisoning: 0x7ffdb5d6e880 760
==17188==unpoisoning: 0x7ffdb5d6f120 8
==17188==unpoisoning: 0x7ffdb5d6f140 4
==17188==poisoning: 0x7ffdb5d6f140 4
==17188==poisoning: 0x7ffdb5d6f120 8
==17188==unpoisoning: 0x7ffdb5d6ef00 80
==17188==unpoisoning: 0x7ffdb5d6efa0 8
==17188==unpoisoning: 0x7ffdb5d6efc0 30
==17188==unpoisoning: 0x7ffdb5d6e8a0 18
==17188==poisoning: 0x7ffdb5d6e8a0 18
==17188==unpoisoning: 0x7ffdb5d6e880 4e0
==17188==poisoning: 0x7ffdb5d6efc0 30
==17188==poisoning: 0x7ffdb5d6efa0 8
==17188==poisoning: 0x7ffdb5d6ef00 80
==17188==unpoisoning: 0x7ffdb5d6eee0 140
disco.static : Host "none" not found
==17189==Processing thread 17188.
==17189==Stack at 0x7ffdb5572000-0x7ffdb5d72000 (SP = 0x7ffdb5d6ee18).
==17189==TLS at 0x7f532df7f000-0x7f532df800c0.
Tracer caught signal 11: addr=0x62d00000b000 pc=0x515c60 sp=0x7f532a2ffce0
==17188==LeakSanitizer has encountered a fatal error.
==17188==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1

The program is mostly compiled statically but still links against a few pieces:

# ldd /scratch/disco.static 
    linux-vdso.so.1 (0x00007ffc725dd000)
    libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007f0e0bbb7000)
    libdl.so.2 => /lib/libdl.so.2 (0x00007f0e0b9b3000)
    libpthread.so.0 => /lib/libpthread.so.0 (0x00007f0e0b796000)
    librt.so.1 => /lib/librt.so.1 (0x00007f0e0b58e000)
    libm.so.6 => /lib/libm.so.6 (0x00007f0e0b28b000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f0e0b075000)
    libc.so.6 => /lib/libc.so.6 (0x00007f0e0acd7000)
    /lib/ld-linux-x86-64.so.2 (0x00007f0e0bdee000)

However if the program takes an early exit no sanitizer warning is thrown:

$ LSAN_OPTIONS=verbosity=1:log_threads=1 /scratch/disco.static
==17409==AddressSanitizer: failed to intercept '__isoc99_printf'
==17409==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==17409==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==17409==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==17409==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==17409==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==17409==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==17409==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==17409==AddressSanitizer: failed to intercept 'xdr_quad_t'
==17409==AddressSanitizer: failed to intercept 'xdr_u_quad_t'
==17409==AddressSanitizer: failed to intercept '__cxa_throw'
==17409==AddressSanitizer: libc interceptors initialized
|| `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
|| `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
|| `[0x00008fff7000, 0x02008fff6fff]` || ShadowGap  ||
|| `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
|| `[0x000000000000, 0x00007fff7fff]` || LowMem     ||
MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00 0x02008fff6fff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
malloc_context_size=30
SHADOW_SCALE: 3
SHADOW_GRANULARITY: 8
SHADOW_OFFSET: 0x7fff8000
==17409==Installed the sigaction for signal 11
==17409==Installed the sigaction for signal 7
==17409==Installed the sigaction for signal 8
==17409==T0: stack [0x7ffc4bd56000,0x7ffc4c556000) size 0x800000; local=0x7ffc4c555370
==17409==LeakSanitizer: Dynamic linker not found. TLS will not be handled correctly.
==17409==AddressSanitizer Init done
[...]
disco.static : Cannot open discovery database in maintenance mode
==17410==Processing thread 17409.
==17410==Stack at 0x7ffc4bd56000-0x7ffc4c556000 (SP = 0x7ffc4c554838).
==17410==TLS at 0x7feb31a63000-0x7feb31a640c0.

When running fully the program makes use of shared memory.

[kcc]

Looks like a SEGV during leak detection.
I afraid that my ability to debug this remotely (w/o a reproducer) is rather limited.

[chefmax]

@sitsofe Could you run with LSAN_OPTIONS=verbosity=1:log_threads=1:log_pointers=1? Also, do you use x86_64?

[sitsofe]

@chefmax Yes I'm using x86_64. Here's a run + log_pointers=1

# LSAN_OPTIONS=disable_coredump=0:handle_segv=0:verbosity=1:log_threads=1:log_pointers=1 /scratch/disco.static --host=none
==20670==AddressSanitizer: failed to intercept '__isoc99_printf'
==20670==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==20670==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==20670==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==20670==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==20670==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==20670==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==20670==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==20670==AddressSanitizer: failed to intercept 'xdr_quad_t'
==20670==AddressSanitizer: failed to intercept 'xdr_u_quad_t'
==20670==AddressSanitizer: failed to intercept '__cxa_throw'
==20670==AddressSanitizer: libc interceptors initialized
|| `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
|| `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
|| `[0x00008fff7000, 0x02008fff6fff]` || ShadowGap  ||
|| `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
|| `[0x000000000000, 0x00007fff7fff]` || LowMem     ||
MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00 0x02008fff6fff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
malloc_context_size=30
SHADOW_SCALE: 3
SHADOW_GRANULARITY: 8
SHADOW_OFFSET: 0x7fff8000
==20670==Installed the sigaction for signal 8
==20670==T0: stack [0x7ffdf3f20000,0x7ffdf4720000) size 0x800000; local=0x7ffdf471f760
==20670==LeakSanitizer: Dynamic linker not found. TLS will not be handled correctly.
==20670==AddressSanitizer Init done
==20670==unpoisoning: 0x7ffdf471ef70 5bc
==20670==unpoisoning: 0x7ffdf471f5b0 3c
==20670==unpoisoning: 0x7ffdf471e6e0 678
==20670==unpoisoning: 0x7ffdf471ede0 4
==20670==unpoisoning: 0x7ffdf471e060 8
==20670==unpoisoning: 0x7ffdf471e080 8
==20670==poisoning: 0x7ffdf471e080 8
==20670==poisoning: 0x7ffdf471e060 8
==20670==unpoisoning: 0x7ffdf471e040 80
==20670==unpoisoning: 0x7ffdf471e060 8
==20670==unpoisoning: 0x7ffdf471e080 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==unpoisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e0a0 8
==20670==poisoning: 0x7ffdf471e080 8
==20670==poisoning: 0x7ffdf471e060 8
==20670==unpoisoning: 0x7ffdf471e040 80
==20670==unpoisoning: 0x7ffdf471e5c0 8
==20670==poisoning: 0x7ffdf471e5c0 8
==20670==unpoisoning: 0x7ffdf471e5a0 40
==20670==poisoning: 0x7ffdf471ede0 4
==20670==poisoning: 0x7ffdf471e6e0 678
==20670==unpoisoning: 0x7ffdf471e6a0 760
==20670==unpoisoning: 0x7ffdf471ef40 8
==20670==unpoisoning: 0x7ffdf471ef60 4
==20670==poisoning: 0x7ffdf471ef60 4
==20670==poisoning: 0x7ffdf471ef40 8
==20670==unpoisoning: 0x7ffdf471ed20 80
==20670==unpoisoning: 0x7ffdf471edc0 8
==20670==unpoisoning: 0x7ffdf471ede0 30
==20670==unpoisoning: 0x7ffdf471e6c0 18
==20670==poisoning: 0x7ffdf471e6c0 18
==20670==unpoisoning: 0x7ffdf471e6a0 4e0
==20670==poisoning: 0x7ffdf471ede0 30
==20670==poisoning: 0x7ffdf471edc0 8
==20670==poisoning: 0x7ffdf471ed20 80
==20670==unpoisoning: 0x7ffdf471ed00 140
disco.static : Host "none" not found
==20671==Ignored: chunk 0x60600000efc0-0x60600000eff7 of size 55.
==20671==Scanning GLOBAL range 0x000000901650-0x0000009332e0.
==20671==Scanning GLOBAL range 0x000000b33568-0x0000015c2a90.
==20671==0x0000015c2200: found 0x61300000de80 pointing into chunk 0x61300000de80-0x61300000dfc8 of size 328.
==20671==0x0000015c2718: found 0x60200000ef90 pointing into chunk 0x60200000ef90-0x60200000ef9a of size 10.
==20671==0x0000015c2728: found 0x60200000efd0 pointing into chunk 0x60200000efd0-0x60200000efd1 of size 1.
==20671==0x0000015c2738: found 0x60200000ef50 pointing into chunk 0x60200000ef50-0x60200000ef5d of size 13.
==20671==0x0000015c2748: found 0x60200000edb0 pointing into chunk 0x60200000edb0-0x60200000edbd of size 13.
==20671==0x0000015c2758: found 0x60300000ed10 pointing into chunk 0x60300000ed10-0x60300000ed21 of size 17.
==20671==0x0000015c2768: found 0x60300000ec80 pointing into chunk 0x60300000ec80-0x60300000ec91 of size 17.
==20671==0x0000015c2778: found 0x60300000ebf0 pointing into chunk 0x60300000ebf0-0x60300000ec01 of size 17.
==20671==0x0000015c2788: found 0x60300000eb60 pointing into chunk 0x60300000eb60-0x60300000eb75 of size 21.
==20671==0x0000015c2798: found 0x60300000ead0 pointing into chunk 0x60300000ead0-0x60300000eae5 of size 21.
==20671==0x0000015c27a8: found 0x60300000ea40 pointing into chunk 0x60300000ea40-0x60300000ea54 of size 20.
==20671==0x0000015c27b8: found 0x60300000e9b0 pointing into chunk 0x60300000e9b0-0x60300000e9c4 of size 20.
==20671==0x0000015c27c8: found 0x60300000e920 pointing into chunk 0x60300000e920-0x60300000e934 of size 20.
==20671==0x0000015c27d8: found 0x60300000e8f0 pointing into chunk 0x60300000e8f0-0x60300000e908 of size 24.
==20671==0x0000015c27e8: found 0x60300000e860 pointing into chunk 0x60300000e860-0x60300000e87f of size 31.
==20671==0x0000015c27f8: found 0x60200000ebf0 pointing into chunk 0x60200000ebf0-0x60200000ebfb of size 11.
==20671==0x0000015c2808: found 0x60200000ebb0 pointing into chunk 0x60200000ebb0-0x60200000ebbc of size 12.
==20671==0x0000015c2818: found 0x60200000eb70 pointing into chunk 0x60200000eb70-0x60200000eb79 of size 9.
==20671==0x0000015c2828: found 0x60200000eb30 pointing into chunk 0x60200000eb30-0x60200000eb3a of size 10.
==20671==0x0000015c2838: found 0x60300000e830 pointing into chunk 0x60300000e830-0x60300000e841 of size 17.
==20671==0x0000015c2848: found 0x60200000ea90 pointing into chunk 0x60200000ea90-0x60200000ea99 of size 9.
==20671==0x0000015c2858: found 0x60200000ea50 pointing into chunk 0x60200000ea50-0x60200000ea55 of size 5.
==20671==0x0000015c2868: found 0x60300000eda0 pointing into chunk 0x60300000eda0-0x60300000edb3 of size 19.
==20671==0x0000015c2878: found 0x60200000ee50 pointing into chunk 0x60200000ee50-0x60200000ee5b of size 11.
==20671==0x0000015c2888: found 0x60200000ea10 pointing into chunk 0x60200000ea10-0x60200000ea19 of size 9.
==20671==Scanning GLOBAL range 0x7ff0d7d72dc8-0x7ff0d7da11e0.
==20671==Scanning GLOBAL range 0x7ff0d7b69d50-0x7ff0d7b6a0f0.
==20671==Scanning GLOBAL range 0x7ff0d7961b88-0x7ff0d7966428.
==20671==Scanning GLOBAL range 0x7ff0d7748d88-0x7ff0d7749bc0.
==20671==Scanning GLOBAL range 0x7ff0d7540da0-0x7ff0d7541130.
==20671==Scanning GLOBAL range 0x7ff0d723e000-0x7ff0d723e620.
==20671==Scanning GLOBAL range 0x7ff0d701f7c8-0x7ff0d70289e0.
==20671==Scanning GLOBAL range 0x7ff0d7fc4bc0-0x7ff0d7fc6168.
==20671==Processing thread 20670.
==20671==Scanning REGISTERS range 0x7ff0d7e3d000-0x7ff0d7e3d0d8.
==20671==Stack at 0x7ffdf3f20000-0x7ffdf4720000 (SP = 0x7ffdf471ec38).
==20671==Scanning STACK range 0x7ffdf471ec38-0x7ffdf4720000.
==20671==TLS at 0x7ff0d7fbe000-0x7ff0d7fbf0c0.
==20671==Scanning TLS range 0x7ff0d7fbe000-0x7ff0d7fbf0c0.
==20671==Scanning HEAP range 0x60200000ea10-0x60200000ea19.
==20671==Scanning HEAP range 0x60200000ee50-0x60200000ee5b.
==20671==Scanning HEAP range 0x60300000eda0-0x60300000edb3.
==20671==Scanning HEAP range 0x60200000ea50-0x60200000ea55.
==20671==Scanning HEAP range 0x60200000ea90-0x60200000ea99.
==20671==Scanning HEAP range 0x60300000e830-0x60300000e841.
==20671==Scanning HEAP range 0x60200000eb30-0x60200000eb3a.
==20671==Scanning HEAP range 0x60200000eb70-0x60200000eb79.
==20671==Scanning HEAP range 0x60200000ebb0-0x60200000ebbc.
==20671==Scanning HEAP range 0x60200000ebf0-0x60200000ebfb.
==20671==Scanning HEAP range 0x60300000e860-0x60300000e87f.
==20671==Scanning HEAP range 0x60300000e8f0-0x60300000e908.
==20671==Scanning HEAP range 0x60300000e920-0x60300000e934.
==20671==Scanning HEAP range 0x60300000e9b0-0x60300000e9c4.
==20671==Scanning HEAP range 0x60300000ea40-0x60300000ea54.
==20671==Scanning HEAP range 0x60300000ead0-0x60300000eae5.
==20671==Scanning HEAP range 0x60300000eb60-0x60300000eb75.
==20671==Scanning HEAP range 0x60300000ebf0-0x60300000ec01.
==20671==Scanning HEAP range 0x60300000ec80-0x60300000ec91.
==20671==Scanning HEAP range 0x60300000ed10-0x60300000ed21.
==20671==Scanning HEAP range 0x60200000edb0-0x60200000edbd.
==20671==Scanning HEAP range 0x60200000ef50-0x60200000ef5d.
==20671==Scanning HEAP range 0x60200000efd0-0x60200000efd1.
==20671==Scanning HEAP range 0x60200000ef90-0x60200000ef9a.
==20671==Scanning HEAP range 0x61300000de80-0x61300000dfc8.
==20671==0x61300000dea0: found 0x62d00000b000 pointing into chunk 0x62d00000b000-0x62d000013000 of size 32768.
==20671==Scanning HEAP range 0x62d00000b000-0x62d000013000.
Tracer caught signal 11: addr=0x62d00000b000 pc=0x515c60 sp=0x7ff0d43ffce0
==20670==LeakSanitizer has encountered a fatal error.
==20670==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1

If there's any more information I can provide, please let me know.

[kcc]

does this happen with clang top-of-tree?

[chefmax]

FWIW, I've seen a strange segfault some time ago on Aarch64. I didn't spend much time on investigation, but the issue seemed to appear in PointsIntoChunk function (I used LSan w/o ASan):

uptr PointsIntoChunk(void* p) {
  uptr addr = reinterpret_cast<uptr>(p);
  uptr chunk = reinterpret_cast<uptr>(allocator.GetBlockBeginFastLocked(p));
  if (!chunk) return 0;
  // LargeMmapAllocator considers pointers to the meta-region of a chunk to be
  // valid, but we don't want that.
  if (addr < chunk) return 0;
  ChunkMetadata *m = Metadata(reinterpret_cast<void *>(chunk));
  CHECK(m);
  if (!m->allocated)
    return 0;
  if (addr < chunk + m->requested_size)
    return chunk;
  if (IsSpecialCaseOfOperatorNew0(chunk, m->requested_size, addr))
    return chunk;
  return 0;
}

In my case, the chunk value actually pointed into metadata chunk, not to user chunk, thus all subsequent logic became invalid. But please note, that x86_64 and AArch64 use different allocation strategies, thus I'm not sure this is the case. I can attach a draft patch here I'm preparing to post upstream if desired.

[sitsofe]

Reproducing the issue turned out to be simple in the end - the program had protected its memory:
prot.c:

#include <stdlib.h>
#include <sys/mman.h>

int main(void) {
    int *protected;
    posix_memalign((void **) &protected, 4096, 32 * 1024);
    mprotect(protected, 1024, PROT_NONE);
    //*protected = 1;

    return 0;
}

$ clang -g -O0 -fsanitize=address prot.c
$ ASAN_OPTIONS="detect_leaks=1" ./a.out 
Tracer caught signal 11: addr=0x62d00000b000 pc=0x502140 sp=0x7feab15ffc60
==16021==LeakSanitizer has encountered a fatal error.
==16021==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1

I guess the leak sanitizer should explicitly mark memory as accessible before trying to access it after the program finishes...

[yugr]

I guess the leak sanitizer should explicitly mark memory as accessible before trying to access it
after the program finishes...

Or just not access protected pages to avoid false negatives.

[sitsofe]

@yugr So if a memory region is protected against reading in any form you simply assume it doesn't contain pointers to any other memory (since you couldn't read it to check anyway...)?

If so perhaps it's the only reasonable thing that can be done (plus a documentation note) - if the program is running and someone asks for a leak check it's probably risky to start fiddling with memory mappings. Further, not fiddling the mapping will just make the program crash. Given this it sounds like a potential false positive is the lesser of the evils...

[yugr]

@sitsofe Yes, that's what I had in mind (just a suggestion though - I'm not a maintainer).

[kcc]

Apparently, this is some kind of user-implemented guard page (aka electric fence),
but I am not even sure it's done correctly, e.g. mprotect does not have granularity of 1024 bytes.

I am not convinced that implementing support for this use case is worth the extra complexity in lsan.
Instead I would suggest to use mmap instead of posix_memalign, or free the memory before exit, or unprotect it before exit.

[sitsofe]

The granularity mistake was introduced by me and wasn't in the original code. I believe the purpose for it stems from a desire to protect some memory that should only be read/written at specific times and later I see it being used with mmap.

If you want to close this feel free (it's worth noting that tcmalloc's leak detection hits the same issue so I guess this type of situation is rare).

[kcc]

Let me close this. If more users complain we'll reconsider.
Thanks for the report!

[stsp]

May I offer a simple fix for this?
In function CheckForLeaks(), lsan_common.cc:453
just replace Die() with return false; and all be well.
I am sure in my code I apply mprotect() only to the
mmap'ed areas, never to the malloc'ed ones, and yet
I am seeing the same crash.
If you don't fix it, please just make it non-fatal.

[Firstyear]

Is it possible that this issue is related to the memalign, not just the mprotect? We have seen some failures in LSAN related to memaligned memory, and the "patch" here apparently resolves that, but I'm not fully across the issue.

[vashirov]

I agree with @stsp, this should not be fatal. At least we can get some information about the leak.
@kcc, could you please reconsider and reopen this issue?
Thank you.

[benvanik]

We're also hitting this in what is likely a very similar condition. It'd be nice to get a warning instead of a failure (even if a flag), as right now we have to disable ASAN :(

[eugenis]

You could also do ASAN_OPTIONS=detect_leaks=0, i.e. disable leak detection but not the rest of ASan.

…

On Thu, Oct 17, 2019 at 5:11 PM Ben Vanik ***@***.***> wrote: We're also hitting this in what is likely a very similar condition. It'd be nice to get a warning instead of a failure (even if a flag), as right now we have to disable ASAN :( — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#723?email_source=notifications&email_token=AADG4SSY4N4PVPB3XNSMOVLQPD5KLA5CNFSM4CPOZL72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBR6ZVQ#issuecomment-543419606>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADG4STCMXKCCIEJE2PZS5LQPD5KLANCNFSM4CPOZL7Q> .

[ThadHouse]

We started seeing this recently, and might have found a culprit. I'm attempting to build a smaller reproduction. We have a library that is dynamically loading an lttng provider, and it seems that unconditionally if we enable this, ASAN results in this crash. Thats a very common use case, so a bit scary that those 2 interfere with each other.

[ThadHouse]

AsanLttng.zip

Yup. I can reproduce it with a small test project. Tested on ubuntu 18.04.

[sitsofe]

@ThadHouse While your part of the example is small it is not self contained because you need the external lttng project to be installed before you can try it. You might wish to contact the LTTng folks and get their take on things first...

[ThadHouse]

ASAN shouldn't be crashing, no matter what the libraries it's being applied to do. In addition, lttng is extremely popular as a logging library, and it's dependency is only required to build. I'll add tomorrow morning exactly what package needs to be installed to get the build to work.

[hossbeast]

I was able to work around this issue by specifying the following runtime options

ASAN_OPTIONS=fast_unwind_on_malloc=0

With this option in effect, LeakSanitizer no longer crashes while reporting leaks in my particular case.

[vibhabiblani]

mprotect() issue doesn’t get resolve with the asan option "ASAN_OPTIONS=fast_unwind_on_malloc=0" for my case.
Getting "LeakSanitizer has encountered a fatal error" when my code uses mprotect() call,
VOIDC mprotect ( (caddr_t) stk->prot1.addr, (size_t) stk->prot1.len, (int) PROT_NONE);
Commenting this call reports leak stacktraces properly.

Any suggestions on this?

[pawanpossible]

Please go through below issue. It might be useful and if you have similar problem workaround has been given on it.
#1322