Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[lua] LeakSanitizer breaks building on ARM64 #11798

Open
ligurio opened this issue Apr 11, 2024 · 1 comment
Open

[lua] LeakSanitizer breaks building on ARM64 #11798

ligurio opened this issue Apr 11, 2024 · 1 comment

Comments

@ligurio
Copy link
Contributor

ligurio commented Apr 11, 2024

Problem

LeakSanitizer breaks building of luaL_loadbuffer_proto for a Lua project on ARM64 (everything is fine on x86_64 and i386).
Part of build log 1:

<snipped>

Step #43 - "compile-libfuzzer-address-aarch64": �[31mFAILED: �[0mtests/capi/luaL_loadbuffer_proto/preamble.lua.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c 
Step #43 - "compile-libfuzzer-address-aarch64": cd /src/testdir/build/tests/capi/luaL_loadbuffer_proto && /usr/bin/echo 'const char preamble_lua[] =' > /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /src/testdir/build/extra/txt2c /src/testdir/tests/capi/luaL_loadbuffer_proto/preamble.lua >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/bin/echo ';' >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/local/bin/cmake -E copy_if_different /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c && /usr/local/bin/cmake -E remove /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_printf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_sprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_snprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_fprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_vprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept 'xdr_destroy'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept 'crypt'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept 'crypt_r'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__cxa_throw'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: failed to intercept '__cxa_rethrow_primary_exception'
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer: libc interceptors initialized
Step #43 - "compile-libfuzzer-address-aarch64": || `[0x002000000000, 0x007fffffffff]` || HighMem    ||
Step #43 - "compile-libfuzzer-address-aarch64": || `[0x001400000000, 0x001fffffffff]` || HighShadow ||
Step #43 - "compile-libfuzzer-address-aarch64": || `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
Step #43 - "compile-libfuzzer-address-aarch64": || `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
Step #43 - "compile-libfuzzer-address-aarch64": || `[0x000000000000, 0x000fffffffff]` || LowMem     ||
Step #43 - "compile-libfuzzer-address-aarch64": MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
Step #43 - "compile-libfuzzer-address-aarch64": redzone=16
Step #43 - "compile-libfuzzer-address-aarch64": max_redzone=2048
Step #43 - "compile-libfuzzer-address-aarch64": quarantine_size_mb=256M
Step #43 - "compile-libfuzzer-address-aarch64": thread_local_quarantine_size_kb=1024K
Step #43 - "compile-libfuzzer-address-aarch64": malloc_context_size=30
Step #43 - "compile-libfuzzer-address-aarch64": SHADOW_SCALE: 3
Step #43 - "compile-libfuzzer-address-aarch64": SHADOW_GRANULARITY: 8
Step #43 - "compile-libfuzzer-address-aarch64": SHADOW_OFFSET: 0x1000000000
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==Installed the sigaction for signal 11
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==Installed the sigaction for signal 7
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==Installed the sigaction for signal 8
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==T0: stack [0x004000001000,0x004000801000) size 0x800000; local=0x0040007fffac
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==AddressSanitizer Init done
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==Failed spawning a tracer thread (errno 22).
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==LeakSanitizer has encountered a fatal error.
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
Step #43 - "compile-libfuzzer-address-aarch64": ==2413==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

Solution

There are a plenty of issues with similar symptoms. In "LeakSanitizer has encountered a fatal error" 2 suggested a fix:

We don't run it privileged (lots of capabilities). For LSan it should be enough only to enable ptrace: --cap-add SYS_PTRACE

And according to PR "use SYS_PTRACE capability" 3 this flag had been added. Probably it was done for x86_64 Docker images and not for ARM64.

Related issues

Footnotes

  1. https://oss-fuzz-build-logs.storage.googleapis.com/log-8b81a639-5e8a-416f-8f6f-10249b9313ee.txt ↩

  2. https://github.com/google/sanitizers/issues/764#issuecomment-276700920 ↩

  3. https://github.com/google/oss-fuzz/pull/212 ↩
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Apr 12, 2024
@ligurio
lua: work around LeakSanitizer internal errors
8e9306e 
When using `-fanitize=address`, we consistently get spurious build
failures on ARM64 due to some issue with the leak detection pass
at the end of execution:

```
==2413==AddressSanitizer: failed to intercept '__isoc99_printf'
==2413==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==2413==AddressSanitizer: failed to intercept 'xdr_destroy'
==2413==AddressSanitizer: failed to intercept 'crypt'
==2413==AddressSanitizer: failed to intercept 'crypt_r'
==2413==AddressSanitizer: failed to intercept '__cxa_throw'
==2413==AddressSanitizer: failed to intercept '__cxa_rethrow_primary_exception'
==2413==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
thread_local_quarantine_size_kb=1024K
```

Note, it does not happen for me locally and on AWS Graviton 2
using the same build-with command line.

Related to google#11798
Related to google/sanitizers#723
@ligurio ligurio mentioned this issue Apr 12, 2024
Merged
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Apr 12, 2024
@ligurio
lua: work around LeakSanitizer internal errors
7fcde05 
When using `-fanitize=address`, we consistently get spurious build
failures on ARM64 due to some issue with the leak detection pass
at the end of execution:

```
==2413==AddressSanitizer: failed to intercept '__isoc99_printf'
==2413==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==2413==AddressSanitizer: failed to intercept 'xdr_destroy'
==2413==AddressSanitizer: failed to intercept 'crypt'
==2413==AddressSanitizer: failed to intercept 'crypt_r'
==2413==AddressSanitizer: failed to intercept '__cxa_throw'
==2413==AddressSanitizer: failed to intercept '__cxa_rethrow_primary_exception'
==2413==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
thread_local_quarantine_size_kb=1024K
```

Note, it does not happen for me locally and on AWS Graviton 2
using the same build-with command line.

Related to google#11798
Related to google/sanitizers#723
Related to ligurio/lua-c-api-tests#72
DavidKorczynski pushed a commit that referenced this issue Apr 15, 2024
@ligurio
lua: work around LeakSanitizer internal errors (#11804)
76e6f7d 
When using `-fanitize=address`, we consistently get spurious build
failures on ARM64 due to some issue with the leak detection pass at the
end of execution:

```
==2413==AddressSanitizer: failed to intercept '__isoc99_printf'
==2413==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==2413==AddressSanitizer: failed to intercept 'xdr_destroy'
==2413==AddressSanitizer: failed to intercept 'crypt'
==2413==AddressSanitizer: failed to intercept 'crypt_r'
==2413==AddressSanitizer: failed to intercept '__cxa_throw'
==2413==AddressSanitizer: failed to intercept '__cxa_rethrow_primary_exception'
==2413==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
thread_local_quarantine_size_kb=1024K
```

Note, it does not happen for me locally and on AWS Graviton 2 using the
same build-with command line.

Related to #11798
Related to google/sanitizers#723
Related to ligurio/lua-c-api-tests#72
@jonathanmetzman
Copy link
Contributor

The ARM builds aren't really supported (yet?).

ligurio added a commit to ligurio/oss-fuzz that referenced this issue Aug 9, 2024
@ligurio
tarantool: work around LeakSanitizer internal errors
ff0e944 
With enabled LeakSanitizer on AArch64 ICU build fails due to
false positive warnings from Leak Sanitizer (part of config.log):

configure:3209: checking whether we are cross compiling
configure:3217: clang -o conftest -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link  -lpthread conftest.c  >&5
configure:3221: $? = 0
configure:3228: ./conftest
==666==LeakSanitizer has encountered a fatal error.
==666==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==666==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
configure:3232: $? = 1
configure:3239: error: in `/src/tarantool/build/icu':
configure:3241: error: cannot run C compiled programs.

The patch disable LeakSanitizer on AArch64 and increase verbosity.

Related to google#11798
Related to google/sanitizers#723
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Aug 9, 2024
@ligurio
tarantool: work around LeakSanitizer internal errors
77c07d4 
With enabled LeakSanitizer on AArch64 ICU build fails due to
false positive warnings from Leak Sanitizer (part of config.log):

```
configure:3209: checking whether we are cross compiling
configure:3217: clang -o conftest -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link  -lpthread conftest.c  >&5
configure:3221: $? = 0
configure:3228: ./conftest
==666==LeakSanitizer has encountered a fatal error.
==666==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==666==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
configure:3232: $? = 1
configure:3239: error: in `/src/tarantool/build/icu':
configure:3241: error: cannot run C compiled programs.
```

The patch disable LeakSanitizer on AArch64 and increase verbosity.

Related to google#11798
Related to google/sanitizers#723
@ligurio ligurio mentioned this issue Aug 9, 2024
Merged
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Aug 9, 2024
@ligurio
tarantool: work around LeakSanitizer internal errors
9a21260 
With enabled LeakSanitizer on AArch64 ICU build fails due to
false positive warnings from Leak Sanitizer (part of config.log):

```
configure:3209: checking whether we are cross compiling
configure:3217: clang -o conftest -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link  -lpthread conftest.c  >&5
configure:3221: $? = 0
configure:3228: ./conftest
==666==LeakSanitizer has encountered a fatal error.
==666==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==666==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
configure:3232: $? = 1
configure:3239: error: in `/src/tarantool/build/icu':
configure:3241: error: cannot run C compiled programs.
```

The patch disable LeakSanitizer on AArch64 and increase verbosity.

Related to google#11798
Related to google/sanitizers#723
DavidKorczynski pushed a commit that referenced this issue Aug 14, 2024
@ligurio
tarantool: work around LeakSanitizer internal errors (#12321)
758349e 
With enabled LeakSanitizer on AArch64 ICU build fails due to false
positive warnings from Leak Sanitizer (part of config.log):

```
configure:3209: checking whether we are cross compiling configure:3217: clang -o conftest -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link  -lpthread conftest.c  >&5 configure:3221: $? = 0
configure:3228: ./conftest
==666==LeakSanitizer has encountered a fatal error. ==666==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1 ==666==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc) configure:3232: $? = 1
configure:3239: error: in `/src/tarantool/build/icu': configure:3241: error: cannot run C compiled programs.
```

The patch disable LeakSanitizer on AArch64 and increase verbosity.

Related to #11798
Related to google/sanitizers#723
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ligurio @jonathanmetzman