Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OSS Fuzz] Building of luaL_loadbuffer_proto_test is broken on aarch64 #72

Closed
ligurio opened this issue Dec 28, 2023 · 2 comments · Fixed by #89
Closed

[OSS Fuzz] Building of luaL_loadbuffer_proto_test is broken on aarch64 #72

ligurio opened this issue Dec 28, 2023 · 2 comments · Fixed by #89

Comments

@ligurio
Copy link
Owner

ligurio commented Dec 28, 2023 

Step #43 - "compile-libfuzzer-address-aarch64": [0/1] Install the project...
Step #43 - "compile-libfuzzer-address-aarch64": -- Install configuration: "Debug"
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/lib/cmake/libprotobuf-mutator/libprotobuf-mutatorTargets.cmake
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/lib/cmake/libprotobuf-mutator/libprotobuf-mutatorTargets-debug.cmake
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/lib/cmake/libprotobuf-mutator/libprotobuf-mutatorConfig.cmake
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/OFF/libprotobuf-mutator.pc
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/port
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/port/gtest.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/port/protobuf.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/mutator.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/random.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/text_format.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/utf8_fix.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/weighted_reservoir_sampler.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/binary_format.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/field_instance.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/libfuzzer
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/libfuzzer/libfuzzer_mutator.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/include/libprotobuf-mutator/src/libfuzzer/libfuzzer_macro.h
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/lib/libprotobuf-mutator.a
Step #43 - "compile-libfuzzer-address-aarch64": -- Installing: /src/testdir/build/tests/external.protobuf_mutator/lib/libprotobuf-mutator-libfuzzer.a
Step #43 - "compile-libfuzzer-address-aarch64": 
[34/41] Completed 'external.protobuf_mutator'�[K
[35/41] Completed 'external.protobuf_mutator'�[K
[35/41] Running cpp protocol buffer compiler on lua_grammar.proto�[K
[36/41] Running cpp protocol buffer compiler on lua_grammar.proto�[K

Step #43 - "compile-libfuzzer-address-aarch64": �[31mFAILED: �[0mtests/luaL_loadbuffer_proto/lua_grammar.pb.h tests/luaL_loadbuffer_proto/lua_grammar.pb.cc /src/testdir/build/tests/luaL_loadbuffer_proto/lua_grammar.pb.h /src/testdir/build/tests/luaL_loadbuffer_proto/lua_grammar.pb.cc 
Step #43 - "compile-libfuzzer-address-aarch64": cd /src/testdir/build/tests/luaL_loadbuffer_proto && /src/testdir/build/tests/external.protobuf_mutator/src/external.protobuf_mutator-build/external.protobuf/bin/protoc --cpp_out /src/testdir/build/tests/luaL_loadbuffer_proto -I /src/testdir/tests/luaL_loadbuffer_proto /src/testdir/tests/luaL_loadbuffer_proto/lua_grammar.proto
Step #43 - "compile-libfuzzer-address-aarch64": ==4449==LeakSanitizer has encountered a fatal error.
Step #43 - "compile-libfuzzer-address-aarch64": ==4449==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
Step #43 - "compile-libfuzzer-address-aarch64": ==4449==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
Step #43 - "compile-libfuzzer-address-aarch64": ninja: build stopped: subcommand failed.
Step #43 - "compile-libfuzzer-address-aarch64": ********************************************************************************
Step #43 - "compile-libfuzzer-address-aarch64": Failed to build.
Step #43 - "compile-libfuzzer-address-aarch64": To reproduce, run:
Step #43 - "compile-libfuzzer-address-aarch64": python infra/helper.py build_image lua
Step #43 - "compile-libfuzzer-address-aarch64": python infra/helper.py build_fuzzers --sanitizer address --engine libfuzzer --architecture aarch64 lua
Step #43 - "compile-libfuzzer-address-aarch64": ********************************************************************************

Finished Step #43 - "compile-libfuzzer-address-aarch64"
ERROR
ERROR: build step 43 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1

Log: https://oss-fuzz-build-logs.storage.googleapis.com/log-36437658-64d6-4c63-8c82-ed9b37c47c22.txt
ligurio added a commit that referenced this issue Dec 29, 2023
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
f2c2915 
Part of #72
@ligurio ligurio mentioned this issue Dec 29, 2023
Merged
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
0c66265 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
07fc88e 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
9c2ae15 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
5f785fc 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
36cc7a0 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
29fae0f 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
8af2871 
Part of #72
ligurio added a commit that referenced this issue Jan 11, 2024
@ligurio
cmake: disable luaL_loadbuffer_proto on OSS Fuzz/aarch64
9294a95 
Part of #72
@ligurio ligurio changed the title Build is broken on aarch64 Building of luaL_loadbuffer_proto_test is broken on aarch64 Jan 15, 2024
ligurio added a commit that referenced this issue Jan 25, 2024
@ligurio
cmake: fix building luaL_loadbuffer_proto
c05e479 
Follows up commit 9294a95 ("cmake: disable luaL_loadbuffer_proto on
OSS Fuzz/aarch64").

Follows up #72
ligurio added a commit that referenced this issue Jan 25, 2024
@ligurio
cmake: fix building luaL_loadbuffer_proto on non-aarch64
b6efd0c 
Commit 9294a95 ("cmake: disable luaL_loadbuffer_proto on OSS
Fuzz/aarch64") accidentally disabled test `luaL_loadbuffer_proto` on
non-aarch64 systems. The patch simplifies CMake condition and fixes
building of the test on non-aarch64 systems.

Follows up #72
ligurio added a commit that referenced this issue Jan 25, 2024
@ligurio
cmake: fix building luaL_loadbuffer_proto on non-aarch64
85a30d5 
Commit 9294a95 ("cmake: disable luaL_loadbuffer_proto on OSS
Fuzz/aarch64") accidentally disabled test `luaL_loadbuffer_proto` on
non-aarch64 systems. The patch simplifies CMake condition and fixes
building of the test on non-aarch64 systems.

Follows up #72
@ligurio ligurio changed the title Building of luaL_loadbuffer_proto_test is broken on aarch64 [OSS Fuzz] Building of luaL_loadbuffer_proto_test is broken on aarch64 Jan 25, 2024
ligurio added a commit that referenced this issue Apr 4, 2024
@ligurio
tests: enable luaL_loadbuffer_proto on AArch64
65f9e4a 
Fixes #72
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Apr 4, 2024
@ligurio
lua: enable verbose mode
3820749 
Need for debugging AArch64 build.

Needed for ligurio/lua-c-api-tests#72
@ligurio ligurio mentioned this issue Apr 4, 2024
Merged
DavidKorczynski pushed a commit to google/oss-fuzz that referenced this issue Apr 4, 2024
@ligurio
lua: enable verbose mode (#11781)
5717414 
Need for debugging AArch64 build.

Needed for ligurio/lua-c-api-tests#72
ligurio added a commit that referenced this issue Apr 5, 2024
@ligurio
tests: enable luaL_loadbuffer_proto on AArch64
3cc3669 
Fixes #72
@ligurio ligurio mentioned this issue Apr 5, 2024
Merged
ligurio added a commit that referenced this issue Apr 5, 2024
@ligurio
tests: enable luaL_loadbuffer_proto on AArch64
a1692c5 
Fixes #72
ligurio added a commit that referenced this issue Apr 5, 2024
@ligurio
tests: remove a comment
36e21aa 
Follows up #72
@ligurio
Copy link
Owner Author

ligurio commented Apr 8, 2024

The problem is not gone and seems it is related to LSAN:

Step #43 - "compile-libfuzzer-address-aarch64": �[31mFAILED: �[0mtests/capi/luaL_loadbuffer_proto/preamble.lua.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c 
Step #43 - "compile-libfuzzer-address-aarch64": cd /src/testdir/build/tests/capi/luaL_loadbuffer_proto && /usr/bin/echo 'const char preamble_lua[] =' > /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /src/testdir/build/extra/txt2c /src/testdir/tests/capi/luaL_loadbuffer_proto/preamble.lua >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/bin/echo ';' >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/local/bin/cmake -E copy_if_different /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c && /usr/local/bin/cmake -E remove /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c
Step #43 - "compile-libfuzzer-address-aarch64": ==2425==LeakSanitizer has encountered a fatal error.
Step #43 - "compile-libfuzzer-address-aarch64": ==2425==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
Step #43 - "compile-libfuzzer-address-aarch64": ==2425==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
  1. https://oss-fuzz-build-logs.storage.googleapis.com/log-fdc76ec1-4c88-49b2-9c71-b1172de56d6d.txt

@ligurio ligurio reopened this Apr 8, 2024
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Apr 8, 2024
@ligurio
lua: enable Leakanitizer debug mode
e07c8d3 
Building `luaL_loadbuffer_proto` on AArch64 has failed with unclear
error:

```
[31mFAILED: �[0mtests/capi/luaL_loadbuffer_proto/preamble.lua.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c
cd /src/testdir/build/tests/capi/luaL_loadbuffer_proto && /usr/bin/echo 'const char preamble_lua[] =' > /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /src/testdir/build/extra/txt2c /src/testdir/tests/capi/luaL_loadbuffer_proto/preamble.lua >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/bin/echo ';' >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/local/bin/cmake -E copy_if_different /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c && /usr/local/bin/cmake -E remove /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c
==2425==LeakSanitizer has encountered a fatal error.
==2425==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==2425==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
```

The patch set an environment variable enabling debug mode in
LeakSanitizer.

Needed for ligurio/lua-c-api-tests#72
@ligurio ligurio mentioned this issue Apr 8, 2024
Merged
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Apr 9, 2024
@ligurio
lua: enable LeakSanitizer debug mode
1ae3f9e 
Building `luaL_loadbuffer_proto` on AArch64 has failed with unclear
error:

```
[31mFAILED: �[0mtests/capi/luaL_loadbuffer_proto/preamble.lua.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c
cd /src/testdir/build/tests/capi/luaL_loadbuffer_proto && /usr/bin/echo 'const char preamble_lua[] =' > /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /src/testdir/build/extra/txt2c /src/testdir/tests/capi/luaL_loadbuffer_proto/preamble.lua >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/bin/echo ';' >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/local/bin/cmake -E copy_if_different /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c && /usr/local/bin/cmake -E remove /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c
==2425==LeakSanitizer has encountered a fatal error.
==2425==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==2425==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
```

The patch set an environment variable enabling debug mode in
LeakSanitizer.

Needed for ligurio/lua-c-api-tests#72
DavidKorczynski pushed a commit to google/oss-fuzz that referenced this issue Apr 9, 2024
@ligurio
lua: enable Leakanitizer debug mode (#11787)
50c8bb5 
Building `luaL_loadbuffer_proto` on AArch64 has failed with unclear
error:

```
[31mFAILED: �[0mtests/capi/luaL_loadbuffer_proto/preamble.lua.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c
cd /src/testdir/build/tests/capi/luaL_loadbuffer_proto && /usr/bin/echo 'const char preamble_lua[] =' > /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /src/testdir/build/extra/txt2c /src/testdir/tests/capi/luaL_loadbuffer_proto/preamble.lua >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/bin/echo ';' >> /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c && /usr/local/bin/cmake -E copy_if_different /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.c && /usr/local/bin/cmake -E remove /src/testdir/build/tests/capi/luaL_loadbuffer_proto/preamble.lua.new.c
==2425==LeakSanitizer has encountered a fatal error.
==2425==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==2425==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
```

The patch set an environment variable enabling debug mode in
LeakSanitizer.

Needed for ligurio/lua-c-api-tests#72
@ligurio ligurio mentioned this issue Apr 11, 2024
Open
ligurio added a commit to ligurio/oss-fuzz that referenced this issue Apr 12, 2024
@ligurio
lua: work around LeakSanitizer internal errors
7fcde05 
When using `-fanitize=address`, we consistently get spurious build
failures on ARM64 due to some issue with the leak detection pass
at the end of execution:

```
==2413==AddressSanitizer: failed to intercept '__isoc99_printf'
==2413==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==2413==AddressSanitizer: failed to intercept 'xdr_destroy'
==2413==AddressSanitizer: failed to intercept 'crypt'
==2413==AddressSanitizer: failed to intercept 'crypt_r'
==2413==AddressSanitizer: failed to intercept '__cxa_throw'
==2413==AddressSanitizer: failed to intercept '__cxa_rethrow_primary_exception'
==2413==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
thread_local_quarantine_size_kb=1024K
```

Note, it does not happen for me locally and on AWS Graviton 2
using the same build-with command line.

Related to google#11798
Related to google/sanitizers#723
Related to ligurio/lua-c-api-tests#72
@ligurio ligurio mentioned this issue Apr 12, 2024
Merged
DavidKorczynski pushed a commit to google/oss-fuzz that referenced this issue Apr 15, 2024
@ligurio
lua: work around LeakSanitizer internal errors (#11804)
76e6f7d 
When using `-fanitize=address`, we consistently get spurious build
failures on ARM64 due to some issue with the leak detection pass at the
end of execution:

```
==2413==AddressSanitizer: failed to intercept '__isoc99_printf'
==2413==AddressSanitizer: failed to intercept '__isoc99_sprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_snprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_fprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
==2413==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
==2413==AddressSanitizer: failed to intercept 'xdr_destroy'
==2413==AddressSanitizer: failed to intercept 'crypt'
==2413==AddressSanitizer: failed to intercept 'crypt_r'
==2413==AddressSanitizer: failed to intercept '__cxa_throw'
==2413==AddressSanitizer: failed to intercept '__cxa_rethrow_primary_exception'
==2413==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size_mb=256M
thread_local_quarantine_size_kb=1024K
```

Note, it does not happen for me locally and on AWS Graviton 2 using the
same build-with command line.

Related to #11798
Related to google/sanitizers#723
Related to ligurio/lua-c-api-tests#72
@ligurio
Copy link
Owner Author

ligurio commented Apr 16, 2024

Fixed by google/oss-fuzz@76e6f7d.

@ligurio ligurio closed this as completed Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tests: enable luaL_loadbuffer_proto on AArch64 ligurio/lua-c-api-tests
1 participant
@ligurio