crypton-0.34: segfault with --ghc-option=-optc-march=znver2

[hololeap]

I ran into this when I was compiling for a Gentoo system. The script for gentoo-haskell passes custom flags for GCC as --ghc-option=-optc-... configure flags. I isolated the problem flag down to --ghc-option=-optc-march=znver2, which is passed in because I have -march=znver2 in my system CFLAGS.

After it compiles successfully, the test suite is run, which is where the segfault occurs. It segfaults immediately after the crypton.Curve448.KATs.1 test. After some investigation, I managed to get a stack trace:

#0  crypton_decaf_448_point_decode_like_eddsa_and_ignore_cofactor (p=p@entry=0x7fffffff9710, 
    enc=enc@entry=0x4200508f20 "_\327D\233Y\264a\375,\347\207\354aj\324j\035\2414$\205\247\016\037\212\016\247]\200\351gx\355\361$v\233F\307\006\033\326x=\361\345\017l\321\372\032\276\257\350%a\200")
    at cbits/decaf/ed448goldilocks/decaf.c:1199
#1  0x00007ffff674d623 in crypton_decaf_ed448_verify (
    signature=0x4200508f70 "S:7\366\273\344W%\037\002<\r\210\371v\256-\373PJ\204>4\322\aO\330#\324\032Y\037+#?\003Ob\202\201\362\375z\"\335\324}x(śТ\033\3759\200\377\r (Ա\212\235\366>", 
    pubkey=0x4200508f20 "_\327D\233Y\264a\375,\347\207\354aj\324j\035\2414$\205\247\016\037\212\016\247]\200\351gx\355\361$v\233F\307\006\033\326x=\361\345\017l\321\372\032\276\257\350%a\200", 
    message=0x42004e45c0 "\220\034\235\365\377\177", message_len=0, prehashed=0 '\000', context=0x0, context_len=0 '\000') at cbits/decaf/ed448goldilocks/eddsa.c:262
#2  0x00007ffff65f8300 in ?? () from /var/tmp/portage/dev-haskell/crypton-0.34/work/crypton-0.34/dist/build/libHScrypton-0.34-FExYDRjm3zqCtjKWPqU6Vg-ghc9.4.8.so
#3  0x0000000000000000 in ?? ()

This seems to point to this line, although why this causes a segfault is above my head:

https://github.com/kazu-yamamoto/crypton/blob/crypton-0.34/cbits/decaf/ed448goldilocks/decaf.c#L1199

---

$ ghc -V
The Glorious Glasgow Haskell Compilation System, version 9.4.8

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/13/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-13.3.1_p20240614/work/gcc-13-20240614/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/13 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/13/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/13 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13 --disable-silent-rules --disable-dependency-tracking --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/13/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 13.3.1_p20240614 p17' --with-gcc-major-version-only --enable-libstdcxx-time --enable-lto --disable-libstdcxx-pch --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --enable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --with-zstd --without-isl --enable-default-pie --enable-default-ssp --disable-fixincludes --with-build-config=bootstrap-cet
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.3.1 20240614 (Gentoo 13.3.1_p20240614 p17)

$ grep 'model name' /proc/cpuinfo | head -1
model name      : AMD Ryzen 7 5700U with Radeon Graphics

---

Full configuration used:

./setup configure --enable-library-profiling --enable-tests --ghc-options=-j16 --ghc-option=-optc-march=znver2 --disable-executable-stripping --verbose --enable-shared --enable-executable-dynamic --disable-library-stripping --flags=-check_alignment --flags=integer-gmp --flags=-old_toolchain_inliner --flags=support_aesni --flag=support-deepseq --flags=support_pclmuldq --flags=support_rdrand --flags=support_sse --flags=use_target_attributes --ghc-option=-optc-ggdb

[hololeap]

I did some more testing and it seems some -march options trigger the segfault, while others do not. The ones I have tested are:

• Triggers segfault
  • x86-64-v3
  • znver1
  • znver2
• Does not trigger segfault
  • generic
  • x86-64
  • x86-64-v2

[hololeap]

I was advised to build with ASAN and UBSAN, which I accomplished by passing these configure flags:

• --ghc-option=-optc-fsanitize=address,undefined
• --ghc-option=-optl-fsanitize=address,undefined
• --ghc-option=-optc-lasan

When run with UBSAN_OPTIONS="print_stacktrace=1", the test suite gave this output:

https://gist.github.com/hololeap/232e2adc9b1a44d831ad6d1740874d0e

---

It was also pointed out that this code likely comes from here and that it hasn't been updated in a long time, so syncing in more recent code would be an easy first step.

[alfarelcynthesis]

Likely related to these, from cryptonite: issue, proposed fix, (and nixpkgs patch).

[kazu-yamamoto]

@hololeap Would you try the proposed fix and send a PR if resolved?