Merge bitcoin/bitcoin#24792: Update libsecp256k1 subtree to current m…

…aster

404c530 key: use secp256k1_schnorrsig_sign32 over deprecated secp256k1_schnorrsig_sign (fanquake)
ee30bf7 build: remove some no-longer-needed var unexporting from configure (fanquake)
2656629 build: remove --enable-experimental from libsecp256k1 configure (fanquake)
d960d4f build: fix MSVC build after subtree update (dhruv)
afb7a6f Squashed 'src/secp256k1/' changes from 0559fc6e41..8746600eec (fanquake)

Pull request description:

  The motivation for this bump is some small build cleanups, including [dropping the `--enable-experimental`](bitcoin-core/secp256k1@80cf4ee) flag from the libsecp configure  invocation, as well as some [now-redundant](bitcoin-core/secp256k1#1090) `pkg-config` variable exporting from our own configure. We also get the benefit of a slightly more efficient libsecp configure due to bitcoin-core/secp256k1#1088.

  This also includes a change in our code to migrate from using the [now deprecated](bitcoin-core/secp256k1@99e6568) `secp256k1_schnorrsig_sign` to `secp256k1_schnorrsig_sign32`.

  Guix Build (on x86_64):
  ```bash
  b9f6ad90c75f7edd7c4444c6c3401d8b6ab29a8da22ae22ddaedd94688227b5d  guix-build-404c53062bb8/output/aarch64-linux-gnu/SHA256SUMS.part
  250d47ae299d8385d5590518fa2adaabde76e2566fd27e12bf36b62663d13e13  guix-build-404c53062bb8/output/aarch64-linux-gnu/bitcoin-404c53062bb8-aarch64-linux-gnu-debug.tar.gz
  48d610dc6f5169f925f782571dac2f082695f89008beadad4adef4c1b583a612  guix-build-404c53062bb8/output/aarch64-linux-gnu/bitcoin-404c53062bb8-aarch64-linux-gnu.tar.gz
  8f04ee26e4079719e3935bd0e4287cc11a2a16875bf01e2a63d67492a1fa5367  guix-build-404c53062bb8/output/arm-linux-gnueabihf/SHA256SUMS.part
  7d7d7fcfb032bda92e53abd8d608257f0ef17b1e3e52a1414260b896786fb2dc  guix-build-404c53062bb8/output/arm-linux-gnueabihf/bitcoin-404c53062bb8-arm-linux-gnueabihf-debug.tar.gz
  30bae2ff3d044f4e39f992a68f6b296b7be2aea350bca4a0415c739a32c20bd9  guix-build-404c53062bb8/output/arm-linux-gnueabihf/bitcoin-404c53062bb8-arm-linux-gnueabihf.tar.gz
  5f550fb0b950250eeffce3480ec6403530b0880570a5860ef6c32a3e92eac92f  guix-build-404c53062bb8/output/arm64-apple-darwin/SHA256SUMS.part
  c10664d13aeec8c860bf72be833c738973ae18e4d28cdf08b2f9bee960ebff1d  guix-build-404c53062bb8/output/arm64-apple-darwin/bitcoin-404c53062bb8-arm64-apple-darwin-unsigned.dmg
  becab75b11cf4ca6f559f8eef835f3574629f6eb932ac716ed4f8c044a85831f  guix-build-404c53062bb8/output/arm64-apple-darwin/bitcoin-404c53062bb8-arm64-apple-darwin-unsigned.tar.gz
  bc86433652fe3552f6a13088191364ae7514c9fe3a244da86a6db096bb4922fc  guix-build-404c53062bb8/output/arm64-apple-darwin/bitcoin-404c53062bb8-arm64-apple-darwin.tar.gz
  1f585cb9a1356343df4b2726ecfe2598c9903304afb047c047c2cef318555dd3  guix-build-404c53062bb8/output/dist-archive/bitcoin-404c53062bb8.tar.gz
  9ede534ba2c6cecb550473eead195627327e826ebb0118e23d60ab482d40e241  guix-build-404c53062bb8/output/powerpc64-linux-gnu/SHA256SUMS.part
  77ddb7d7d639b1dd4508468a8ef27e45b35c8b2f8624584a70e6b64798a4ea7a  guix-build-404c53062bb8/output/powerpc64-linux-gnu/bitcoin-404c53062bb8-powerpc64-linux-gnu-debug.tar.gz
  36178c1f1c12942ff05275daa3570f8b45419ee8d9f391d750afb405219986f0  guix-build-404c53062bb8/output/powerpc64-linux-gnu/bitcoin-404c53062bb8-powerpc64-linux-gnu.tar.gz
  8a15a4da7a9a5e00c49d9aeedf3c6fc666c0d230be1369eac7caf4571d5905e0  guix-build-404c53062bb8/output/powerpc64le-linux-gnu/SHA256SUMS.part
  400c58113f2d07c87e03c8528b292c6aca808a2bccae4b041cad3a26a05b6aad  guix-build-404c53062bb8/output/powerpc64le-linux-gnu/bitcoin-404c53062bb8-powerpc64le-linux-gnu-debug.tar.gz
  3b9f9d8614ac3a27416e53354b2b0a64d364f91493e9d0f41583a6f492546824  guix-build-404c53062bb8/output/powerpc64le-linux-gnu/bitcoin-404c53062bb8-powerpc64le-linux-gnu.tar.gz
  98506b23ee08ad8af958f816da2e4518d661e88d5c6308de1f5e3b2fc787b86c  guix-build-404c53062bb8/output/riscv64-linux-gnu/SHA256SUMS.part
  c701a7b77cea4fdc2588b511f1b2c71b89c83bfba19fdb2ac113a5a4b14ac392  guix-build-404c53062bb8/output/riscv64-linux-gnu/bitcoin-404c53062bb8-riscv64-linux-gnu-debug.tar.gz
  34d58e6392cd58b3c76e30cd8600c0dbefba7e9c6d5df78c3ef23e81c4e4d26a  guix-build-404c53062bb8/output/riscv64-linux-gnu/bitcoin-404c53062bb8-riscv64-linux-gnu.tar.gz
  92fa30e9c6d81dd1e1514b65d3e1abe68ded897237cd99f66aa760d445109c04  guix-build-404c53062bb8/output/x86_64-apple-darwin/SHA256SUMS.part
  bee180b02f178ae9980ef159f65913a71cbd037c4aff5f2906af5f174a677da3  guix-build-404c53062bb8/output/x86_64-apple-darwin/bitcoin-404c53062bb8-x86_64-apple-darwin-unsigned.dmg
  ad7d18d779ab7a7944817d1f368d0a6bdd174bf1211b0f90180c8ccf04ec4062  guix-build-404c53062bb8/output/x86_64-apple-darwin/bitcoin-404c53062bb8-x86_64-apple-darwin-unsigned.tar.gz
  7489d1d5d48ad95cf58bb11b5fdeccadac6fa758784fb498529fca2330abe069  guix-build-404c53062bb8/output/x86_64-apple-darwin/bitcoin-404c53062bb8-x86_64-apple-darwin.tar.gz
  74660fb0ebce2a08b03980a57bffcad62e078dc967a74d2395660ff51c019640  guix-build-404c53062bb8/output/x86_64-linux-gnu/SHA256SUMS.part
  cd377fa6b46276c2f8a32e199e6f9adf6aa67315688656709d6dc0744d54a837  guix-build-404c53062bb8/output/x86_64-linux-gnu/bitcoin-404c53062bb8-x86_64-linux-gnu-debug.tar.gz
  919c521950369d8ad46db2d15b00abb488abfb080d157a41b2db429122a428ed  guix-build-404c53062bb8/output/x86_64-linux-gnu/bitcoin-404c53062bb8-x86_64-linux-gnu.tar.gz
  2debca995d432965a8786b6ff74aed42e9e2f1cb0fecbe2d9fc5b850c192fcff  guix-build-404c53062bb8/output/x86_64-w64-mingw32/SHA256SUMS.part
  e33169f684fb031ec18ed39812617d3eb263257f6c7564b8f4c974ad05fe672c  guix-build-404c53062bb8/output/x86_64-w64-mingw32/bitcoin-404c53062bb8-win64-debug.zip
  029d0a4180cb908d517fcf689dcf46d42fbf383e11dc609711617066ae039ab0  guix-build-404c53062bb8/output/x86_64-w64-mingw32/bitcoin-404c53062bb8-win64-setup-unsigned.exe
  7e349c688cac66436562c4805f420b0536db5a3b3abf54d0e8c7752f59874a5c  guix-build-404c53062bb8/output/x86_64-w64-mingw32/bitcoin-404c53062bb8-win64-unsigned.tar.gz
  1bff98e82e95c93d6060227408502f5e2d8597d526b912cb6dc0a90ae3094a8f  guix-build-404c53062bb8/output/x86_64-w64-mingw32/bitcoin-404c53062bb8-win64.zip
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 404c530, I checked the changes to our tree thoroughly but didn't review all upstream secp256k1 changes in detail.
  gruve-p:
    ACK bitcoin/bitcoin@404c530
  real-or-random:
    utACK 404c530 I reviewed the diff to Core, I'm with updating to libsecp256k1 master, but I haven't verified that the libsecp256k1 tree here has been updated correctly

Tree-SHA512: e6a6db93ea60ed500df5065178784a915da94adfa7bd45fdbd7b19d701154987ff38c1df7f318119e6c2cb98e28e1ea2eb725bef93d4088403e14537ebffb032

build_msvc/libsecp256k1/libsecp256k1.vcxproj

@@ -8,6 +8,8 @@
     <ConfigurationType>StaticLibrary</ConfigurationType>
   </PropertyGroup>
   <ItemGroup>
+    <ClCompile Include="..\..\src\secp256k1\src\precomputed_ecmult.c" />
+    <ClCompile Include="..\..\src\secp256k1\src\precomputed_ecmult_gen.c" />
     <ClCompile Include="..\..\src\secp256k1\src\secp256k1.c" />
   </ItemGroup>
   <ItemDefinitionGroup>

configure.ac

@@ -1959,15 +1959,7 @@ LIBS_TEMP="$LIBS"
 unset LIBS
 LIBS="$LIBS_TEMP"
 
-PKGCONFIG_PATH_TEMP="$PKG_CONFIG_PATH"
-unset PKG_CONFIG_PATH
-PKG_CONFIG_PATH="$PKGCONFIG_PATH_TEMP"
-
-PKGCONFIG_LIBDIR_TEMP="$PKG_CONFIG_LIBDIR"
-unset PKG_CONFIG_LIBDIR
-PKG_CONFIG_LIBDIR="$PKGCONFIG_LIBDIR_TEMP"
-
-ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --enable-module-recovery --enable-module-schnorrsig --enable-experimental"
+ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --enable-module-recovery --enable-module-schnorrsig"
 AC_CONFIG_SUBDIRS([src/secp256k1])
 
 AC_OUTPUT

src/key.cpp

@@ -288,7 +288,7 @@ bool CKey::SignSchnorr(const uint256& hash, Span<unsigned char> sig, const uint2
         uint256 tweak = XOnlyPubKey(pubkey_bytes).ComputeTapTweakHash(merkle_root->IsNull() ? nullptr : merkle_root);
         if (!secp256k1_keypair_xonly_tweak_add(GetVerifyContext(), &keypair, tweak.data())) return false;
     }
-    bool ret = secp256k1_schnorrsig_sign(secp256k1_context_sign, sig.data(), hash.data(), &keypair, aux.data());
+    bool ret = secp256k1_schnorrsig_sign32(secp256k1_context_sign, sig.data(), hash.data(), &keypair, aux.data());
     if (ret) {
         // Additional verification step to prevent using a potentially corrupted signature
         secp256k1_xonly_pubkey pubkey_verify;

src/secp256k1/.cirrus.yml

@@ -4,10 +4,10 @@ env:
   # Specific warnings can be disabled with -Wno-error=foo.
   # -pedantic-errors is not equivalent to -Werror=pedantic and thus not implied by -Werror according to the GCC manual.
   WERROR_CFLAGS: -Werror -pedantic-errors
-  MAKEFLAGS: -j2
+  MAKEFLAGS: -j4
   BUILD: check
   ### secp256k1 config
-  STATICPRECOMPUTATION: yes
+  ECMULTWINDOW: auto
   ECMULTGENPRECISION: auto
   ASM: no
   WIDEMUL: auto
@@ -23,6 +23,8 @@ env:
   BENCH: yes
   SECP256K1_BENCH_ITERS: 2
   CTIMETEST: yes
+  # Compile and run the tests
+  EXAMPLES: yes
 
 cat_logs_snippet: &CAT_LOGS
   always:
@@ -50,28 +52,32 @@ merge_base_script_snippet: &MERGE_BASE
     - git config --global user.name "ci"
     - git merge FETCH_HEAD  # Merge base to detect silent merge conflicts
 
-task:
-  name: "x86_64: Linux (Debian stable)"
+linux_container_snippet: &LINUX_CONTAINER
   container:
     dockerfile: ci/linux-debian.Dockerfile
     # Reduce number of CPUs to be able to do more builds in parallel.
     cpu: 1
+    # Gives us more CPUs for free if they're available.
+    greedy: true
     # More than enough for our scripts.
     memory: 1G
+
+task:
+  name: "x86_64: Linux (Debian stable)"
+  << : *LINUX_CONTAINER
   matrix: &ENV_MATRIX
     - env: {WIDEMUL:  int64,  RECOVERY: yes}
-    - env: {WIDEMUL:  int64,                 ECDH: yes, EXPERIMENTAL: yes, SCHNORRSIG: yes}
+    - env: {WIDEMUL:  int64,                 ECDH: yes, SCHNORRSIG: yes}
     - env: {WIDEMUL: int128}
-    - env: {WIDEMUL: int128,  RECOVERY: yes,            EXPERIMENTAL: yes, SCHNORRSIG: yes}
-    - env: {WIDEMUL: int128,                 ECDH: yes, EXPERIMENTAL: yes, SCHNORRSIG: yes}
+    - env: {WIDEMUL: int128,  RECOVERY: yes,            SCHNORRSIG: yes}
+    - env: {WIDEMUL: int128,                 ECDH: yes, SCHNORRSIG: yes}
     - env: {WIDEMUL: int128,  ASM: x86_64}
-    - env: {                  RECOVERY: yes,            EXPERIMENTAL: yes, SCHNORRSIG: yes}
-    - env: {                  STATICPRECOMPUTATION: no}
+    - env: {                  RECOVERY: yes,            SCHNORRSIG: yes}
     - env: {BUILD: distcheck, WITH_VALGRIND: no, CTIMETEST: no, BENCH: no}
     - env: {CPPFLAGS: -DDETERMINISTIC}
     - env: {CFLAGS: -O0, CTIMETEST: no}
-    - env: { ECMULTGENPRECISION: 2 }
-    - env: { ECMULTGENPRECISION: 8 }
+    - env: { ECMULTGENPRECISION: 2, ECMULTWINDOW: 2 }
+    - env: { ECMULTGENPRECISION: 8, ECMULTWINDOW: 4 }
   matrix:
     - env:
         CC: gcc
@@ -84,15 +90,11 @@ task:
 
 task:
   name: "i686: Linux (Debian stable)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     HOST: i686-linux-gnu
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
   matrix:
     - env:
@@ -134,8 +136,10 @@ task:
   ##   - rm /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
   ##
   brew_valgrind_pre_script:
+    # Retry a few times because this tends to fail randomly.
+    - for i in {1..5}; do brew update && break || sleep 15; done
     - brew config
-    - brew tap --shallow LouisBrunner/valgrind
+    - brew tap LouisBrunner/valgrind
     # Fetch valgrind source but don't build it yet.
     - brew fetch --HEAD LouisBrunner/valgrind/valgrind
   brew_valgrind_cache:
@@ -165,18 +169,14 @@ task:
 
 task:
   name: "s390x (big-endian): Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-s390x
     SECP256K1_TEST_ITERS: 16
     HOST: s390x-linux-gnu
     WITH_VALGRIND: no
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
   << : *MERGE_BASE
@@ -188,42 +188,34 @@ task:
 
 task:
   name: "ARM32: Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-arm
     SECP256K1_TEST_ITERS: 16
     HOST: arm-linux-gnueabihf
     WITH_VALGRIND: no
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
   matrix:
     - env: {}
-    - env: {ASM: arm}
+    - env: {EXPERIMENTAL: yes, ASM: arm}
   << : *MERGE_BASE
   test_script:
     - ./ci/cirrus.sh
   << : *CAT_LOGS
 
 task:
   name: "ARM64: Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-aarch64
     SECP256K1_TEST_ITERS: 16
     HOST: aarch64-linux-gnu
     WITH_VALGRIND: no
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
   << : *MERGE_BASE
@@ -233,18 +225,14 @@ task:
 
 task:
   name: "ppc64le: Linux (Debian stable, QEMU)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: qemu-ppc64le
     SECP256K1_TEST_ITERS: 16
     HOST: powerpc64le-linux-gnu
     WITH_VALGRIND: no
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
   << : *MERGE_BASE
@@ -254,18 +242,14 @@ task:
 
 task:
   name: "x86_64 (mingw32-w64): Windows (Debian stable, Wine)"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     WRAPPER_CMD: wine64-stable
     SECP256K1_TEST_ITERS: 16
     HOST: x86_64-w64-mingw32
     WITH_VALGRIND: no
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
   << : *MERGE_BASE
@@ -275,23 +259,23 @@ task:
 
 # Sanitizers
 task:
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 2G
+  << : *LINUX_CONTAINER
   env:
     ECDH: yes
     RECOVERY: yes
-    EXPERIMENTAL: yes
     SCHNORRSIG: yes
     CTIMETEST: no
   matrix:
     - name: "Valgrind (memcheck)"
+      container:
+        cpu: 2
       env:
         # The `--error-exitcode` is required to make the test fail if valgrind found errors, otherwise it'll return 0 (https://www.valgrind.org/docs/manual/manual-core.html)
         WRAPPER_CMD: "valgrind --error-exitcode=42"
         SECP256K1_TEST_ITERS: 2
     - name: "UBSan, ASan, LSan"
+      container:
+        memory: 2G
       env:
         CFLAGS: "-fsanitize=undefined,address -g"
         UBSAN_OPTIONS: "print_stacktrace=1:halt_on_error=1"
@@ -302,11 +286,10 @@ task:
   matrix:
     - env:
         ASM: auto
-        STATICPRECOMPUTATION: yes
     - env:
         ASM: no
-        STATICPRECOMPUTATION: no
         ECMULTGENPRECISION: 2
+        ECMULTWINDOW: 2
   matrix:
     - env:
         CC: clang
@@ -320,21 +303,24 @@ task:
 
 task:
   name: "C++ -fpermissive"
-  container:
-    dockerfile: ci/linux-debian.Dockerfile
-    cpu: 1
-    memory: 1G
+  << : *LINUX_CONTAINER
   env:
     # ./configure correctly errors out when given CC=g++.
     # We hack around this by passing CC=g++ only to make.
     CC: gcc
-    MAKEFLAGS: -j2 CC=g++ CFLAGS=-fpermissive\ -g
+    MAKEFLAGS: -j4 CC=g++ CFLAGS=-fpermissive\ -g
     WERROR_CFLAGS:
-    EXPERIMENTAL: yes
     ECDH: yes
     RECOVERY: yes
     SCHNORRSIG: yes
   << : *MERGE_BASE
   test_script:
     - ./ci/cirrus.sh
   << : *CAT_LOGS
+
+task:
+  name: "sage prover"
+  << : *LINUX_CONTAINER
+  test_script:
+    - cd sage
+    - sage prove_group_implementations.sage

src/secp256k1/.gitattributes

@@ -1,2 +1,2 @@
-src/ecmult_static_pre_g.h linguist-generated
-src/ecmult_gen_static_prec_table.h linguist-generated
+src/precomputed_ecmult.c linguist-generated
+src/precomputed_ecmult_gen.c linguist-generated

src/secp256k1/.gitignore

@@ -3,14 +3,19 @@ bench_ecmult
 bench_internal
 tests
 exhaustive_tests
-gen_ecmult_gen_static_prec_table
-gen_ecmult_static_pre_g
+precompute_ecmult_gen
+precompute_ecmult
 valgrind_ctime_test
+ecdh_example
+ecdsa_example
+schnorr_example
 *.exe
 *.so
 *.a
 *.csv
 !.gitignore
+*.log
+*.trs
 
 Makefile
 configure
@@ -41,6 +46,7 @@ coverage.*.html
 
 src/libsecp256k1-config.h
 src/libsecp256k1-config.h.in
+build-aux/ar-lib
 build-aux/config.guess
 build-aux/config.sub
 build-aux/depcomp