Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recalibrate unit tests #1036

Open
sipa opened this issue Dec 7, 2021 · 2 comments
Open

Recalibrate unit tests #1036

sipa opened this issue Dec 7, 2021 · 2 comments

Comments

@sipa
Copy link
Contributor

sipa commented Dec 7, 2021

The tests (src/tests.cpp and included modules) consist of some tests whose runtime is independent of the TEST_ITERS variable, and some which are. The ones that are independent take a significant amount of time (compared to the other ones, at a normal TEST_ITERS level). The ones that do depend on it do so in a rather inconsistent way.

So to improve upon that:

  • Turn constant-time tests into TEST_ITERS-proportional ones were possible.
  • Perhaps disable some very slow constant-time ones entirely at very low TEST_ITERS levels.
  • Run benchmarks on each of the proportional ones, and adjust to make sure they all get a somewhat representative amount of runtime in typical configurations.
@real-or-random
Copy link
Contributor

gprof of ./tests 1 shows that the new test_ecmult_constants() tests is indeed very heavy.

gprof-tests1.txt

@real-or-random
Copy link
Contributor

Related comment introduced in 17065f4:

secp256k1/src/tests.c

Lines 7091 to 7093 in 3efeb9d

/* Randomize the context only with probability 15/16
to make sure we test without context randomization from time to time.
TODO Reconsider this when recalibrating the tests. */

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants