Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4 packages from c-cube/qcheck at 0.22 #26196

Merged
merged 1 commit into from
Jul 9, 2024

Conversation

jmid
Copy link
Contributor

@jmid jmid commented Jul 4, 2024

This pull-request concerns:

  • qcheck.0.22: Compatibility package for qcheck
  • qcheck-alcotest.0.22: Alcotest backend for qcheck
  • qcheck-core.0.22: Core qcheck library
  • qcheck-ounit.0.22: OUnit backend for qcheck


🐫 Pull-request generated by opam-publish v2.3.0

@jmid
Copy link
Contributor Author

jmid commented Jul 4, 2024

I expect runtest to fail an expect test on the new opam-repo Windows workflow as it did on the repo: c-cube/qcheck#293

@jmid
Copy link
Contributor Author

jmid commented Jul 4, 2024

Ah, it seems the opam-repo Windows runner is not installing package --with-test, so all is well 😃

Note, there are no issues building and running QCheck under Windows, there is just a single expect test failure caused by a small difference in a floating point computation 🤷

@jmid
Copy link
Contributor Author

jmid commented Jul 5, 2024

I spotted expect test differences in 4 runs:

  • opam-2.0-ocaml-4.14 - qcheck-alcotest.0.22
  • opam-2.0-ocaml-5.2 - qcheck-alcotest.0.22
  • opam-2.1-ocaml-4.14 - qcheck-alcotest.0.22
  • opam-2.1-ocaml-5.2 - qcheck-alcotest.0.22

All involving alcotest.1.0.1 and caused by slight output difference, see, e.g.
https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/956dfbfb1a64b7045d4e8fd7ab8ccafd71bafb2f/variant/extras,opam-2.1-ocaml-5.2,qcheck-alcotest.0.22,tests

Full example pasted below.

As a consequence, we'll require at least alcotest.1.2.0 to guard against such output variations.

#=== ERROR while compiling qcheck-alcotest.0.22 ===============================#
# context              2.1.6 | linux/x86_64 | ocaml-base-compiler.5.2.0 | pinned(https://github.com/c-cube/qcheck/archive/v0.22.tar.gz)
# path                 ~/.opam/5.2/.opam-switch/build/qcheck-alcotest.0.22
# command              ~/.opam/opam-init/hooks/sandbox.sh build dune runtest -p qcheck-alcotest -j 39
# exit-code            1
# env-file             ~/.opam/log/qcheck-alcotest-7-b1c4e9.env
# output-file          ~/.opam/log/qcheck-alcotest-7-b1c4e9.out
### output ###
# File "example/alcotest/QCheck_alcotest_test.expected", line 1, characters 0-0:
# /usr/bin/git --no-pager diff --no-index --color=always -u _build/default/example/alcotest/QCheck_alcotest_test.expected _build/default/example/alcotest/QCheck_alcotest_test.output
# diff --git a/_build/default/example/alcotest/QCheck_alcotest_test.expected b/_build/default/example/alcotest/QCheck_alcotest_test.output
# index aaf5db1..7180645 100644
# --- a/_build/default/example/alcotest/QCheck_alcotest_test.expected
# +++ b/_build/default/example/alcotest/QCheck_alcotest_test.output
# @@ -1,49 +1,40 @@
#  qcheck random seed: 1234
# -Testing `my test'.
# -  [OK]          suite              0   list_rev_is_involutive.
# -  [FAIL]        suite              1   fail_sort_id.
# -  [FAIL]        suite              2   error_raise_exn.
# -  [OK]          suite              3   neg test pass (failing as expected).
# -  [FAIL]        suite              4   neg test unexpected success.
# -  [FAIL]        suite              5   neg fail with error.
# -  [FAIL]        suite              6   fail_check_err_message.
# -  [OK]          suite              7   tree_rev_is_involutive.
# -  [FAIL]        shrinking          0   debug_shrink.
# -┌──────────────────────────────────────────────────────────────────────────────┐
# -│ [FAIL]        suite              1   fail_sort_id.                           │
# -└──────────────────────────────────────────────────────────────────────────────┘
# +Testing my test.
# + ...                suite              0   list_rev_is_involutive.
[OK]                suite              0   list_rev_is_involutive.
# + ...                suite              1   fail_sort_id.
[FAIL]              suite              1   fail_sort_id.
# + ...                suite              2   error_raise_exn.
[FAIL]              suite              2   error_raise_exn.
# + ...                suite              3   neg test pass (failing as expected).
[OK]                suite              3   neg test pass (failing as expected).
# + ...                suite              4   neg test unexpected success.
[FAIL]              suite              4   neg test unexpected success.
# + ...                suite              5   neg fail with error.
[FAIL]              suite              5   neg fail with error.
# + ...                suite              6   fail_check_err_message.
[FAIL]              suite              6   fail_check_err_message.
# + ...                suite              7   tree_rev_is_involutive.
[OK]                suite              7   tree_rev_is_involutive.
# + ...                shrinking          0   debug_shrink.
[FAIL]              shrinking          0   debug_shrink.
# +-- suite.001 [fail_sort_id.] Failed --

@jmid jmid force-pushed the opam-publish-qcheck.0.22 branch from e3d5f3c to 2cb7236 Compare July 5, 2024 15:01
@jmid
Copy link
Contributor Author

jmid commented Jul 5, 2024

The above no pass with the alcotest.1.2.0 lower bound.

@jmid
Copy link
Contributor Author

jmid commented Jul 5, 2024

Overwise, so far I'm seeing

  • tezos-test-helpers.9.7 (failed: Failed to get sources of tezos-test-helpers.9.7: Bad checksum)
  • base32.1.0.0 (failed: Failed to get sources of base32.1.0.0: curl error code 404)
  • cborl.0.1.0 (failed: Failed to get sources of cborl.0.1.0: curl error code 404)
  • moonpool.0.6 (failed: The compilation of moonpool.0.6 failed at "dune build -p moonpool -j 255 @install @runtest".)
    which seems like an expect test failure due to different fiber behavior
  • bytestring.0.0.8 (failed: The compilation of bytestring.0.0.8 failed at "dune build -p bytestring -j 255 @install @runtest".) which seems like an expect test failure due to a different ppxlib pretty print

None of these are related to the qcheck.0.22 changes (one reducing GC pressure, one removing a needless mutability warning)

@jmid jmid mentioned this pull request Jul 5, 2024
@jmid
Copy link
Contributor Author

jmid commented Jul 5, 2024

  • octez-protocol-017-PtNairob-libs.18.0
  • tezos-baking-016-PtMumbai.17.3
  • tezos-baking-017-PtNairob.17.3
  • tezos-baking-alpha.17.3

all fail with an unexpected injection on the node A

  • saturn.0.4.1 also trigger 2 timeouts

and

  • hacl-star.0.7.0 and hacl-star.0.7.1 both fail with an Unbound value bytes

Again none of these are related to the small adjustments from the 0.22 release.

I've also filed #26204 so that we can get rid of base32 and cborl 404s going forward...

@mseri
Copy link
Member

mseri commented Jul 9, 2024

Thanks

@mseri mseri merged commit 6ec1413 into ocaml:master Jul 9, 2024
2 of 3 checks passed
@jmid jmid deleted the opam-publish-qcheck.0.22 branch July 25, 2024 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants