gh-105540: Convert `pytest` tests of `cases_generator` to regular tests #106713

sobolevn · 2023-07-13T08:38:19Z

I went ahead and converted them :)

Now they will be executed the same way as regular tests.
I've taken the approach we use for https://github.com/python/cpython/blob/main/Lib/test/test_clinic.py

Local run:

» ./python.exe -m test -v test_generated_cases
== CPython 3.13.0a0 (heads/main:da98ed0aa04, Jul 12 2023, 23:50:11) [Clang 14.0.3 (clang-1403.0.22.14.1)]
== macOS-13.4.1-arm64-arm-64bit little-endian
== Python build: debug
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_77465æ
== CPU count: 12
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.56 Run tests sequentially
0:00:00 load avg: 1.56 [1/1] test_generated_cases
test_effect_sizes (test.test_generated_cases.TestEffects.test_effect_sizes) ... ok
test_array_error_if (test.test_generated_cases.TestGeneratedCases.test_array_error_if) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpnwp2ip6m
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp83kj4oij
ok
test_array_input (test.test_generated_cases.TestGeneratedCases.test_array_input) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpyq_8mt3h
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmptssmvzyp
ok
test_array_input_output (test.test_generated_cases.TestGeneratedCases.test_array_input_output) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp9ej806j9
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_rqt1jh7
ok
test_array_output (test.test_generated_cases.TestGeneratedCases.test_array_output) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp1j80dud3
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp0a1lgx20
ok
test_binary_op (test.test_generated_cases.TestGeneratedCases.test_binary_op) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp6tkafxpr
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8nc34ekw
ok
test_cache_effect (test.test_generated_cases.TestGeneratedCases.test_cache_effect) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp0cbo72dj
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8qhl_sfc
ok
test_cond_effect (test.test_generated_cases.TestGeneratedCases.test_cond_effect) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpx4adjvf8
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpq81jd5gj
ok
test_error_if_plain (test.test_generated_cases.TestGeneratedCases.test_error_if_plain) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmplae6z0p3
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpiritl87t
ok
test_error_if_plain_with_comment (test.test_generated_cases.TestGeneratedCases.test_error_if_plain_with_comment) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpc9w7n9zq
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmposc9ygia
ok
test_error_if_pop (test.test_generated_cases.TestGeneratedCases.test_error_if_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpyu93upkp
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpcual0bhu
ok
test_inst_no_args (test.test_generated_cases.TestGeneratedCases.test_inst_no_args) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpcchbpx7s
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpbqgi_phs
ok
test_inst_one_pop (test.test_generated_cases.TestGeneratedCases.test_inst_one_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpn9cco114
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp1s9r5wbx
ok
test_inst_one_push (test.test_generated_cases.TestGeneratedCases.test_inst_one_push) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7c48tflr
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpda_z7uoc
ok
test_inst_one_push_one_pop (test.test_generated_cases.TestGeneratedCases.test_inst_one_push_one_pop) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7alht925
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_0m7n5zw
ok
test_macro_cond_effect (test.test_generated_cases.TestGeneratedCases.test_macro_cond_effect) ... Read 2 instructions/ops, 1 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpm55x7lbt
Wrote 0 instructions, 1 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp2mad7oyz
ok
test_macro_instruction (test.test_generated_cases.TestGeneratedCases.test_macro_instruction) ... Read 3 instructions/ops, 1 macros, 0 pseudos, and 1 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp7igsdpls
Wrote 2 instructions, 1 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpzqxw_efb
ok
test_overlap (test.test_generated_cases.TestGeneratedCases.test_overlap) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp_ct08u9k
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmphn9i5p0i
ok
test_predictions_and_eval_breaker (test.test_generated_cases.TestGeneratedCases.test_predictions_and_eval_breaker) ... Read 2 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp8w5ljdry
Wrote 2 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmp6jjdnwrr
ok
test_suppress_dispatch (test.test_generated_cases.TestGeneratedCases.test_suppress_dispatch) ... Read 1 instructions/ops, 0 macros, 0 pseudos, and 0 families from /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpbdh1a1wn
Wrote 1 instructions, 0 macros, and 0 pseudos to /var/folders/vw/n7c5l8m94zb072h487pnlpzm0000gn/T/tmpygejojvf
ok

----------------------------------------------------------------------
Ran 20 tests in 0.021s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 81 ms
Tests result: SUCCESS

Issue: cases_generator tests aren't run by CI, and are broken #105540

…ar tests

sobolevn · 2023-07-13T09:06:42Z

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

Fixing tests failure. I don't have access to Win, so I will experiment :)

gvanrossum

Thank you, thank you, thank you!

Tools/cases_generator/generate_cases.py

gvanrossum · 2023-07-14T00:00:17Z

Tools/cases_generator/generate_cases.py

@@ -1584,6 +1573,17 @@ def wrap_macro(self, mac: MacroInstruction):
            self.out.emit(f"DISPATCH();")


+def prettify_filename(filename: str) -> str:
+    # Make filename more user-friendly and less platform-specific,
+    # it is only used for error reporting at this point.


That is incorrect, and is not the reason why we go through this much effort. The prettified filename is incorporated in the output (as part of the header explaining that this was a generated file) and must be the same regardless of where it was generated, otherwise you get spurious diffs when e.g. running the tool on Windows.

Lib/test/test_generated_cases.py

sobolevn · 2023-07-14T06:54:35Z

Looks like something broke our tests :(

gvanrossum · 2023-07-16T22:32:11Z

Huh, tests pass for me locally. Let me see if I can merge.

gvanrossum · 2023-07-16T22:39:49Z

If the tests pass this time I will merge this upstream. I don't care enough about the one comment to hold it up, and I'd like to add some new tests.

gvanrossum · 2023-07-16T23:08:57Z

Oh dang. I wish the closing """ of the input/output text fragments were one tab stop to the right. But it doesn't matter enough for another PR.

sobolevn · 2023-07-17T08:15:29Z

Thanks! I will continue to make them better, since they now are a part of Lib/test :)

gvanrossum · 2023-07-17T14:16:19Z

Great, we need more eyeballs on this code generator.

pythongh-105540: Convert pytest tests of cases_generator to regul…

96ebd50

…ar tests

sobolevn requested a review from gvanrossum July 13, 2023 08:38

bedevere-bot added awaiting review tests Tests in the Lib/test dir labels Jul 13, 2023

bedevere-bot mentioned this pull request Jul 13, 2023

cases_generator tests aren't run by CI, and are broken #105540

Closed

4 tasks

sobolevn added the skip news label Jul 13, 2023

sobolevn added 5 commits July 13, 2023 12:24

Fix test run on Windows

b7ea637

Fix Linux build

f22fd88

Fix Windows build

0bfb1e1

Fix Windows build

79f01b8

Fix Windows build

2286f99

gvanrossum reviewed Jul 14, 2023

View reviewed changes

Add unittest.main

58ebcc7

Merge branch 'main' into issue-105540

e5ed260

gvanrossum merged commit c413207 into python:main Jul 16, 2023

bedevere-bot removed the awaiting review label Jul 16, 2023

efimov-mikhail mentioned this pull request Oct 14, 2024

gh-119786: [doc] broken link and typo fix in interpreter_definition.md #125455

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-105540: Convert `pytest` tests of `cases_generator` to regular tests #106713

gh-105540: Convert `pytest` tests of `cases_generator` to regular tests #106713

sobolevn commented Jul 13, 2023 •

edited

Loading

sobolevn commented Jul 13, 2023 •

edited

Loading

gvanrossum left a comment

gvanrossum Jul 14, 2023

sobolevn commented Jul 14, 2023

gvanrossum commented Jul 16, 2023

gvanrossum commented Jul 16, 2023

gvanrossum commented Jul 16, 2023

sobolevn commented Jul 17, 2023

gvanrossum commented Jul 17, 2023

gh-105540: Convert pytest tests of cases_generator to regular tests #106713

gh-105540: Convert pytest tests of cases_generator to regular tests #106713

Conversation

sobolevn commented Jul 13, 2023 • edited Loading

sobolevn commented Jul 13, 2023 • edited Loading

gvanrossum left a comment

Choose a reason for hiding this comment

gvanrossum Jul 14, 2023

Choose a reason for hiding this comment

sobolevn commented Jul 14, 2023

gvanrossum commented Jul 16, 2023

gvanrossum commented Jul 16, 2023

gvanrossum commented Jul 16, 2023

sobolevn commented Jul 17, 2023

gvanrossum commented Jul 17, 2023

gh-105540: Convert `pytest` tests of `cases_generator` to regular tests #106713

gh-105540: Convert `pytest` tests of `cases_generator` to regular tests #106713

sobolevn commented Jul 13, 2023 •

edited

Loading

sobolevn commented Jul 13, 2023 •

edited

Loading