Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report C++ Info: via pytest_report_header() #4046

Merged
merged 2 commits into from
Jul 8, 2022

Conversation

rwgk
Copy link
Collaborator

@rwgk rwgk commented Jul 6, 2022

Description

Report C++ Info: via pytest_report_header(), to ease harvesting CI logs, and validating that ci.yml & cmake are working as intended.

Example output with context:

Running tests in directory "/usr/local/google/home/rwgk/forked/pybind11/tests":
=========================================================== test session starts ============================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
C++ Info: Debian Clang 13.0.1 C++17 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
rootdir: /usr/local/google/home/rwgk/forked/pybind11/tests, configfile: pytest.ini
collected 557 items

test_async.py ..                                                                                                                     [  0%]
...

Note that there appears to be an existing inconsistency (to be looked at separately):

$ grep 'C++ Info:' 3______3_____GCC_10_____C++20____x64.txt

2022-07-06T22:59:10.4150865Z C++ Info: 10.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1014__

All results:

$ unzip logs_21206.zip

$ simplify_filenames.py .

$ grep 'C++ Info:' *.txt | sed 's/2022-07-06T[^Z]*Z //' | sort -n

1______3.6_____MSVC_2019_____x86.txt:  C++ Info: MSVC 192930145 C++14 __pybind11_internals_v4_msvc_debug__
1______3.6_____ubuntu-latest_____x64_-DPYBIND11_FINDPYTHON=ON_-DCM.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
1______3.6_____ubuntu-latest_____x64_-DPYBIND11_FINDPYTHON=ON_-DCM.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
1______3.6_____ubuntu-latest_____x64_-DPYBIND11_FINDPYTHON=ON_-DCM.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
1______3.7_____Debian_____x86______Install.txt:C++ Info: 8.3.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
1______3.8_____CUDA_11.2_____Ubuntu_20.04.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
1______3.8_____MSVC_2019__Debug______x86_-DCMAKE_CXX_STANDARD=17.txt:  C++ Info: MSVC 192930145 C++17 __pybind11_internals_v4_msvc_debug__
1______3.9-dbg__deadsnakes______Valgrind_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
1______3.9-dbg__deadsnakes______Valgrind_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
1______3.9_____MSVC_2022_C++20_____x64.txt:  C++ Info: MSVC 193231332 C++20 __pybind11_internals_v4_msvc_debug__
1______3_____CentOS7__PGI_22.3_____x64.txt:C++ Info: EDG g++ 4.8.5 mode C++11 __pybind11_internals_v4_pgi_libstdcpp__
1______3_____centos7_____x64.txt:C++ Info: 4.8.5 20150623 (Red Hat 4.8.5-44) C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1002__
1______3_____Clang_3.6_____C++11_____x64.txt:C++ Info: 4.2.1 Compatible Clang 3.6.2 (tags/RELEASE_362/final) C++11 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
1______3_____GCC_7_____C++11____x64.txt:C++ Info: 7.5.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1011__
1______3_____ICC_latest_____x64.txt:C++ Info: Intel(R) C++ g++ 9.4 mode C++11 __pybind11_internals_v4_icc_libstdcpp_cxxabi1010__
1______3_____ICC_latest_____x64.txt:C++ Info: Intel(R) C++ g++ 9.4 mode C++17 __pybind11_internals_v4_icc_libstdcpp_cxxabi1010__
1______3_____windows-latest_____mingw64.txt:C++ Info: 12.1.0 C++11 __pybind11_internals_v4_mingw_libstdcpp_cxxabi1017__
1______3_____windows-latest_____mingw64.txt:C++ Info: 12.1.0 C++14 __pybind11_internals_v4_mingw_libstdcpp_cxxabi1017__
1______3_____windows-latest_____mingw64.txt:C++ Info: 12.1.0 C++17 __pybind11_internals_v4_mingw_libstdcpp_cxxabi1017__
2______3.11-dev__deadsnakes______x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
2______3.7_____MSVC_2019_____x86.txt:  C++ Info: MSVC 192930145 C++14 __pybind11_internals_v4_msvc_debug__
2______3.9_____MSVC_2019__Debug______x86_-DCMAKE_CXX_STANDARD=20.txt:  C++ Info: MSVC 192930145 C++20 __pybind11_internals_v4_msvc_debug__
2______3.9_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
2______3.9_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
2______3.9_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
2______3_____almalinux8_____x64.txt:C++ Info: 8.5.0 20210514 (Red Hat 8.5.0-10) C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
2______3_____Clang_3.7_____C++11_____x64.txt:C++ Info: 4.2.1 Compatible Clang 3.7.1 (tags/RELEASE_371/final) C++11 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
2______3_____GCC_latest_____C++11____x64.txt:C++ Info: 12.1.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1017__
2______3_____windows-latest_____mingw32.txt:C++ Info: 12.1.0 C++11 __pybind11_internals_v4_mingw_libstdcpp_cxxabi1017__
2______3_____windows-latest_____mingw32.txt:C++ Info: 12.1.0 C++14 __pybind11_internals_v4_mingw_libstdcpp_cxxabi1017__
2______3_____windows-latest_____mingw32.txt:C++ Info: 12.1.0 C++17 __pybind11_internals_v4_mingw_libstdcpp_cxxabi1017__
3______3.10_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
3______3.10_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
3______3.10_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
3______3.8_____MSVC_2019_____x86_-DCMAKE_CXX_STANDARD=17.txt:  C++ Info: MSVC 192930145 C++17 __pybind11_internals_v4_msvc_debug__
3______3_____almalinux9_____x64.txt:C++ Info: 11.2.1 20220127 (Red Hat 11.2.1-9) C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1016__
3______3_____Clang_3.9_____C++11_____x64.txt:C++ Info: 4.2.1 Compatible Clang 3.9.1 (tags/RELEASE_391/final) C++11 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
3______3_____GCC_10_____C++20____x64.txt:C++ Info: 10.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1014__
4______3.11-dev_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
4______3.11-dev_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
4______3.11-dev_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
4______3.9_____MSVC_2019_____x86_-DCMAKE_CXX_STANDARD=20.txt:  C++ Info: MSVC 192930145 C++20 __pybind11_internals_v4_msvc_debug__
4______3_____Clang_7_____C++11_____x64.txt:C++ Info: 4.2.1 Compatible Clang 7.0.1 (tags/RELEASE_701/final) C++11 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
5______3_____Clang_9_____C++11_____x64.txt:C++ Info: Clang 9.0.0 (tags/RELEASE_900/final) C++11 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
5______pypy-3.7_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
5______pypy-3.7_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
5______pypy-3.7_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
6______3_____Clang_dev_____C++11_____x64.txt:C++ Info: Debian Clang 15.0.0 C++11 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
6______pypy-3.8_____ubuntu-latest_____x64_-DPYBIND11_FINDPYTHON=ON.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
6______pypy-3.8_____ubuntu-latest_____x64_-DPYBIND11_FINDPYTHON=ON.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
6______pypy-3.8_____ubuntu-latest_____x64_-DPYBIND11_FINDPYTHON=ON.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
7______3_____Clang_5_____C++14_____x64.txt:C++ Info: 4.2.1 Compatible Clang 5.0.2 (tags/RELEASE_502/final) C++14 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
7______pypy-3.9_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++11 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
7______pypy-3.9_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v10000000_gcc_libstdcpp_cxxabi1013__
7______pypy-3.9_____ubuntu-latest_____x64.txt:C++ Info: 9.4.0 C++17 __pybind11_internals_v4_gcc_libstdcpp_cxxabi1013__
8______3.6_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
8______3.6_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
8______3.6_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
8______3_____Clang_10_____C++20_____x64.txt:C++ Info: Clang 10.0.0  C++20 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
9______3.9_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
9______3.9_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
9______3.9_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
9______3_____Clang_10_____C++17_____x64.txt:C++ Info: Clang 10.0.0  C++17 __pybind11_internals_v4_clang_libstdcpp_cxxabi1002__
10______3.10_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
10______3.10_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
10______3.10_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
11______3.11-dev_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
11______3.11-dev_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
11______3.11-dev_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
12______pypy-3.7_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
12______pypy-3.7_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
12______pypy-3.7_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
13______pypy-3.8_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
13______pypy-3.8_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
13______pypy-3.8_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
14______pypy-3.9_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++14 __pybind11_internals_v4_msvc_debug__
14______pypy-3.9_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v10000000_msvc_debug__
14______pypy-3.9_____windows-2022_____x64.txt:  C++ Info: MSVC 193231332 C++17 __pybind11_internals_v4_msvc_debug__
15______3.6_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
15______3.6_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
15______3.6_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
16______3.9_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
16______3.9_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
16______3.9_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
17______3.10_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
17______3.10_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
17______3.10_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
18______3.11-dev_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
18______3.11-dev_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
18______3.11-dev_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
19______pypy-3.7_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
19______pypy-3.7_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
19______pypy-3.7_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
20______pypy-3.8_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
20______pypy-3.8_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
20______pypy-3.8_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
21______pypy-3.9_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
21______pypy-3.9_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__
21______pypy-3.9_____macos-latest_____x64.txt:C++ Info: Apple LLVM 13.0.0 (clang-1300.0.29.30) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__
22______3.6_____windows-2019_____x64_-DPYBIND11_FINDPYTHON=ON.txt:  C++ Info: MSVC 192930145 C++14 __pybind11_internals_v4_msvc_debug__
22______3.6_____windows-2019_____x64_-DPYBIND11_FINDPYTHON=ON.txt:  C++ Info: MSVC 192930145 C++17 __pybind11_internals_v10000000_msvc_debug__
22______3.6_____windows-2019_____x64_-DPYBIND11_FINDPYTHON=ON.txt:  C++ Info: MSVC 192930145 C++17 __pybind11_internals_v4_msvc_debug__
23______3.9_____windows-2019_____x64.txt:  C++ Info: MSVC 192930145 C++14 __pybind11_internals_v4_msvc_debug__
23______3.9_____windows-2019_____x64.txt:  C++ Info: MSVC 192930145 C++17 __pybind11_internals_v10000000_msvc_debug__
23______3.9_____windows-2019_____x64.txt:  C++ Info: MSVC 192930145 C++17 __pybind11_internals_v4_msvc_debug__

Suggested changelog entry:

@rwgk rwgk marked this pull request as ready for review July 6, 2022 23:39
@rwgk rwgk requested review from henryiii and Skylion007 July 6, 2022 23:39
Copy link
Collaborator

@Skylion007 Skylion007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rwgk You should just add pytest_report_header(config) to the conftest.py file instead. See the documentation here: https://docs.pytest.org/en/7.1.x/example/simple.html#adding-info-to-test-report-header

@rwgk
Copy link
Collaborator Author

rwgk commented Jul 7, 2022

@rwgk You should just add pytest_report_header(config) to the conftest.py file instead. See the documentation here: https://docs.pytest.org/en/7.1.x/example/simple.html#adding-info-to-test-report-header

Done, thanks or the link!

Copy link
Collaborator

@Skylion007 Skylion007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure to change the name of this PR before merging and it looks good.

@rwgk rwgk changed the title Report C++ Info: from pytest_configure() Report C++ Info: via pytest_report_header() Jul 8, 2022
@rwgk
Copy link
Collaborator Author

rwgk commented Jul 8, 2022

Make sure to change the name of this PR before merging and it looks good.

Done, thanks!

@rwgk rwgk merged commit 85bc088 into pybind:master Jul 8, 2022
@rwgk rwgk deleted the pytest_configure_cpp_info branch July 8, 2022 00:51
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label Jul 8, 2022
@rwgk rwgk removed the needs changelog Possibly needs a changelog entry label Jul 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants