Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for wildcards in CLI test and tag filters #56

Merged
merged 17 commits into from
Jan 16, 2023
Merged

Conversation

cschreib
Copy link
Member

@cschreib cschreib commented Jan 11, 2023

This PR does the following:

  • Adds support for wildcards (* match-all character) in filters on the command line interface, for both tags and test case names.
  • Adds support for exclusions patterns (~...).
  • Adds support for multiple filters on the command line, mimicking the Catch2 behavior.
  • Changed the full test name for typed tests from NAME [TYPE] to NAME <TYPE> to prevent ambiguity with tags.
  • Use std::chrono::steady_clock instead of high_resolution_clock, since the latter can end up being a typedef to system_clock, which is less desirable.

Note on benchmarks: For some reason I have not yet been able to identify, the runtime benchmarks on my machine have generally become slower in the past weeks. I had to re-run all the benchmarks for the competitor libraries, to compare on the same grounds. It seems that most of the run time in snitch is spent in std::chrono::steady_clock::now(), which is used to report test duration. Setting SNITCH_WITH_TIMINGS = 0 brings the snitch run time performance on par with Boost.UT. I don't know what could have changed in my machine to make time measurements much slower, but I did upgrade linux kernel and BIOS.

@codecov-commenter
Copy link

codecov-commenter commented Jan 14, 2023

Codecov Report

Merging #56 (3497bc6) into main (758451c) will decrease coverage by 0.19%.
The diff coverage is 89.67%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #56      +/-   ##
==========================================
- Coverage   92.75%   92.56%   -0.20%     
==========================================
  Files           2        2              
  Lines         994     1049      +55     
==========================================
+ Hits          922      971      +49     
- Misses         72       78       +6     
Impacted Files Coverage Δ
include/snitch/snitch.hpp 96.06% <ø> (+0.02%) ⬆️
src/snitch.cpp 90.95% <89.67%> (-0.19%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 758451c...3497bc6. Read the comment docs.

I re-ran all benchmarks, since all runtime timings for snitch inexplicably went up significantly on my machine (even going back to previous commits with recorded runtime timings). It could be the BIOS update I did a week ago, or the kernel update. This time I used a script to run all benchmarks, which may reduce variability.
@cschreib cschreib merged commit ea200a0 into main Jan 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants