Analyse C-level coverage of pyperformance suite

[mdboom]

Results from gcov

We should look at this to see if there are any potential hot spots that aren't currently covered by the performance suite.

[Fidget-Spinner]

Those are some interesting results! I'm surprised ceval.c's coverage is so low. Just eyeballing the code, some hot spots we don't seem to cover in pyperformance:

• Async operations
• Pattern matching

Looks like we could benefit from some benchmarks in those areas.
The rest of the cold paths in ceval.c look like error handling cases which IMO are usually fine to ignore.

[mdboom]

Thanks for looking at these.

For async, there are some uses of this in the Pyston benchmarks we are trying to port to Python 3.11 so maybe those will soon be covered (though I haven't actually run gcov on it to confirm).

For pattern matching, that's a newer feature so hasn't been as critical to performance thus far, but I agree we should add coverage going forward.

Part of my motivation for doing this was the recently discovered regression in tracing performance -- what other usual-yet-critical features might we be missing. I haven't found any other red flags yet, but I also don't have a good intuition about the code base yet either.

[da-woods]

I recently wrote a small benchmark as part of writing Cython's implementation of pattern matching (https://github.com/da-woods/cython/blob/match-benchmark/Demos/benchmarks/patma.py)

It's pretty much the definition of "artificial microbenchmark", although it does at least cover some useful cases: exact dict vs inexact-dict mapping, unpacking large sequences into a star pattern. I suspect it's too simple for what you'd want, but I'm mentioning it on the off-change that it's useful.

[brandtbucher]

Sorry, late to the discussion. I've been farming some pattern matching microbenchmarks for a while now over at https://github.com/brandtbucher/patmaperformance. All it's really missing is coverage of mapping patterns.

I'm trying to avoid using totally artificial code, while still keeping the time mostly spent just doing pattern-matching stuff.

[mdboom]

I'm marking this as done for now. If we get a lot of value out of this, we may want to automate publishing these results, but I'm going to consider that a different, follow-on work item.