add badges, minimal codecov, mk2

[bollwyvl]

• follow-on to Add README badges, branch coverage #351
• hoping to see coverage reports here 🤞

[bollwyvl]

Oooh, i do see a check already (presumably because 99ba18e was already reported on).

[bollwyvl]

Cool. So this PR now shows all the bells and whistles. We can turn them down by customizing a codecov.yml but the defaults seem fine and shouty at me for having broken coverage.

The goal on this isn't to make anybody bad about their contributions, but rather to help reduce any back-and-forth of hey, it'd be nice if we had a test for this new whizzbang feature.

[jorisvandenbossche]

Personally, I would maybe prefer to only have the codecov status check, and not the comment (my experience with codecov from other projects is that the commenting can be quite noisy ..)

[bollwyvl]

Great, I'll see if I can find the knobs to turn it down to just show the red x in the checks.

[drammock]

FYI, in the "files changed" view of PRs, you can show/hide codecov's (and everyone else's) comments by pressing "a". My personal experience (with codecov in MNE-Python repo) is that the inline comments can be helpful if there's just one or two of them, but more than that and they quickly become a nuisance. Learning the "a" shortcut allowed us to keep inline comments turned on.

[choldgraf]

Why would this PR trigger any red in the codecov though? Isn't it just CI/CD changes?

[bollwyvl]

Isn't it just CI/CD changes?

I added the --cov-branch switch.

update: took it out again so it should come up ✔️

[bollwyvl]

Ha, i deleted the comment, and it put it back. But have pushed with a minimal codecov. Guess if i delete it again (after it reports) it shouldn't put it back.

us to keep inline comments turned on.

I do find the annotations useful, but we'll see how it plays out. a is a good hotkey to know, for sure! I'd love to figure out a way to tie that all the way back in #294, but that sounds ... hard.

[bollwyvl]

Ok, it reported, threw the threshold flag. Will try re-deleting the comment, maybe re-kicking...

[bollwyvl]

I do think it would be good to put the --cov-branch back... but perhaps on a PR intended to actually improve coverage :)

[choldgraf]

@bollwyvl wanna open up an issue for "improve test coverage and add the flag bag in when that happens"? I'm assuming this is ready to go otherwise?

[bollwyvl]

Yeah, this one is ready to go, and we can see whatever other things we want to tweak. If I'm taking a real run at coverage on a project, I usually reach for hypothesis... The question is just how much we can do when sphinx is in the loop, for the reasons mentioned elsewhere, mostly performance... If it's touching disk, it's probably too slow to be worth it. Will issue up.

[choldgraf]

cool - thanks for adding this!