WIP More precise coverage

[carnaval]

Make coverage depends on lowered code statement number instead of text
line numbers. The reported number is the minimum count of every statements
mapping to the relevant line.

        - function f(b)
        1     b && (x = 4)
        1     x = 2
        - end

        - f(true)

        - function f(b)
        0     b && (x = 4)
        1     x = 2
        - end

        - f(false)

just hacked this, not tested at all, consider this my argument against #11792 :-)

[timholy]

👍

[ScottPJones]

Hey, you won't get any argument from me! I did #11792 just because people said the by line coverage couldn't be fixed...
This is exactly what I wanted originally!
👍 💯

[tkelman]

what's pc ?

[ScottPJones]

pc = program counter = ip = instruction pointer

[carnaval]

yeah the comment is wrong now also :-) and the code a bit ugly, I'll do a cleanup pass tomorrow

[StefanKarpinski]

This seems great. Does reporting fractional coverage make sense? I.e. 0.5/1 or something like that.

[JeffBezanson]

Love it!

[kshyatt]

💯 !

[mschauer]

How about reporting the range of counts so "partial bottlenecks" are visible (bottlenecks constraint to parts of the line).

[ScottPJones]

That sounds like a good idea @mschauer

[tkelman]

Right now the .cov format is sort of designed for reporting a single number, right? So either we would have to extend that format to report additional information, or output multiple copies of the file for different choices of a single number to report?

[ScottPJones]

I'd figured a simple extension would be done, like min:max instead of count.
BTW, what is done to handle multiple processes trying to write coverage information at once?

[IainNZ]

So we now have codecov.io support, which can track partial coverage. Just to kinda incentivize getting this over the line.

[waldyrious]

So we now have codecov.io support

Noob comment here, but doesn't that mean something should be visible on this page? Is it just a matter of time, or am I looking at the wrong place?

[IainNZ]

Did anyone say it should be? For now its living at https://codecov.io/github/kshyatt/julia?ref=master

[kshyatt]

So the buildbots aren't running right now, @waldyrious, for a variety of reasons (ask @tkelman or @staticfloat for more). I've been running coverage on my fork of Julia for now. If anyone would like some help getting a copy of Julia coverage running on their own machines, drop me a line any time.

[waldyrious]

Did anyone say it should be? For now its living at https://codecov.io/github/kshyatt/julia?ref=master

I fully admitted that I could be looking at the wrong place. Thanks for the link.

@kshyatt thanks for the clarifications :)

[jiahao]

bump

[Keno]

This is somewhat stale, but @vtjnash has done some work on more precise coverage recently, which I think had roughly the same effect.

[vtjnash]

This is a significantly different piece of design from my recent work. Though I agree we probably can’t use the PR in its current state.