Coveralls integration

[greg0ire]

Q	A
Type	improvement
BC Break	no

Todo

• Travis
• Remove Scrutinizer
• Update badges in the README

[greg0ire]

One thing left to figure out: how do we get the coveralls bar in the Github UI?

[morozov]

One thing left to figure out: how do we get the coveralls bar in the Github UI?

You mean the badge in README? You already replaced it.

[greg0ire]

One thing left to figure out: how do we get the coveralls bar in the Github UI?

No I mean this:

Screenshot from Sonata BTW, I'm trying to understand how this works.

[morozov]

Honestly, I think it's a very annoying and useless thing. Often, if you remove fully-covered code (e.g. by replacing 10 lines with 3 lines that do the same), your average coverage drops. And this is very confusing/demotivating.

[greg0ire]

I see we can set a theshold in the setting to avoid that kind of disheartening failure. That way we catch the big variations.

[morozov]

I'd rather like to have diff coverage to be automatically reported: the percentage of lines affected by the diff that are covered with tests. Not sure if it's supported.

[greg0ire]

Ah it's probably just a matter of clicking the green buttons here: https://github.com/marketplace/coveralls

I don't think what you want is supported, so let's just leave things as is.

[morozov]

@greg0ire after having filed a couple of PRs and not being able to see the code coverage, I think we should try to enable it. I do not want the coverage delta to be automatically reported on the PR but I want to be able to see it if needed. Do you think we could enable it?

[greg0ire]

Sure, let's try.

EDIT: I think that app is in fact already enabled for us, so there must be some other issue.

[greg0ire]

@morozov the coverage seems to be accessible, in fact: https://coveralls.io/jobs/62814748
I think it's just an issue with the UI. I found this link in the output of the Travis job that does coverage.

[greg0ire]

Found the URL for all the DBAL builds: https://coveralls.io/repos/6217/builds

[morozov]

Thanks, @greg0ire. From that list, I can navigate to the build for which I'm trying to see the coverage change but having it available as build status would be more convenient.

It looks like I cannot install the Coveralls app from the market because I'm not an organization owner.

[morozov]

BTW, not to spoil the fun, the more I look at Codecov, the more I think we should have done a better job comparing the services. E.g. Codecov seems to have nicer code coverage change reporting as a summary as well as provides all the lower-level details as Coveralls does.

At the same time, they charge their customers on the per-user basis which indirectly means they have more manpower to maintain the service. Additionally, they may not have the problem integrating with Appveyor.

[greg0ire]

Well it's still time to change, only the DBAL has Coveralls for now.