Wrong num tests when using group filter

[jrbasso]

When running the phpunit with --group param it do not affect the number of tests in the output, affecting the percentage too.

Ie,

$ phpunit --group shop
PHPUnit 3.7.1 by Sebastian Bergmann.

Configuration read from **************/phpunit.xml

...............................................................  63 / 672 (  9%)
............................................................... 126 / 672 ( 18%)
..................

Time: 02:08, Memory: 28.00Mb

$ phpunit
PHPUnit 3.7.1 by Sebastian Bergmann.

Configuration read from **************/phpunit.xml

...............................................................  63 / 672 (  9%)
............................................................... 126 / 672 ( 18%)
............................................................... 189 / 672 ( 28%)
............................................................... 252 / 672 ( 37%)
............................................................... 315 / 672 ( 46%)
............................................................... 378 / 672 ( 56%)
............................................................... 441 / 672 ( 65%)
............................................................... 504 / 672 ( 75%)
............................................................... 567 / 672 ( 84%)
............................................................... 630 / 672 ( 93%)
..........................................

Time: 03:34, Memory: 41.00Mb

Note that in the first block the number of tests are the same of the tests without filter. The correct value in this case should be ?? / 144.

[jrbasso]

Looking into the code to send a pull request I see the PHPUnit_TextUI_ResultPrinter::startTestSuite() use count($suite), but the groups to be filtered are sent as parameter in the PHPUnit_Framework_TestSuite::run(). It means that the count() method in the test suite are not able to filter it.

It could be possible adding an attribute with the groups to filter, but the count() lose the goal, which is inform the number of tests from the suite, not the current running state (let me know if I am wrong saying that).

So, what should be the approach to fix it? Change the count() behavior, or change ResultPrinter to use a different method (ie, call $suite->numRunningTests(), which will call $this->count() or custom in case of group filters)?

[sebastianbergmann]

I am afraid there is not clean way to do this before #10

[edorian]

See #358 and #541

[edorian]

Fixed with merging #781

[zerkms]

It's still reproducible in phpunit 3.7.21

[whatthejeff]

Thanks, @zerkms. I just tested and can confirm this issue still exists.

[whatthejeff]

@zerkms The fix for this is in the 3.8 release as it required public API changes.