Surprising behavior when running TestTest.php

[taenfu]

On PHPUnit 3.6.10, PHP 5.3.5, running a testcase named TestTest.php and containing 1 false assertion leads to this error :

phpunit unit\TestTest.php

PHP Fatal error: Uncaught exception 'PHPUnit_Framework_ExpectationFailedException' with message 'KO1
Failed asserting that 2 matches expected 3.' in C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Constraint\IsEqual.php:171
Stack trace:
#0 C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Assert.php(2100): PHPUnit_Framework_Constraint_IsEqual->evaluate(2, 'KO1')
#1 C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Assert.php(441): PHPUnit_Framework_Assert::assertThat(2, Object(PHPUnit_Framework_Constraint_IsEqual), 'KO1')
#2 C:\Apps\Wampee-2.1-beta-2\www\consultAssur\protected\tests\unit\TestTest.php(7): PHPUnit_Framework_Assert::assertEquals(3, 2, 'KO1')
#3 C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\TestSuite.php(476): TestTest->testTest()
#4 C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\TestSuite.php(831): PHPUnit_Framework_TestSuite::crea in C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Constraint\IsEqual.php on line 171

Fatal error: Uncaught exception 'PHPUnit_Framework_ExpectationFailedException' with message ' in C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Constraint\IsEqual.php on line 171

PHPUnit_Framework_ExpectationFailedException: KO1
Failed asserting that 2 matches expected 3. in C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Constraint\IsEqual.php on line 171

Call Stack:
0.0006 329432 1. {main}() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\phpunit.php:0
0.0061 745912 2. PHPUnit_TextUI_Command::main() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\phpunit.php:48
0.0061 746328 3. PHPUnit_TextUI_Command->run() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\TextUI\Command.php:130
0.0582 2914120 4. PHPUnit_Runner_BaseTestRunner->getTest() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\TextUI\Command.php:150
0.0630 3286696 5. PHPUnit_Framework_TestSuite->__construct() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Runner\BaseTestRunner.php:142
0.0632 3375736 6. PHPUnit_Framework_TestSuite->addTestMethod() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\TestSuite.php:214
0.0633 3375792 7. PHPUnit_Framework_TestSuite::createTest() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\TestSuite.php:831
0.0645 3501264 8. TestTest->testTest() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\TestSuite.php:476
0.0732 3758608 9. PHPUnit_Framework_Assert::assertEquals() C:\Apps\Wampee-2.1-beta-2\www\consultAssur\protected\tests\unit\TestTest.php:7
0.0732 3759240 10. PHPUnit_Framework_Assert::assertThat() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Assert.php:441
0.0732 3759240 11. PHPUnit_Framework_Constraint_IsEqual->evaluate() C:\Apps\Wampee-2.1-beta-2\bin\php\php5.3.5\includes\PHPUnit-3.6.10\PHPUnit\Framework\Assert.php:2100

The (simplest) run testcase is :

TestTest.php

<?php
class TestTest extends PHPUnit_Framework_TestCase
{
    public function testTest()
    {
        $this->assertEquals(4,2+2,'OK1');
        $this->assertEquals(3,1+1,'KO1');
        $this->assertEquals(5,2+3,'OK2');
    }
}
?>

The fun is :

• if the testcase is all ok (without any failed assertion), all run as expected
• if the testcase filename and the testcase class are renamed to TinyTest, for example, all run as expected too !

I agree that it's a tyny subject but which can block new users, asking for themselves if their setup is good ...

Cordially,
Jean-Michel

[edorian]

The test case is names like the class so it acts as the classes __construct (PHP 4 Style Constructor).

So the code is not executed as part of a test case and the exception is not caught.

Maybe we should catch all exceptions thrown during test case creation and report them nicer @sebastianbergmann ?

Anyways thanks for the report

[sebastianbergmann]

That could be done, yes. But I do not really want to add code to workaround PHP 4 constructor edge case weirdnesses like this :)

[edorian]

I'd rather have a generic "we couldn't construct your test case.. wtf are you doing there" type of warning :)

[sebastianbergmann]

Sounds good. But: that will not help you with catching errors such as not calling PHPUnit_Framework_TestCase::__construct() in your constructor and passing along all original arguments. Custom constructors should not be required in normal test case classes. Having them has a high potential for breaking things.