Annotations to opt-out per test class and test method

[B-Galati]

Hello !

What do you think about an annotation that can disable the begin transaction/rollback of the phpunit extension ?

At the moment I am running a lot of test that don't require any database query or are just read-only but the extension is still triggered.

Cheers!

[dmaicher]

So out of interest I just benchmarked this.

The test case:

use PHPUnit\Framework\TestCase;

class FooTest extends TestCase
{
    /**
     * @dataProvider dataProvider
     */
    public function testNothing(int $foo): void
    {
    }

    public function dataProvider(): \Generator
    {
        foreach (range(0, 100000) as $i) {
            yield [$i];
        }
    }
}

With the extension (and roll-back) of this bundle enabled:

.......................                                   100001 / 100001 (100%)

Time: 00:07.450, Memory: 318.52 MB

And now without the extension (and thus roll-back) enabled:

.......................                                   100001 / 100001 (100%)

Time: 00:06.439, Memory: 318.52 MB

So wondering: is it worth it? For 100k tests this takes 1s on my laptop (using dockerized mysql + SSD). Maybe you could also benchmark this?

[dmaicher]

Actually the test above was not valid since no DB connection was ever created. Will have a look at a proper benchmark later today

[dmaicher]

Ok here a proper benchmark now with 2 tests where the first one actually creates a DB connection and makes sure the roll-back logic kicks in:

namespace Tests\Foo;

use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;

class BarTest extends KernelTestCase
{
    public function testBar(): void
    {
        self::bootKernel()->getContainer()->get('doctrine')->getConnection()->executeQuery('SELECT 1');
    }
}

namespace Tests\Foo;

use PHPUnit\Framework\TestCase;

class FooTest extends TestCase
{
    /**
     * @dataProvider dataProvider
     */
    public function testNothing(int $foo): void
    {
    }

    public function dataProvider(): \Generator
    {
        foreach (range(0, 100000) as $i) {
            yield [$i];
        }
    }
}

Now it looks a bit different:

With extension: Time: 00:23.036, Memory: 353.52 MB
Without extension: Time: 00:11.041, Memory: 353.52 MB

So the performance penalty for 100k tests is roughly 12s for me.

[B-Galati]

Thank you so much for these!

Indeed that does not look to worth the effort. I will do some benchmarks on my side too 👍

[B-Galati]

I did the same benchmark and I have 40 to 45 seconds of penalty for 100k tests.

Don't know why we have so much difference.

Anyway, it still does not worth the effort to my eyes.

Feel free to close the issue or not 👍

[dmaicher]

Ok thanks 👍 I'm closing this for now as I also don't think its worth the effort for now.