Behat, foundry and manager registry of the app on different database connection instances

[much-rebel]

I'm not sure why or how this happens, but it seems that foundry somehow gets different connection instance. I found this while trying to make foundry work with my behat setup, which I'm trying to make to work in database transaction mode, as in my case wiping data from database is not the option (for system to work some huge data package needs to be in the database and importing 0.5gb data on every scenario would make execution time too long).

Some simplified examples how to reproduce:

Feature: Reproduce the bug
  
  Background:
    Given system defaults are loaded
    
  Scenario: Fail
    When mock step

<?php

use Behat\Behat\Context\Context;
use Doctrine\Common\Persistence\ManagerRegistry;

class FeatureContext implements Context
{
    private ManagerRegistry $managerRegistry;

    public function __construct(ManagerRegistry $managerRegistry)
    {
        $this->managerRegistry = $managerRegistry;
    }

    /**
     * @Given /^mock step$/
     */
    public function mockStep()
    {
        return;
    }

    /**
     * @Given /^system defaults are loaded$/
     */
    public function systemDefaultsAreLoaded()
    {
        $manager = $this->managerRegistry->getManager();
        
        /** @var \Doctrine\DBAL\Connection $conn */
        $conn = $manager->getConnection();
        $conn->beginTransaction();

        $foo = \App\Tests\Factory\FooFactory::new()->create();

        $conn->rollback(); // Has no effect, data is already written and commited into database
    }
}

Or:

<?php

use Behat\Behat\Context\Context;
use Doctrine\Common\Persistence\ManagerRegistry;

class FeatureContext implements Context
{
    private ManagerRegistry $managerRegistry;

    public function __construct(ManagerRegistry $managerRegistry)
    {
        $this->managerRegistry = $managerRegistry;
    }

    /**
     * @Given /^mock step$/
     */
    public function mockStep()
    {
        return;
    }

    /**
     * @Given /^system defaults are loaded$/
     */
    public function systemDefaultsAreLoaded()
    {
        $manager = $this->managerRegistry->getManager();
        
        /** @var \Doctrine\DBAL\Connection $conn */
        $conn = $manager->getConnection();
        $conn->beginTransaction();

        $foo = \App\Tests\Factory\FooFactory::new()->withoutPersisting()->create();
        
        $manager->persist($foo->object());
        $manager->flush();
        
        $foosInFoundry = \App\Tests\Factory\FooFactory::repository()->findAll(); // Result - []
        $foosInManager = $manager->getRepository(Foo::class)->findAll(); // Result - [Object Foo::class]
        
        $manager->rollback(); // Rollbacks the Foo insert, which can't be found by repository() method of foundry
    }
}

The suggested lib to have database resets - dama/doctrine-test-bundle would fail in the same fashion and using it would just result in data written into database.

Unfortunately I'm not sure why this happens so can't make MR with solution proposed.

Some additional info:

php: 7.4
mysql: 5.7
symfony: 4.4.36
foundry: 1.15.0

[kbond]

I believe foundry is being booted by the bundle earlier (with a different kernel).

For testing, foundry is optimized for PHPUnit. This trait for your TestCase solves this issue. Perhaps we can come up with a foundry-specific behat context to do the same?

@wouterj, do you use foundry with behat? If so, do you have any insights?

[much-rebel]

@kbond well that makes a lot of sense, though I don't get the part how bundle is getting registered with another kernel different from what is in behat contexts.

Anyway, did some testing and seems the issue is resolved by manually booting foundry... So what I did, I've turned off the bundle and added extra method in the FeatureContext class:

<?php
...
class FeatureContext implements Context
{
    ...

    /**
     * @BeforeScenario
     */
    public function bootFoundry()
    {
        if (Factory::isBooted()) {
            return;
        }

        $configuration = new Configuration();

        try {
            $configuration->setManagerRegistry($this->managerRegistry);
        } catch (NotFoundExceptionInterface $e) {
            throw new \LogicException('Could not boot Foundry, is the DoctrineBundle installed/configured?', 0, $e);
        }

        TestState::bootFoundry($configuration);
    }

And it works!

[kbond]

Glad to hear! You may want to add an "after scenario" hook to shutdown foundry. BTW, you can use foundry's TestState to modify the configuration without the bundle.

I'm thinking we should include this context within foundry itself. This would remove the need for you to use @internal functionality (this could break between releases).

though I don't get the part how bundle is getting registered with another kernel different from what is in behat contexts.

What gives the nice DX of foundry is a lot of static state. This does create a lot of complexity and weird edge cases. Do you have some "before suite" hooks that boots the kernel then shuts it down? This would cause foundry to be booted with a different kernel instance.

[much-rebel]

Well it comes with a price... All the magic of using factories/stories as services and ect. from the extension is unavailable this way, so it is not 100% fix for the situation. Trying to figure out the working configuration rn.

Regards BeforeSuite or any other extra kernel boots - no, I've just introduced behat to this project and it's configuration is in a very early stage.

[kbond]

All the magic of using factories/stories as services and ect. from the extension is unavailable this way,

That's true. What happens when you just force boot foundry in your before scenario hook? This is what I do in the PHPUnit trait I linked above. Something like:

use Zenstruck\Foundry\Test\TestState;

class FeatureContext implements Context
{
    ...

    /**
     * @BeforeScenario
     */
    public function bootFoundry()
    {
        TestState::bootFromContainer($this->container); // inject the container into this context
    }

    /**
     * @AfterScenario
     */
    public function shutdownFoundry()
    {
        TestState::shutdownFoundry();
    }
}

[much-rebel]

That works!

Unfortunately it seems behat mink is booting another kernel for resolving endpoints in Contexts which gets another connection instance and then nothing in my Context's transaction is available for endpoint. It seems the whole concept might be not very possible in behat anyways :(

[kbond]

You are using the SymfonyExtension I assume? Could FriendsOfBehat/SymfonyExtension#149 be related? I don't have much knowledge of the behat ecosystem but what you described reminded me of that issue.

[kbond]

For the transactions, are you using dama/doctrine-test-bundle? I thought it solves this problem with a StaticDriver.

[much-rebel]

That is brilliant! I did have dama bundle disabled (was handling transaction begin and rollback manually), but have it re-enabled now and it does the job combined with TestState::bootFromContainer($this->container);!

Thanks a lot!

[much-rebel]

Not sure what to do next with the issue though. I think using behat with DB transactions is quite an edge case as the ORM Purger is the common approach. Adding hook context would be helpful, but it would make sense only in use together dama bundle. Might be tricky to explain the case in the documentation 😅

[kbond]

The ResetDatabase trait has logic to detect if using Dama and changes tactics accordingly. Hoping we could do something similar for Behat integration.

[wouterj]

Hi! Yes, we are using Behat + FriendsOfBehat SymfonyExtension + Foundry in our tests. We are indeed using Factory::boot($configuration); to connect the correct container to Foundry. For your information, the SymfonyExtension uses a different driver container (which is used by the app) and the contexts, this is causing Foundry to get booted using the driver container (through the app), instead of the context container.

It would be cool to write a Behat extension that does this correctly. I believe we still have some weird edge cases (not sure if it's due to Foundry or something else). I personally have the feeling it would be better to not register the FoundryBundle when you're using Behat, and instead set-up Foundry using a BehatExtension.

[kbond]

Thanks for the insights @wouterj. I'm closing this issue to continue discussion in #235.