Issue with transactions

[toooni]

Hi everyone

I am in the progress of migrating our (travel/booking) application to FrankenPHP. We deploy to kubernetes (AWS EKS) for a pretty big application. I've had a few spots where I was using local vars to temporary store doctrine entities. After fixing these, it seems that the app runs pretty well (on our staging environment). Except for one issue which prevents us from going prod:

• User processes to checkout
• User enters their personal information and saves the page
• The save endpoint in our php app creates a "booking" and returns the id.
• The frontend receives this id and redirects to the payment page which makes to async calls to read.json which returns everything related to the booking and info.json which returns more informations about the product and so on.
• The backend in both endpoints call this function at one point:

public function getExistingBooking(Request $request): ?Booking
    {
        $bookingId = $request->query->getInt('bookingId', $request->attributes->getInt('bookingId'));

        if ($bookingId === 0) {
            return null;
        }

        $booking = $this->bookingRepository->find($bookingId); // this is the default doctrine repository for the booking entity

        if ($booking === null) {
            throw new AccessDeniedException(); // regularly, one of the both requests end up here !?!?!
        }

        return $booking;
    }

Regularly one of these two requests return the AccessDeniedException (the logs show exactly that this line is the origin). We never had an issue with this exact process when using apache/php-fpm. Does anybody have an idea how FrankenPHP could affect this?

[dunglas]

Hi, are you using the worker mode?

[toooni]

Hi @dunglas, yes, I do. And I also just learned something new. The save request saves the booking, the info request works as intended and then the read one fails. And the funny thing is - there is no entry with this id in the database after. I do not work with transactions there.

[withinboredom]

Doctrine is "fun" in worker mode. /s

Are you forgetting a UnitOfWork->flush() after your save, or maybe it is failing for some reason? Perhaps you have a shutdown hook that does this (FrankenPHP doesn't support shutdown hooks)?

[toooni]

Unfortunately. This is a very simple save request. It does convert the json to an object, does some validation and then does a flush() direclty after persist() 😒

Edit: It doesn't seem to fail. The entity uses auto_increment and the returned id is correct.
MySQL even has 43761 as the next id. For some reason the booking seems to be deleted (after the next request).

[toooni]

Another update: I've set the monolog log level to debug, so I get the chance to see what could be happening. Usually I've had to try a few times to reproduce the error. With debug I am not able to reproduce the issue at all. This leads me to believe that it might be some kind of race condition (this is also indicated by the fact that it doesn't happen every single time I try to reproduce the issue).

[toooni]

@dunglas I think I've got it. We've got a partner which uses our staging environment to test their staging environment. Unfortunately or in this case fortunately, they send us a pretty big request, every 10 minutes which runs for ~1minute. While processing this request we are using transactions.

So it seems that if we have another request running with transactions, this completely unrelated request which saves to the database (without using transactions) fails, or probably this insert will land in the transaction of the other request. Is this possible?

I've searched for issues of frankenphp with transactions but couldn't find any. Maybe this is an edge case because the request is running for this long? Maybe in our case we were able to catch it because of this long running transaction. In other cases the issue might be much more subtle, causing an issue even harder to reproduce or catch.

[withinboredom]

Are you using pdo_mysql or mysqli?

[toooni]

I am using pdo (doctrine).

[withinboredom]

🤦 You're using doctrine, so I'm not sure why I asked haha.

I'm using postgres and I haven't seen this behavior, but we are always using transactions... This seems easy enough to test:

1. create an endpoint that starts a transaction sleeps for a minute, then commits it.
2. create another endpoint that simply inserts some data, without a transaction.

You should see the data immediately if nothing funky is going on.

Something I've been known to do in production, is go directly into the vendor of doctrine (We use postgres, so actual folders won't match for you) and drop this little nugget in a function of the driver:

file_put_contents('/tmp/log.txt', "Your log message\n", FILE_APPEND);

Then tailing the log to diagnose the issue. It's not uncommon for me to have the vendor directory littered with these on a container, which I then delete when I'm done.

We used to flush after the request was replied to, and when it failed, we weren't getting any logs that things were going wrong. We just had bizarre behavior because the request replied that everything went well to the client, but actually, nothing happened. Having specific logs coming from the driver itself helped us greatly.

[toooni]

Hi @withinboredom
I am pretty confident that the unrelated request with the transactions was the issue. But there were ~2400 transactions created and rolled back in this request (Complicated story, but only happening on our staging env, caused by an external partners staging environment).
I will now try to recreate the issue in a demo app.

Thanks for your help!