Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[5.4] Fix 'Too Many Connections' error during testing using $connectionsToTransact #20340

Merged
merged 1 commit into from
Aug 2, 2017
Merged

[5.4] Fix 'Too Many Connections' error during testing using $connectionsToTransact #20340

merged 1 commit into from
Aug 2, 2017

Conversation

twig1337
Copy link
Contributor

@twig1337 twig1337 commented Jul 31, 2017

Terminate user defined database connections after rollback while using the DatabaseTransactions trait.

Note: This bug fix is pointed to the 5.4 branch as the corresponding code doesn't exist in the 5.1 release.

…ting while using the DatabaseTransactions trait.
@thecrypticace
Copy link
Contributor

I run into this one all the time. The point where I created a trait that does this (but does so by overriding tearDown by keeping temp variables around to be able to call setPDO(null) on it afterwards).

Never noticed connection had a disconnect method. I quite like this approach. How well does it work when used with the DatabaseMigrations trait?

@tillkruss tillkruss changed the title Fix 'Too Many Connections' error during testing using $connectionsToTransact [5.4] Fix 'Too Many Connections' error during testing using $connectionsToTransact Aug 1, 2017
@taylorotwell
Copy link
Member

What causes the too many connections? Obviously this doesn't happen on every test suite. Is adding this going to seriously affect performance of the tests?

@themsaid
Copy link
Member

themsaid commented Aug 1, 2017

Maybe related: #18471

@thecrypticace
Copy link
Contributor

What causes the too many connections?

I think something is holding onto the connection object. Maybe there's a reference cycle that's not being cleaned up?
Thinking out loud: I wonder if running gc_collect_cycles() would fix anything…

… Obviously this doesn't happen on every test suite

Most likely because:

  1. Most test suites use SQLite for testing instead of MySQL
  2. The default concurrent connection count for MySQL is 300 connections
  3. I'd guess its unlikely that there'd be 300+ test cases that all use the database for at least smallish apps.

If you have an application that connects to more than one database (I have one that uses 4 for instance) it makes this problem pop up much, much quicker.

@twig1337
Copy link
Contributor Author

twig1337 commented Aug 1, 2017

In my case I am using MySql for the testing.
As @thecrypticace suggested my 'max_connections' value is 151.
I have 5 database connections that are created on every request to my API.
I have a suite of 36 tests.
36 * 5 = 180 = 💥

I'm not sure where the default 'mysql' connection is terminated, but upon testing it appears that it is terminated after each test is ran, preventing this sort of issue from occurring if a user is not using the $connectionsToTransact variable. But if you are none of the connections are terminated until after the entire test suite is completed.

@twig1337
Copy link
Contributor Author

twig1337 commented Aug 1, 2017

@taylorotwell If I increase my max_connections value and run my test suite without my changes the elapsed time, according to PHPUnit, is 33 seconds.
If I run my test suite with my changes it is 31 seconds.

So these changes actually increased performance in my scenario, which makes sense because my database isn't trying to maintain hundreds of connections simultaneously.

@twig1337
Copy link
Contributor Author

twig1337 commented Aug 1, 2017

I would also think that any proponent of TDD would say that my suite of 36 tests is child's play, not suitable as a true testing solution. In fact the tests in question are only my acceptance tests, this issue would be greatly exacerbated if I include integration tests across components that reference multiple database connections.

@thecrypticace
Copy link
Contributor

thecrypticace commented Aug 1, 2017

Edit: never mind using $app->flush() like the base test class does solves the issue with the code below. Back to investigating


I did a bit of testing. I can reproduce the error with this script:

require __DIR__.'/bootstrap/autoload.php';

for ($i = 0; $i < 300; $i++) {
    $app = require __DIR__.'/bootstrap/app.php';
    $app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();
    $app["db"]->connection(null)->beginTransaction();
    $app["db"]->connection(null)->rollBack();
    $app = null;

    echo ".";
}

And it appears that gc_collect_cycles(); does not help.

@taylorotwell taylorotwell merged commit 17176b7 into laravel:5.4 Aug 2, 2017
@twig1337 twig1337 deleted the terminate-testing-connections branch August 2, 2017 14:15
@mfn
Copy link
Contributor

mfn commented Aug 3, 2017

Many thanks for the PR! Can also confirm it works.

I feel a bit bad because I was using a custom trait with exactly this changes for years but failed to provide an upstream PR :-/

@denitsa-md
Copy link

I am still having this problem in Laravel 5.6.8 with the $connectionsToTransact property in a base test case (with 2 connections) and a mysql database.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants