Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate db tool and db backup/restore services for version 2.x #398

Merged
merged 16 commits into from
Mar 6, 2018
21 changes: 16 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -316,7 +316,18 @@ Tips for Fixture Loading Tests
cache_sqlite_db: true
```

3. Load your Doctrine fixtures in your tests:
3. For create custom database cache service

**Attention: Don't work with `liip_functional_test.cache_sqlite_db` use only `liip_functional_test.cache_db.*`.**
```yaml
# app/config/config_test.yml
liip_functional_test:
cache_db:
sqlite: liip_functional_test.database_backup.sqllite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sqlite ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's typo

mysql: liip_functional_test.services_database_backup.mysql_custom
```

4. Load your Doctrine fixtures in your tests:

```php
use Liip\FunctionalTestBundle\Test\WebTestCase;
@@ -339,7 +350,7 @@ Tips for Fixture Loading Tests
}
```

4. If you don't need any fixtures to be loaded and just want to start off with
5. If you don't need any fixtures to be loaded and just want to start off with
an empty database (initialized with your schema), you can simply call
`loadFixtures` without any argument.

@@ -359,7 +370,7 @@ Tips for Fixture Loading Tests
}
```

5. Given that you want to exclude some of your doctrine tables from being purged
6. Given that you want to exclude some of your doctrine tables from being purged
when loading the fixtures, you can do so by passing an array of tablenames
to the `setExcludedDoctrineTables` method before loading the fixtures.

@@ -379,7 +390,7 @@ Tips for Fixture Loading Tests
}
```

6. If you want to append fixtures instead of clean database and load them, you have
7. If you want to append fixtures instead of clean database and load them, you have
to consider use the second parameter $append with value true.

```php
@@ -398,7 +409,7 @@ Tips for Fixture Loading Tests
}
```

7. This bundle uses Doctrine ORM by default. If you are using another driver just
8. This bundle uses Doctrine ORM by default. If you are using another driver just
specify the service id of the registry manager:

```php
9 changes: 9 additions & 0 deletions src/DependencyInjection/Configuration.php
Original file line number Diff line number Diff line change
@@ -31,6 +31,15 @@ public function getConfigTreeBuilder(): TreeBuilder
$rootNode
->children()
->booleanNode('cache_sqlite_db')->defaultFalse()->end()
->arrayNode('cache_db')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we then remove cache_sqlite_db ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'll remove it.
#398 (comment)

->addDefaultsIfNotSet()
->ignoreExtraKeys(false)
->children()
->scalarNode('sqlite')
->defaultNull()
->end()
->end()
->end()
->scalarNode('command_verbosity')->defaultValue('normal')->end()
->booleanNode('command_decoration')->defaultTrue()->end()
->arrayNode('query')
1 change: 1 addition & 0 deletions src/DependencyInjection/LiipFunctionalTestExtension.php
Original file line number Diff line number Diff line change
@@ -33,6 +33,7 @@ public function load(array $configs, ContainerBuilder $container): void
$loader = new XmlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
$loader->load('commands.xml');
$loader->load('functional_test.xml');
$loader->load('database_tools.xml');
if (interface_exists('Symfony\Component\Validator\Validator\ValidatorInterface')) {
$loader->load('validator.xml');
}
55 changes: 55 additions & 0 deletions src/Resources/config/database_tools.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
<?xml version="1.0" ?>

<container xmlns="http://symfony.com/schema/dic/services"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://symfony.com/schema/dic/services http://symfony.com/schema/dic/services/services-1.0.xsd">

<services>
<service id="liip_functional_test.services.fixtures_loader_factory" class="Liip\FunctionalTestBundle\Services\FixturesLoaderFactory" public="true">
<argument type="service" id="service_container" />
</service>

<service id="liip_functional_test.services_database_backup.sqlite" class="Liip\FunctionalTestBundle\Services\DatabaseBackup\SqliteDatabaseBackup" public="true">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
</service>

<service id="liip_functional_test.services_database_backup.mysql_custom" class="Liip\FunctionalTestBundle\Services\DatabaseBackup\MysqlCustomDatabaseBackup" public="true">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
</service>

<service id="liip_functional_test.services_database_tools.ormdatabase_tool" class="Liip\FunctionalTestBundle\Services\DatabaseTools\ORMDatabaseTool" public="false">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
</service>
<service id="liip_functional_test.services_database_tools.ormsqllite_database_tool" class="Liip\FunctionalTestBundle\Services\DatabaseTools\ORMSqlliteDatabaseTool" public="false">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
</service>
<service id="liip_functional_test.services_database_tools.mongo_dbdatabase_tool" class="Liip\FunctionalTestBundle\Services\DatabaseTools\MongoDBDatabaseTool" public="false">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
</service>
<service id="liip_functional_test.services_database_tools.phpcrdatabase_tool" class="Liip\FunctionalTestBundle\Services\DatabaseTools\PHPCRDatabaseTool" public="false">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
</service>
<service id="liip_functional_test.services.database_tool_collection" class="Liip\FunctionalTestBundle\Services\DatabaseToolCollection" public="true">
<argument type="service" id="service_container" />
<argument type="service" id="liip_functional_test.services.fixtures_loader_factory" />
<call method="add">
<argument type="service" id="liip_functional_test.services_database_tools.ormdatabase_tool" />
</call>
<call method="add">
<argument type="service" id="liip_functional_test.services_database_tools.ormsqllite_database_tool" />
</call>
<call method="add">
<argument type="service" id="liip_functional_test.services_database_tools.mongo_dbdatabase_tool" />
</call>
<call method="add">
<argument type="service" id="liip_functional_test.services_database_tools.phpcrdatabase_tool" />
</call>
</service>
</services>
</container>
116 changes: 116 additions & 0 deletions src/Services/DatabaseBackup/AbstractDatabaseBackup.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
<?php

/*
* This file is part of the Liip/FunctionalTestBundle
*
* (c) Lukas Kahwe Smith <smith@pooteeweet.org>
*
* This source file is subject to the MIT license that is bundled
* with this source code in the file LICENSE.
*/

namespace Liip\FunctionalTestBundle\Services\DatabaseBackup;

use Doctrine\Common\DataFixtures\Executor\AbstractExecutor;
use Doctrine\DBAL\Connection;
use Liip\FunctionalTestBundle\Services\FixturesLoaderFactory;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
* @author Aleksey Tupichenkov <alekseytupichenkov@gmail.com>
*/
abstract class AbstractDatabaseBackup
{
protected $container;

protected $fixturesLoaderFactory;

/**
* @var Connection
*/
protected $connection;

/**
* @var array
*/
protected $metadatas;

/**
* The fixture classnames.
*
* @var array
*/
protected $classNames;

public function __construct(ContainerInterface $container, FixturesLoaderFactory $fixturesLoaderFactory)
{
$this->container = $container;
$this->fixturesLoaderFactory = $fixturesLoaderFactory;
}

public function init(Connection $connection, array $metadatas, array $classNames): void
{
$this->connection = $connection;
$this->metadatas = $metadatas;
$this->classNames = $classNames;
}

abstract public function getBackupName(): string;

abstract public function isBackupActual(): bool;

abstract public function backup(AbstractExecutor $executor): void;

abstract public function restore(AbstractExecutor $executor): void;

/**
* Determine if the Fixtures that define a database backup have been
* modified since the backup was made.
*
* @param string $backup The fixture backup SQLite database file path
*
* @return bool TRUE if the backup was made since the modifications to the
* fixtures; FALSE otherwise
*/
protected function isBackupUpToDate(string $backup): bool
{
$backupLastModifiedDateTime = \DateTime::createFromFormat('U', filemtime($backup));

$loader = $this->fixturesLoaderFactory->getFixtureLoader($this->classNames);

// Use loader in order to fetch all the dependencies fixtures.
foreach ($loader->getFixtures() as $className) {
$fixtureLastModifiedDateTime = $this->getFixtureLastModified($className);
if ($backupLastModifiedDateTime < $fixtureLastModifiedDateTime) {
return false;
}
}

return true;
}

/**
* This function finds the time when the data blocks of a class definition
* file were being written to, that is, the time when the content of the
* file was changed.
*
* @param string $class The fully qualified class name of the fixture class to
* check modification date on
*
* @return \DateTime|null
*/
protected function getFixtureLastModified($class): ?\DateTime
{
$lastModifiedDateTime = null;

$reflClass = new \ReflectionClass($class);
$classFileName = $reflClass->getFileName();

if (file_exists($classFileName)) {
$lastModifiedDateTime = new \DateTime();
$lastModifiedDateTime->setTimestamp(filemtime($classFileName));
}

return $lastModifiedDateTime;
}
}
73 changes: 73 additions & 0 deletions src/Services/DatabaseBackup/MysqlCustomDatabaseBackup.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<?php

/*
* This file is part of the Liip/FunctionalTestBundle
*
* (c) Lukas Kahwe Smith <smith@pooteeweet.org>
*
* This source file is subject to the MIT license that is bundled
* with this source code in the file LICENSE.
*/

namespace Liip\FunctionalTestBundle\Services\DatabaseBackup;

use Doctrine\Common\DataFixtures\Executor\AbstractExecutor;

/**
* @author Aleksey Tupichenkov <alekseytupichenkov@gmail.com>
*
* It's class created just for example that how to create database backup/restore service
*/
class MysqlCustomDatabaseBackup extends AbstractDatabaseBackup
{
public function getBackupName(): string
{
return $this->container->getParameter('kernel.cache_dir').'/test_mysql_'.md5(serialize($this->metadatas).serialize($this->classNames)).'.sql';
}

public function isBackupActual(): bool
{
$backupDBFileName = $this->getBackupName();
$backupReferenceFileName = $backupDBFileName.'.ser';

return file_exists($backupDBFileName) && file_exists($backupReferenceFileName) && $this->isBackupUpToDate($backupDBFileName);
}

public function backup(AbstractExecutor $executor): void
{
$params = $this->connection->getParams();
if (isset($params['master'])) {
$params = $params['master'];
}

$dbName = isset($params['dbname']) ? $params['dbname'] : '';
$dbHost = isset($params['host']) ? $params['host'] : '';
$dbPort = isset($params['port']) ? $params['port'] : '';
$dbUser = isset($params['user']) ? $params['user'] : '';
$dbPass = isset($params['password']) ? $params['password'] : '';

$executor->getReferenceRepository()->save($this->getBackupName());

exec("mysqldump -h $dbHost -u $dbUser -p$dbPass $dbName > {$this->getBackupName()}");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the MySQL database is in a Docker container, this command won't work?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use docker too, it will works fine if you install mysql-client in container with symfony app :)

}

public function restore(AbstractExecutor $executor): void
{
$params = $this->connection->getParams();
if (isset($params['master'])) {
$params = $params['master'];
}

$dbName = isset($params['dbname']) ? $params['dbname'] : '';
$dbHost = isset($params['host']) ? $params['host'] : '';
$dbPort = isset($params['port']) ? $params['port'] : '';
$dbUser = isset($params['user']) ? $params['user'] : '';
$dbPass = isset($params['password']) ? $params['password'] : '';

exec("mysqladmin -h $dbHost -u $dbUser -p$dbPass drop $dbName -f");
exec("mysqladmin -h $dbHost -u $dbUser -p$dbPass create $dbName");
exec("mysql -h $dbHost -u $dbUser -p$dbPass $dbName < {$this->getBackupName()}");

$executor->getReferenceRepository()->load($this->getBackupName());
}
}
60 changes: 60 additions & 0 deletions src/Services/DatabaseBackup/SqliteDatabaseBackup.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
<?php

/*
* This file is part of the Liip/FunctionalTestBundle
*
* (c) Lukas Kahwe Smith <smith@pooteeweet.org>
*
* This source file is subject to the MIT license that is bundled
* with this source code in the file LICENSE.
*/

namespace Liip\FunctionalTestBundle\Services\DatabaseBackup;

use Doctrine\Common\DataFixtures\Executor\AbstractExecutor;

/**
* @author Aleksey Tupichenkov <alekseytupichenkov@gmail.com>
*/
class SqliteDatabaseBackup extends AbstractDatabaseBackup
{
public function getBackupName(): string
{
return $this->container->getParameter('kernel.cache_dir').'/test_sqllite_'.md5(serialize($this->metadatas).serialize($this->classNames)).'.db';
}

private function getDatabaseName(): string
{
$params = $this->connection->getParams();
if (isset($params['master'])) {
$params = $params['master'];
}

$name = $params['path'] ?? ($params['dbname'] ?? false);
if (!$name) {
throw new \InvalidArgumentException("Connection does not contain a 'path' or 'dbname' parameter and cannot be dropped.");
}

return $name;
}

public function isBackupActual(): bool
{
$backupDBFileName = $this->getBackupName();
$backupReferenceFileName = $backupDBFileName.'.ser';

return file_exists($backupDBFileName) && file_exists($backupReferenceFileName) && $this->isBackupUpToDate($backupDBFileName);
}

public function backup(AbstractExecutor $executor): void
{
$executor->getReferenceRepository()->save($this->getBackupName());
copy($this->getDatabaseName(), $this->getBackupName());
}

public function restore(AbstractExecutor $executor): void
{
copy($this->getBackupName(), $this->getDatabaseName());
$executor->getReferenceRepository()->load($this->getBackupName());
}
}
Loading