diff --git a/docs/design/AZURE_FEDERATIONS.md b/docs/design/AZURE_FEDERATIONS.md deleted file mode 100644 index fda5591b105..00000000000 --- a/docs/design/AZURE_FEDERATIONS.md +++ /dev/null @@ -1,94 +0,0 @@ -# Azure Federations - -Implementing Federations inside a new Doctrine Sharding Extension. Some extensions to the DBAL and ORM core have to be done to get this working. - -1. DBAL (Database Abstraction Layer) - -* Add support for Database Schema Operations - * CREATE FEDERATION - * CREATE TABLE ... FEDERATED ON - * Add support to create a multi-tenant schema from any given schema -* Add API to pick a shard based on distribution key and atomic value -* Add API to ask about federations, federation members and so on. -* Add Sharding Abstraction - * If a shard is picked via distribution key and atomic value fire queries against this only - * Or query the global database. - -2. ORM (Object-Relational Mapper) - -* Federation Key has to be part of the clustered index of the table - * Test with a pure Multi-Tenant App with Filtering = ON (TaskList) - * Test with sharded app (Weather) - -## Implementation Details - -SQL Azure requires one and exactly one clustered index. It makes no difference if the primary key -or any other key is the clustered index. Sharding requires an external ID generation (no auto-increment) -such as GUIDs. GUIDs have negative properties with regard to clustered index performance, so that -typically you would add a "created" timestamp for example that holds the clustered index instead -of making the GUID a clustered index. - -## Example API: - - @@@ php - 'tcp:dbname.database.windows.net', - 'sharding' => array( - 'federationName' => 'Orders_Federation', - 'distributionKey' => 'CustID', - 'distributionType' => 'integer', - 'filteringEnabled' => false, - ), - // ... - ); - - $conn = DriverManager::getConnection($dbParams); - $shardManager = $conn->getShardManager(); - - // Example 1: query against root database - $sql = "SELECT * FROM Products"; - $rows = $conn->executeQuery($sql); - - // Example 2: query against the selected shard with CustomerId = 100 - $aCustomerID = 100; - $shardManager->selectShard($aCustomerID); // Using Default federationName and distributionKey - // Query: "USE FEDERATION Orders_Federation (CustID = $aCustomerID) WITH RESET, FILTERING OFF;" - - $sql = "SELECT * FROM Customers"; - $rows = $conn->executeQuery($sql); - - // Example 3: Reset API to root database again - $shardManager->selectGlobal(); - -## ID Generation - -With sharding all the ids have to be generated for global uniqueness. There are three strategies for this. - -1. Use GUIDs as described here http://blogs.msdn.com/b/cbiyikoglu/archive/2011/06/20/id-generation-in-federations-identity-sequences-and-guids-uniqueidentifier.aspx -2. Having a central table that is accessed with a second connection to generate sequential ids -3. Using natural keys from the domain. - -The second approach has the benefit of having numerical primary keys, however also a central failure location. The third strategy can seldom be used, because the domains don't allow this. Identity columns cannot be used at all. - - @@@ php - 'dbname.database.windows.net', - // ... - ); - $conn = DriverManager::getConnection($dbParams); - - $idGenerator = new TableHiLoIdGenerator($conn, 'id_table_name', $multiplicator = 1); - // only once, create this table - $idGenerator->createTable(); - - $nextId = $idGenerator->generateId('for_table_name'); - $nextOtherId = $idGenerator->generateId('for_other_table'); - -The connection for the table generator has to be a different one than the one used for the main app to avoid transaction clashes. diff --git a/docs/design/SHARDING.md b/docs/design/SHARDING.md deleted file mode 100644 index c5256b386d5..00000000000 --- a/docs/design/SHARDING.md +++ /dev/null @@ -1,74 +0,0 @@ -# Doctrine Shards - -Doctrine Extension to support horizontal sharding in the Doctrine ORM. - -## Idea - -Implement sharding inside Doctrine at a level that is as unobtrusive to the developer as possible. - -Problems to tackle: - -1. Where to send INSERT statements? -2. How to generate primary keys? -3. How to pick shards for update, delete statements? -4. How to pick shards for select operations? -5. How to merge select queries that span multiple shards? -6. How to handle/prevent multi-shard queries that cannot be merged (GROUP BY)? -7. How to handle non-sharded data? (static metadata tables for example) -8. How to handle multiple connections? -9. Implementation on the DBAL or ORM level? - -## Roadmap - -Version 1: DBAL 2.3 (Multi-Tenant Apps) - - 1. ID Generation support (in DBAL + ORM done) - 2. Multi-Tenant Support: Either pick a global metadata database or exactly one shard. - 3. Fan-out queries over all shards (or a subset) by result appending - -Version 2: ORM related (complex): - - 4. ID resolving (Pick shard for a new ID) - 5. Query resolving (Pick shards a query should send to) - 6. Shard resolving (Pick shards an ID could be on) - 7. Transactions - 8. Read Only objects - -## Technical Requirements for Database Schemas - -Sharded tables require the sharding-distribution key as one of their columns. This will affect your code compared to a normalized db-schema. If you have a Blog <-> BlogPost <-> PostComments entity setup sharded by `blog_id` then even the PostComment table needs this column, even if an "unsharded", normalized DB-Schema does not need this information. - -## Implementation Details - -Assumptions: - -* For querying you either want to query ALL or just exactly one shard. -* IDs for ALL sharded tables have to be unique across all shards. -* Non-sharded data is replicated between all shards. They redundantly keep the information available. This is necessary so join queries on shards to reference data work. -* If you retrieve an object A from a shard, then all references and collections of this object reside on the same shard. -* The database schema on all shards is the same (or compatible) - -### SQL Azure Federations - -SQL Azure is a special case, points 1, 2, 3, 4, 7 and 8 are partly handled on the database level. This makes it a perfect test-implementation for just the subset of features in points 5-6. However there needs to be a way to configure SchemaTool to generate the correct Schema on SQL Azure. - -* SELECT Operations: The most simple assumption is to always query all shards unless the user specifies otherwise explicitly. -* Queries can be merged in PHP code, this obviously does not work for DISTINCT, GROUP BY and ORDER BY queries. - -### Generic Sharding - -More features are necessary to implement sharding on the PHP level, independent from database support: - -1. Configuration of multiple connections, one connection = one shard. -2. Primary Key Generation mechanisms (UUID, central table, sequence emulation) - -## Primary Use-Cases - -1. Multi-Tenant Applications - -These are easier to support as you have some value to determine the shard id for the whole request very early on. -Here also queries can always be limited to a single shard. - -2. Scale-Out by some attribute (Round-Robin?) - -This strategy requires access to multiple shards in a single request based on the data accessed. diff --git a/docs/en/reference/sharding.rst b/docs/en/reference/sharding.rst deleted file mode 100644 index dcdd7d8f660..00000000000 --- a/docs/en/reference/sharding.rst +++ /dev/null @@ -1,477 +0,0 @@ -Sharding -======== - -.. note:: - - The sharding extension is currently in transition from a separate Project - into DBAL. Class names may differ. - -Starting with 2.3 Doctrine DBAL contains some functionality to simplify the -development of horizontally sharded applications. In this first release it -contains a ``ShardManager`` interface. This interface allows to programatically -select a shard to send queries to. At the moment there are no functionalities -yet to dynamically pick a shard based on ID, query or database row yet. That -means the sharding extension is primarily suited for: - -- multi-tenant applications or -- applications with completely separated datasets (example: weather data). - -Both kind of application will work with both DBAL and ORM. - -.. note:: - - Horizontal sharding is an evasive architecture that will affect your application code and using this - extension to Doctrine will not make it work "magically". - -You have to understand and integrate the following drawbacks: - -- Pre-generation of IDs that are unique across all shards required. -- No transaction support across shards. -- No foreign key support across shards (meaning no "real" relations). -- Very complex (or impossible) to query aggregates across shards. -- Denormalization: Composite keys required where normalized non-sharded db schemas don't need them. -- Schema Operations have to be done on all shards. - -The primary questions in a sharding architecture are: - -* Where is my data located? -* Where should I save this new data to find it later? - -To answer these questions you generally have to craft a function that will tell -you for a given ID, on which shard the data for this ID is located. To simplify -this approach you will generally just pick a table which is the root of a set of -related data and decide for the IDs of this table. All the related data that -belong to this table are saved on the same shard. - -Take for example a multi-user blog application with the following tables: - -- Blog [id, name] -- Post [id, blog_id, subject, body, author_id] -- Comment [id, post_id, comment, author_id] -- User [id, username] - -A sensible sharding architecture will split the application by blog. That means -all the data for a particular blog will be on a single shard and scaling is -done by putting the amount of blogs on many different database servers. - -Now users can post and comment on different blogs that reside on different -shards. This makes the database schema above slightly tricky, because both -`author_id` columns cannot have foreign keys to `User (id)`. Instead the User -table is located in an entirely different "dimension" of the application in -terms of the sharding architecture. - -To simplify working with this kind of multi-dimensional database schema, you -can replace the author_ids with something more "meaningful", for example the -e-mail address of the users if that is always known. The "user" table can then -be separated from the database schema above and put on a second horizontally -scaled sharding architecture. - -As you can see, even with just the four tables above, sharding actually becomes -quite complex to think about. - -The rest of this section discusses Doctrine sharding functionality in technical -detail. - -ID Generation -------------- - -To solve the issue of unique ID-generation across all shards are several -approaches you should evaluate: - -Use GUID/UUIDs -~~~~~~~~~~~~~~ - -The most simple ID-generation mechanism for sharding are -universally unique identifiers. These are 16-byte -(128-bit) numbers that are guaranteed to be unique across different servers. -You can `read up on UUIDs on Wikipedia `_. - -The drawback of UUIDs is the segmentation they cause on indexes. Because UUIDs -are not sequentially generated, they can have negative impact on index access -performance. Additionally they are much bigger -than numerical primary keys (which are normally 4-bytes in length). - -At the moment Doctrine DBAL drivers MySQL and SQL Server support the generation -of UUID/GUIDs. You can use the following bit of code to generate them across -platforms: - -.. code-block:: php - - insert('my_table', [ - 'id' => $guid->toString(), - 'foo' => 'bar', - ]); - -In your application you should hide this details in Id-Generation services: - -.. code-block:: php - - `_. - -Table Generator -~~~~~~~~~~~~~~~ - -In some scenarios there is no way around a numerical, automatically -incrementing id. The way Auto incrementing IDs are implemented in MySQL and SQL -Server however is completely unsuitable for sharding. Remember in a sharding -architecture you have to know where the row for a specific ID is located and -IDs have to be globally unique across all servers. Auto-Increment Primary Keys -are missing both properties. - -To get around this issue you can use the so-called "table-generator" strategy. -In this case you define a single database that is responsible for the -generation of auto-incremented ids. You create a table on this database and -through the use of locking create new sequential ids. - -There are three important drawbacks to this strategy: - -- Single point of failure -- Bottleneck when application is write-heavy -- A second independent database connection is needed to guarantee transaction - safety. - -If you can live with this drawbacks then you can use table-generation with the -following code in Doctrine: - -.. code-block:: php - - nextValue("sequence_name1"); - $id2 = $tableGenerator->nextValue("sequence_name2"); - -The table generator obviously needs a table to work. The schema of this table -is described in the ``TableGenerator`` class-docblock. Alternatively you -can use the ``Doctrine\DBAL\Id\TableGeneratorSchemaVisitor`` and apply it to your -``Doctrine\DBAL\Schema\Schema`` instance. It will automatically add the required -sequence table. - -Natural Identifiers -~~~~~~~~~~~~~~~~~~~ - -Sometimes you are lucky and your application data-model comes with a natural -id. This is mostly the case for applications who get their IDs generated -somewhere else (exogeneous ID-generation) or that work with temporal data. In -that case you can just define the natural primary key and shard your -application based on this data. - -Transactions ------------- - -Transactions in sharding can only work for data that is located on a single -shard. If you need transactions in your sharding architecture then you have to -make sure that the data updated during a transaction is located on a single -shard. - -Foreign Keys ------------- - -Since you cannot create foreign keys between remote database servers, in a -sharding architecture you should put the data on a shard that belongs to each -other. But even if you can isolate most of the rows on a single shard there may -exist relations between tables that exist on different shards. In this case -your application should be aware of the potential inconsistencies and handle -them graciously. - -Complex Queries ---------------- - -GROUP BY, DISTINCT and ORDER BY are clauses that cannot be easily used in a -sharding architecture. If you have to execute these queries against multiple -shards then you cannot just append the different results to each other. - -You have to be aware of this problem and design your queries accordingly or -shard the data in a way that you never have to query multiple shards to -calculate a result. - -ShardManager Interface ----------------------- - -The central API of the sharding extension is the ``ShardManager`` interface. -It contains two different groups of functions with regard to sharding. - -First, it contains the Shard Selection API. You can pick a shard based on a -so-called "distribution-value" or reset the connection to the "global" shard, -a necessary database that often contains heavily cached, sharding independent -data such as meta tables or the "user/tenant" table. - -.. code-block:: php - - array( - 'federationName' => 'my_database', - 'distributionKey' => 'customer_id', - ) - )); - $shardManager = new SQLAzureShardManager($conn); - - $currentCustomerId = 1234; - $shardManager->selectShard($currentCustomerId); - // all queries after this call hit the shard - // where customer with id 1234 is on. - - $shardManager->selectGlobal(); - // the global database is selected. - -To access the currently selected distribution value use the following API -method: - -.. code-block:: php - - getCurrentDistributionValue(); - -The shard manager will prevent you switching shards when a transaction is open. -This is especially important when using sharding with the ORM. Because the ORM -uses a single transaction during the flush-operation this means that you can -only ever use one ``EntityManager`` with data from a single shard. - -The second API is the "fan-out" query API. This allows you to execute queries against -ALL shards. The order of the results of this operation is undefined, that means -your query has to return the data in a way that works for the application, or -you have to sort the data in the application. - -.. code-block:: php - - queryAll($sql, $params); - -Schema Operations: SchemaSynchronizer Interface ------------------------------------------------ - -Schema Operations in a sharding architecture are tricky. You have to perform -them on all databases instances (shards) at the same time. Also Doctrine -has problems with this in particular as you cannot generate an SQL file with -changes on any development machine anymore and apply this on production. The -required changes depend on the amount of shards. - -To allow the Doctrine Schema API operations on a sharding architecture we -performed a refactored from code inside ORM ``Doctrine\ORM\Tools\SchemaTool`` -class and extracted the code for operations on Schema instances into a new -``Doctrine\Shards\DBAL\SchemaSynchronizer`` interface. - -Every sharding implementation can implement this interface and allow schema -operations to take part on multiple shards. - -SQL Azure Federations ---------------------- - -Doctrine Shards ships with a custom implementation for Microsoft SQL -Azure. The Azure platform provides a native sharding functionality. In SQL -Azure the sharding functionality is called Federations. This -functionality applies the following restrictions (in line with the ones listed -above): - -- IDENTITY columns are not allowed on sharded tables (federated tables) -- Each table may only have exactly one clustered index and this index has to - have the distribution key/sharding-id as one column. -- Every unique index (or primary key) has to contain the - distribution-key/sharding-id. - -Especially the requirements 2 and 3 prevent normalized database schemas. You -have to put the distribution key on every sharded table, which can affect your -application code quite a bit. This may lead to the creation of composite keys -where you normally wouldn't need them. - -The benefit of SQL Azure Federations is that they implement all the -shard-picking logic on the server. You only have to make use of the ``USE -FEDERATION`` statement. You don't have to maintain a list of all the shards -inside your application and more importantly, resizing shards is done -transparently on the server. - -Features of SQL Azure are: - -- Central server to log into federations architecture. No need to know all - connection details of all shards. -- Database level operation to split shards, taking away the tediousness of this - operation for application developers. -- A global tablespace that can contain global data to all shards. -- One or many different federations (this library only supports working with - one) -- Sharded or non-sharded tables inside federations -- Allows filtering SELECT queries on the database based on the selected - sharding key value. This allows to implement sharded Multi-Tenant Apps very easily. - -To setup an SQL Azure ShardManager use the following code: - -.. code-block:: php - - 'my_database', - 'host' => 'tcp:dbname.windows.net', - 'user' => 'user@dbname', - 'password' => 'XXX', - 'sharding' => array( - 'federationName' => 'my_federation', - 'distributionKey' => 'customer_id', - 'distributionType' => 'integer', - ) - )); - $shardManager = new SQLAzureShardManager($conn); - -Currently you are limited to one federation in your application. - -You can inspect all the currently known shards on SQL Azure using the -``ShardManager#getShards()`` function: - -.. code-block:: php - - getShards() as $shard) { - echo $shard['id'] . " " . $shard['rangeLow'] . " - " . $shard['rangeHigh']; - } - -Schema Operations -~~~~~~~~~~~~~~~~~ - -Schema Operations on SQL Azure Federations are possible with the -``SQLAzureSchemaSynchronizer``. You can instantiate this from your code: - -.. code-block:: php - - createTable('Users'); - //... - - // marked as sharded, but no distribution column given: - // non-federated table inside the federation - $products = $schema->createTable('Products'); - $products->addOption('azure.federated', true); - //... - - // shared + distribution column: - // federated table - $customers = $schema->createTable('Customers'); - $customers->addColumn('CustomerID', 'integer'); - //... - $customers->addOption('azure.federated', true); - $customers->addOption('azure.federatedOnColumnName', 'CustomerID'); - -SQLAzure Filtering -~~~~~~~~~~~~~~~~~~ - -SQL Azure comes with a powerful filtering feature, that allows you to -automatically implement a multi-tenant application for a formerly single-tenant -application. The restriction to make this work is that your application does not work with -IDENTITY columns. - -Normally when you select a shard using ``ShardManager#selectShard()`` any query -executed against this shard will return data from ALL the tenants located on -this shard. With the "FILTERING=ON" flag on the ``USE FEDERATION`` query -however SQL Azure can automatically filter all SELECT queries with the chosen -distribution value. Additionally you can automatically set the currently -selected distribution value in every INSERT statement using a function for this -value as the ``DEFAULT`` part of the column. If you are using GUIDs for every -row then UPDATE and DELETE statements using only GUIDs will work out perfectly -as well, as they are by definition for unique rows. This feature allows you to -build multi-tenant applications, even though they were not originally designed -that way. - -To enable filtering you can use the -``SQLAzureShardManager#setFilteringEnabled()`` method. This method is not part -of the interface. You can also set a default value for filtering by passing it -as the "sharding.filteringEnabled" parameter to -``DriverManager#getConnection()``. - -Generic SQL Sharding Support ----------------------------- - -Besides the custom SQL Azure support there is a generic implementation that -works with all database drivers. It requires to specify all database -connections and will switch between the different connections under the hood -when using the ``ShardManager`` API. This is also the biggest drawback of this -approach, since fan-out queries need to connect to all databases in a single -request. - -See the configuration for a sample sharding connection: - -.. code-block:: php - - 'Doctrine\DBAL\Sharding\PoolingShardConnection', - 'driver' => 'pdo_sqlite', - 'global' => array('memory' => true), - 'shards' => array( - array('id' => 1, 'memory' => true), - array('id' => 2, 'memory' => true), - ), - 'shardChoser' => 'Doctrine\DBAL\Sharding\ShardChoser\MultiTenantShardChoser', - )); - -You have to configure the following options: - -- 'wrapperClass' - Selecting the PoolingShardConnection as above. -- 'global' - An array of database parameters that is used for connecting to the - global database. -- 'shards' - An array of shard database parameters. You have to specify an - 'id' parameter for each of the shard configurations. -- 'shardChoser' - Implementation of the - ``Doctrine\Shards\DBAL\ShardChoser\ShardChoser`` interface. - -The Shard Choser interface maps the distribution value to a shard-id. This -gives you the freedom to implement your own strategy for sharding the data -horizontally. diff --git a/docs/en/reference/sharding_azure_tutorial.rst b/docs/en/reference/sharding_azure_tutorial.rst deleted file mode 100644 index 71a4fab5594..00000000000 --- a/docs/en/reference/sharding_azure_tutorial.rst +++ /dev/null @@ -1,423 +0,0 @@ -SQLAzure Sharding Tutorial -========================== - -.. note:: - - The sharding extension is currently in transition from a separate Project - into DBAL. Class names may differ. - -This tutorial builds upon the `Brian Swans tutorial `_ -on SQLAzure Sharding and turns all the examples into examples using the Doctrine Sharding support. - -It introduces SQL Azure Sharding, which is an abstraction layer in SQL Azure to -support sharding. Many features for sharding are implemented on the database -level, which makes it much easier to work with than generic sharding -implementations. - -For this tutorial you need an Azure account. You don't need to deploy the code -on Azure, you can run it from your own machine against the remote database. - -.. note:: - - You can look at the code from the 'examples/sharding' directory. - -Install Doctrine ----------------- - -For this tutorial we will install Doctrine and the Sharding Extension through -`Composer `_ which is the easiest way to install -Doctrine. Composer is a new package manager for PHP. Download the -``composer.phar`` from their website and put it into a newly created folder for -this tutorial. Now create a ``composer.json`` file in this project root with -the following content: - - { - "require": { - "doctrine/dbal": "2.2.2", - "doctrine/shards": "0.2" - } - } - -Open up the commandline and switch to your tutorial root directory, then call -``php composer.phar install``. It will grab the code and install it into the -``vendor`` subdirectory of your project. It also creates an autoloader, so that -we don't have to care about this. - -Setup Connection ----------------- - -The first thing to start with is setting up Doctrine and the database connection: - -.. code-block:: php - - 'pdo_sqlsrv', - 'dbname' => 'SalesDB', - 'host' => 'tcp:dbname.windows.net', - 'user' => 'user@dbname', - 'password' => 'XXX', - 'platform' => new \Doctrine\DBAL\Platforms\SQLAzurePlatform(), - 'driverOptions' => array('MultipleActiveResultSets' => false), - 'sharding' => array( - 'federationName' => 'Orders_Federation', - 'distributionKey' => 'CustId', - 'distributionType' => 'integer', - ) - )); - - $shardManager = new SQLAzureShardManager($conn); - -Create Database ---------------- - -Create a new database using the Azure/SQL Azure management console. - -Create Schema -------------- - -Doctrine has a powerful schema API. We don't need to use low-level DDL -statements to generate the database schema. Instead you can use an Object-Oriented API -to create the database schema and then have Doctrine turn it into DDL -statements. - -We will recreate Brians example schema with Doctrine DBAL. Instead of having to -create federations and schema separately as in his example, Doctrine will do it -all in one step: - -.. code-block:: php - - createTable('Products'); - $products->addColumn('ProductID', 'integer'); - $products->addColumn('SupplierID', 'integer'); - $products->addColumn('ProductName', 'string'); - $products->addColumn('Price', 'decimal', array('scale' => 2, 'precision' => 12)); - $products->setPrimaryKey(array('ProductID')); - $products->addOption('azure.federated', true); - - $customers = $schema->createTable('Customers'); - $customers->addColumn('CustomerID', 'integer'); - $customers->addColumn('CompanyName', 'string'); - $customers->addColumn('FirstName', 'string'); - $customers->addColumn('LastName', 'string'); - $customers->setPrimaryKey(array('CustomerID')); - $customers->addOption('azure.federated', true); - $customers->addOption('azure.federatedOnColumnName', 'CustomerID'); - - $orders = $schema->createTable('Orders'); - $orders->addColumn('CustomerID', 'integer'); - $orders->addColumn('OrderID', 'integer'); - $orders->addColumn('OrderDate', 'datetime'); - $orders->setPrimaryKey(array('CustomerID', 'OrderID')); - $orders->addOption('azure.federated', true); - $orders->addOption('azure.federatedOnColumnName', 'CustomerID'); - - $orderItems = $schema->createTable('OrderItems'); - $orderItems->addColumn('CustomerID', 'integer'); - $orderItems->addColumn('OrderID', 'integer'); - $orderItems->addColumn('ProductID', 'integer'); - $orderItems->addColumn('Quantity', 'integer'); - $orderItems->setPrimaryKey(array('CustomerID', 'OrderID', 'ProductID')); - $orderItems->addOption('azure.federated', true); - $orderItems->addOption('azure.federatedOnColumnName', 'CustomerID'); - - // Create the Schema + Federation: - $synchronizer = new SQLAzureSchemaSynchronizer($conn, $shardManager); - $synchronizer->createSchema($schema); - - // Or jut look at the SQL: - echo implode("\n", $synchronizer->getCreateSchema($schema)); - -View Federation Members ------------------------ - -To see how many shard instances (called Federation Members) your SQLAzure database currently has -you can ask the ``ShardManager`` to enumerate all shards: - -.. code-block:: php - - getShards(); - foreach ($shards as $shard) { - print_r($shard); - } - -Insert Data ------------ - -Now we want to insert some test data into the database to see the behavior when -we split the shards. We use the same test data as Brian, but use the Doctrine -API to insert them. To insert data into federated tables we have to select the -shard we want to put the data into. We can use the ShardManager to execute this -operation for us: - -.. code-block:: php - - selectShard(0); - - $conn->insert("Products", array( - "ProductID" => 386, - "SupplierID" => 1001, - "ProductName" => 'Titanium Extension Bracket Left Hand', - "Price" => 5.25, - )); - $conn->insert("Products", array( - "ProductID" => 387, - "SupplierID" => 1001, - "ProductName" => 'Titanium Extension Bracket Right Hand', - "Price" => 5.25, - )); - $conn->insert("Products", array( - "ProductID" => 388, - "SupplierID" => 1001, - "ProductName" => 'Fusion Generator Module 5 kV', - "Price" => 10.50, - )); - $conn->insert("Products", array( - "ProductID" => 388, - "SupplierID" => 1001, - "ProductName" => 'Bypass Filter 400 MHz Low Pass', - "Price" => 10.50, - )); - - $conn->insert("Customers", array( - 'CustomerID' => 10, - 'CompanyName' => 'Van Nuys', - 'FirstName' => 'Catherine', - 'LastName' => 'Abel', - )); - $conn->insert("Customers", array( - 'CustomerID' => 20, - 'CompanyName' => 'Abercrombie', - 'FirstName' => 'Kim', - 'LastName' => 'Branch', - )); - $conn->insert("Customers", array( - 'CustomerID' => 30, - 'CompanyName' => 'Contoso', - 'FirstName' => 'Frances', - 'LastName' => 'Adams', - )); - $conn->insert("Customers", array( - 'CustomerID' => 40, - 'CompanyName' => 'A. Datum Corporation', - 'FirstName' => 'Mark', - 'LastName' => 'Harrington', - )); - $conn->insert("Customers", array( - 'CustomerID' => 50, - 'CompanyName' => 'Adventure Works', - 'FirstName' => 'Keith', - 'LastName' => 'Harris', - )); - $conn->insert("Customers", array( - 'CustomerID' => 60, - 'CompanyName' => 'Alpine Ski House', - 'FirstName' => 'Wilson', - 'LastName' => 'Pais', - )); - $conn->insert("Customers", array( - 'CustomerID' => 70, - 'CompanyName' => 'Baldwin Museum of Science', - 'FirstName' => 'Roger', - 'LastName' => 'Harui', - )); - $conn->insert("Customers", array( - 'CustomerID' => 80, - 'CompanyName' => 'Blue Yonder Airlines', - 'FirstName' => 'Pilar', - 'LastName' => 'Pinilla', - )); - $conn->insert("Customers", array( - 'CustomerID' => 90, - 'CompanyName' => 'City Power & Light', - 'FirstName' => 'Kari', - 'LastName' => 'Hensien', - )); - $conn->insert("Customers", array( - 'CustomerID' => 100, - 'CompanyName' => 'Coho Winery', - 'FirstName' => 'Peter', - 'LastName' => 'Brehm', - )); - - $conn->executeUpdate("DECLARE @orderId INT - - DECLARE @customerId INT - - SET @orderId = 10 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Hensien' and FirstName = 'Kari' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 388, 4) - - SET @orderId = 20 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Harui' and FirstName = 'Roger' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 389, 2) - - SET @orderId = 30 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Brehm' and FirstName = 'Peter' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 387, 3) - - SET @orderId = 40 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Pais' and FirstName = 'Wilson' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 388, 1)" - ); - -This puts the data into the currently only existing federation member. We -selected that federation member by picking 0 as distribution value, which is by -definition part of the only existing federation. - -Split Federation ----------------- - -Now lets split the federation, creating a second federation member. SQL Azure -will automatically redistribute the data into the two federations after you -executed this command. - -.. code-block:: php - - splitFederation(60); - -This little script uses the shard manager with a special method only existing -on the SQL AZure implementation ``splitFederation``. It accepts a value at -at which the split is executed. - -If you reexecute the ``view_federation_members.php`` script you can now see -that there are two federation members instead of just one as before. You can -see with the ``rangeLow`` and ``rangeHigh`` parameters what customers and -related entries are now served by which federation. - -Inserting Data after Split --------------------------- - -Now after we splitted the data we now have to make sure to be connected to the -right federation before inserting data. Lets add a new customer with ID 55 and -have him create an order. - -.. code-block:: php - - selectShard($newCustomerId); - - $conn->insert("Customers", array( - "CustomerID" => $newCustomerId, - "CompanyName" => "Microsoft", - "FirstName" => "Brian", - "LastName" => "Swan", - )); - - $conn->insert("Orders", array( - "CustomerID" => 55, - "OrderID" => 37, - "OrderDate" => date('Y-m-d H:i:s'), - )); - - $conn->insert("OrderItems", array( - "CustomerID" => 55, - "OrderID" => 37, - "ProductID" => 387, - "Quantity" => 1, - )); - -As you can see its very important to pick the right distribution key in your -sharded application. Otherwise you have to switch the shards very often, which -is not really easy to work with. If you pick the sharding key right then it -should be possible to select the shard only once per request for the major -number of use-cases. - -Fan-out the queries accross multiple shards should only be necessary for a -small number of queries, because these kind of queries are complex. - -Querying data with filtering off --------------------------------- - -To access the data you have to pick a shard again and then start selecting data -from it. - -.. code-block:: php - - selectShard(0); - - $data = $conn->fetchAll('SELECT * FROM Customers'); - print_r($data); - -This returns all customers from the shard with distribution value 0. This will -be all customers with id 10 to less than 60, since we split federations at 60. - -Querying data with filtering on -------------------------------- - -One special feature of SQL Azure is the possibility to database level filtering -based on the sharding distribution values. This means that SQL Azure will add -WHERE clauses with distributionkey=current distribution value conditions to -each distribution key. - -.. code-block:: php - - setFilteringEnabled(true); - $shardManager->selectShard(55); - - $data = $conn->fetchAll('SELECT * FROM Customers'); - print_r($data); - -Now you only get the customer with id = 55. The same holds for queries on the -``Orders`` and ``OrderItems`` table, which are restricted by customer id = 55. - diff --git a/docs/en/sidebar.rst b/docs/en/sidebar.rst index 0935c8dc179..c053421724a 100644 --- a/docs/en/sidebar.rst +++ b/docs/en/sidebar.rst @@ -13,8 +13,6 @@ reference/schema-representation reference/events reference/security - reference/sharding - reference/sharding_azure_tutorial reference/supporting-other-databases reference/portability reference/caching diff --git a/docs/examples/sharding/README.md b/docs/examples/sharding/README.md deleted file mode 100644 index 3680e544f40..00000000000 --- a/docs/examples/sharding/README.md +++ /dev/null @@ -1,26 +0,0 @@ -# Sharding with SQLAzure Example - -This example demonstrates Sharding with SQL Azure Federations. - -## Requirements - -1. Windows Azure Account -2. SQL Azure Database -3. Composer for dependencies - -## Install - - composer install - -Change "examples/sharding/bootstrap.php" to contain Database connection. - -## Order to execute Scripts - -1. create_schema.php -2. view_federation_members.php -3. insert_data.php -4. split_federation.php -5. insert_data_after_split.php -6. query_filtering_off.php -7. query_filtering_on.php - diff --git a/docs/examples/sharding/bootstrap.php b/docs/examples/sharding/bootstrap.php deleted file mode 100644 index fe174f1c1b7..00000000000 --- a/docs/examples/sharding/bootstrap.php +++ /dev/null @@ -1,26 +0,0 @@ - 'SalesDB', - 'host' => 'tcp:dbname.windows.net', - 'user' => 'user@dbname', - 'password' => 'XXX', - 'sharding' => array( - 'federationName' => 'Orders_Federation', - 'distributionKey' => 'CustId', - 'distributionType' => 'integer', - ) -); - -if ($config['host'] == "tcp:dbname.windows.net") { - die("You have to change the configuration to your Azure account.\n"); -} - -$conn = DriverManager::getConnection($config); -$shardManager = new SQLAzureShardManager($conn); - diff --git a/docs/examples/sharding/composer.json b/docs/examples/sharding/composer.json deleted file mode 100644 index 214f922178e..00000000000 --- a/docs/examples/sharding/composer.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "require": { - "doctrine/dbal": "*", - "doctrine/shards": "0.3" - } -} diff --git a/docs/examples/sharding/create_schema.php b/docs/examples/sharding/create_schema.php deleted file mode 100644 index afdf7cbacf7..00000000000 --- a/docs/examples/sharding/create_schema.php +++ /dev/null @@ -1,51 +0,0 @@ -createTable('Products'); -$products->addColumn('ProductID', 'integer'); -$products->addColumn('SupplierID', 'integer'); -$products->addColumn('ProductName', 'string'); -$products->addColumn('Price', 'decimal', array('scale' => 2, 'precision' => 12)); -$products->setPrimaryKey(array('ProductID')); -$products->addOption('azure.federated', true); - -$customers = $schema->createTable('Customers'); -$customers->addColumn('CustomerID', 'integer'); -$customers->addColumn('CompanyName', 'string'); -$customers->addColumn('FirstName', 'string'); -$customers->addColumn('LastName', 'string'); -$customers->setPrimaryKey(array('CustomerID')); -$customers->addOption('azure.federated', true); -$customers->addOption('azure.federatedOnColumnName', 'CustomerID'); - -$orders = $schema->createTable('Orders'); -$orders->addColumn('CustomerID', 'integer'); -$orders->addColumn('OrderID', 'integer'); -$orders->addColumn('OrderDate', 'datetime'); -$orders->setPrimaryKey(array('CustomerID', 'OrderID')); -$orders->addOption('azure.federated', true); -$orders->addOption('azure.federatedOnColumnName', 'CustomerID'); - -$orderItems = $schema->createTable('OrderItems'); -$orderItems->addColumn('CustomerID', 'integer'); -$orderItems->addColumn('OrderID', 'integer'); -$orderItems->addColumn('ProductID', 'integer'); -$orderItems->addColumn('Quantity', 'integer'); -$orderItems->setPrimaryKey(array('CustomerID', 'OrderID', 'ProductID')); -$orderItems->addOption('azure.federated', true); -$orderItems->addOption('azure.federatedOnColumnName', 'CustomerID'); - -// Create the Schema + Federation: -$synchronizer = new SQLAzureSchemaSynchronizer($conn, $shardManager); - -// Or just look at the SQL: -echo implode("\n", $synchronizer->getCreateSchema($schema)); - -$synchronizer->createSchema($schema); - diff --git a/docs/examples/sharding/insert_data.php b/docs/examples/sharding/insert_data.php deleted file mode 100644 index 57aeda6c9f8..00000000000 --- a/docs/examples/sharding/insert_data.php +++ /dev/null @@ -1,132 +0,0 @@ -selectShard(0); - -$conn->insert("Products", array( - "ProductID" => 386, - "SupplierID" => 1001, - "ProductName" => 'Titanium Extension Bracket Left Hand', - "Price" => 5.25, -)); -$conn->insert("Products", array( - "ProductID" => 387, - "SupplierID" => 1001, - "ProductName" => 'Titanium Extension Bracket Right Hand', - "Price" => 5.25, -)); -$conn->insert("Products", array( - "ProductID" => 388, - "SupplierID" => 1001, - "ProductName" => 'Fusion Generator Module 5 kV', - "Price" => 10.50, -)); -$conn->insert("Products", array( - "ProductID" => 389, - "SupplierID" => 1001, - "ProductName" => 'Bypass Filter 400 MHz Low Pass', - "Price" => 10.50, -)); - -$conn->insert("Customers", array( - 'CustomerID' => 10, - 'CompanyName' => 'Van Nuys', - 'FirstName' => 'Catherine', - 'LastName' => 'Abel', -)); -$conn->insert("Customers", array( - 'CustomerID' => 20, - 'CompanyName' => 'Abercrombie', - 'FirstName' => 'Kim', - 'LastName' => 'Branch', -)); -$conn->insert("Customers", array( - 'CustomerID' => 30, - 'CompanyName' => 'Contoso', - 'FirstName' => 'Frances', - 'LastName' => 'Adams', -)); -$conn->insert("Customers", array( - 'CustomerID' => 40, - 'CompanyName' => 'A. Datum Corporation', - 'FirstName' => 'Mark', - 'LastName' => 'Harrington', -)); -$conn->insert("Customers", array( - 'CustomerID' => 50, - 'CompanyName' => 'Adventure Works', - 'FirstName' => 'Keith', - 'LastName' => 'Harris', -)); -$conn->insert("Customers", array( - 'CustomerID' => 60, - 'CompanyName' => 'Alpine Ski House', - 'FirstName' => 'Wilson', - 'LastName' => 'Pais', -)); -$conn->insert("Customers", array( - 'CustomerID' => 70, - 'CompanyName' => 'Baldwin Museum of Science', - 'FirstName' => 'Roger', - 'LastName' => 'Harui', -)); -$conn->insert("Customers", array( - 'CustomerID' => 80, - 'CompanyName' => 'Blue Yonder Airlines', - 'FirstName' => 'Pilar', - 'LastName' => 'Pinilla', -)); -$conn->insert("Customers", array( - 'CustomerID' => 90, - 'CompanyName' => 'City Power & Light', - 'FirstName' => 'Kari', - 'LastName' => 'Hensien', -)); -$conn->insert("Customers", array( - 'CustomerID' => 100, - 'CompanyName' => 'Coho Winery', - 'FirstName' => 'Peter', - 'LastName' => 'Brehm', -)); - -$conn->executeUpdate(" - DECLARE @orderId INT - - DECLARE @customerId INT - - SET @orderId = 10 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Hensien' and FirstName = 'Kari' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 388, 4) - - SET @orderId = 20 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Harui' and FirstName = 'Roger' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 389, 2) - - SET @orderId = 30 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Brehm' and FirstName = 'Peter' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 387, 3) - - SET @orderId = 40 - SELECT @customerId = CustomerId FROM Customers WHERE LastName = 'Pais' and FirstName = 'Wilson' - - INSERT INTO Orders (CustomerId, OrderId, OrderDate) - VALUES (@customerId, @orderId, GetDate()) - - INSERT INTO OrderItems (CustomerID, OrderID, ProductID, Quantity) - VALUES (@customerId, @orderId, 388, 1)"); diff --git a/docs/examples/sharding/insert_data_aftersplit.php b/docs/examples/sharding/insert_data_aftersplit.php deleted file mode 100644 index 312e90b235b..00000000000 --- a/docs/examples/sharding/insert_data_aftersplit.php +++ /dev/null @@ -1,27 +0,0 @@ -selectShard($newCustomerId); - -$conn->insert("Customers", array( - "CustomerID" => $newCustomerId, - "CompanyName" => "Microsoft", - "FirstName" => "Brian", - "LastName" => "Swan", -)); - -$conn->insert("Orders", array( - "CustomerID" => 55, - "OrderID" => 37, - "OrderDate" => date('Y-m-d H:i:s'), -)); - -$conn->insert("OrderItems", array( - "CustomerID" => 55, - "OrderID" => 37, - "ProductID" => 387, - "Quantity" => 1, -)); diff --git a/docs/examples/sharding/query_filtering_off.php b/docs/examples/sharding/query_filtering_off.php deleted file mode 100644 index c0b24fa087d..00000000000 --- a/docs/examples/sharding/query_filtering_off.php +++ /dev/null @@ -1,8 +0,0 @@ -selectShard(0); - -$data = $conn->fetchAll('SELECT * FROM Customers'); -print_r($data); diff --git a/docs/examples/sharding/query_filtering_on.php b/docs/examples/sharding/query_filtering_on.php deleted file mode 100644 index e7d9e14546c..00000000000 --- a/docs/examples/sharding/query_filtering_on.php +++ /dev/null @@ -1,9 +0,0 @@ -setFilteringEnabled(true); -$shardManager->selectShard(55); - -$data = $conn->fetchAll('SELECT * FROM Customers'); -print_r($data); diff --git a/docs/examples/sharding/split_federation.php b/docs/examples/sharding/split_federation.php deleted file mode 100644 index ff681edfd2a..00000000000 --- a/docs/examples/sharding/split_federation.php +++ /dev/null @@ -1,5 +0,0 @@ -splitFederation(60); diff --git a/docs/examples/sharding/view_federation_members.php b/docs/examples/sharding/view_federation_members.php deleted file mode 100644 index 497e4df6d2f..00000000000 --- a/docs/examples/sharding/view_federation_members.php +++ /dev/null @@ -1,8 +0,0 @@ -getShards(); -foreach ($shards as $shard) { - print_r($shard); -} diff --git a/lib/Doctrine/DBAL/DriverManager.php b/lib/Doctrine/DBAL/DriverManager.php index 3699efc71e5..efbec683f0a 100644 --- a/lib/Doctrine/DBAL/DriverManager.php +++ b/lib/Doctrine/DBAL/DriverManager.php @@ -162,17 +162,6 @@ public static function getConnection( } } - // URL support for PoolingShardConnection - if (isset($params['global'])) { - $params['global'] = self::parseDatabaseUrl($params['global']); - } - - if (isset($params['shards'])) { - foreach ($params['shards'] as $key => $shardParams) { - $params['shards'][$key] = self::parseDatabaseUrl($shardParams); - } - } - self::_checkParams($params); $className = $params['driverClass'] ?? self::$_driverMap[$params['driver']]; diff --git a/lib/Doctrine/DBAL/Platforms/SQLAzurePlatform.php b/lib/Doctrine/DBAL/Platforms/SQLAzurePlatform.php deleted file mode 100644 index e3ed527b619..00000000000 --- a/lib/Doctrine/DBAL/Platforms/SQLAzurePlatform.php +++ /dev/null @@ -1,35 +0,0 @@ -hasOption('azure.federatedOnColumnName')) { - $distributionName = $table->getOption('azure.federatedOnDistributionName'); - $columnName = $table->getOption('azure.federatedOnColumnName'); - $stmt = ' FEDERATED ON (' . $distributionName . ' = ' . $columnName . ')'; - - $sql[0] .= $stmt; - } - - return $sql; - } -} diff --git a/lib/Doctrine/DBAL/Sharding/Exception/ActiveTransaction.php b/lib/Doctrine/DBAL/Sharding/Exception/ActiveTransaction.php deleted file mode 100644 index 3f21667986d..00000000000 --- a/lib/Doctrine/DBAL/Sharding/Exception/ActiveTransaction.php +++ /dev/null @@ -1,15 +0,0 @@ - "client" to the ShardChoser interface. - * - An exception is thrown if trying to switch shards during an open - * transaction. - * - * Instantiation through the DriverManager looks like: - * - * @example - * - * $conn = DriverManager::getConnection(array( - * 'wrapperClass' => 'Doctrine\DBAL\Sharding\PoolingShardConnection', - * 'driver' => 'pdo_mysql', - * 'global' => array('user' => '', 'password' => '', 'host' => '', 'dbname' => ''), - * 'shards' => array( - * array('id' => 1, 'user' => 'slave1', 'password', 'host' => '', 'dbname' => ''), - * array('id' => 2, 'user' => 'slave2', 'password', 'host' => '', 'dbname' => ''), - * ), - * 'shardChoser' => 'Doctrine\DBAL\Sharding\ShardChoser\MultiTenantShardChoser', - * )); - * $shardManager = $conn->getShardManager(); - * $shardManager->selectGlobal(); - * $shardManager->selectShard($value); - */ -class PoolingShardConnection extends Connection -{ - /** @var array|array */ - private $activeConnections = []; - - /** @var string|int|null */ - private $activeShardId; - - /** @var array>|array> */ - private $connectionParameters = []; - - /** - * {@inheritDoc} - * - * @throws InvalidArgumentException - */ - public function __construct( - array $params, - Driver $driver, - ?Configuration $config = null, - ?EventManager $eventManager = null - ) { - if (! isset($params['global'], $params['shards'])) { - throw new InvalidArgumentException('Connection Parameters require "global" and "shards" configurations.'); - } - - if (! isset($params['shardChoser'])) { - throw new InvalidArgumentException('Missing Shard Choser configuration "shardChoser".'); - } - - if (is_string($params['shardChoser'])) { - $params['shardChoser'] = new $params['shardChoser'](); - } - - if (! ($params['shardChoser'] instanceof ShardChoser)) { - throw new InvalidArgumentException('The "shardChoser" configuration is not a valid instance of Doctrine\DBAL\Sharding\ShardChoser\ShardChoser'); - } - - $this->connectionParameters[0] = array_merge($params, $params['global']); - - foreach ($params['shards'] as $shard) { - if (! isset($shard['id'])) { - throw new InvalidArgumentException('Missing "id" for one configured shard. Please specify a unique shard-id.'); - } - - if (! is_numeric($shard['id']) || $shard['id'] < 1) { - throw new InvalidArgumentException('Shard Id has to be a non-negative number.'); - } - - if (isset($this->connectionParameters[$shard['id']])) { - throw new InvalidArgumentException(sprintf('Shard "%s" is duplicated in the configuration.', $shard['id'])); - } - - $this->connectionParameters[$shard['id']] = array_merge($params, $shard); - } - - parent::__construct($params, $driver, $config, $eventManager); - } - - /** - * Get active shard id. - * - * @return string|int|null - */ - public function getActiveShardId() - { - return $this->activeShardId; - } - - /** - * {@inheritdoc} - */ - public function getParams() : array - { - return $this->activeShardId ? $this->connectionParameters[$this->activeShardId] : $this->connectionParameters[0]; - } - - /** - * Connects to a given shard. - * - * @param string|int|null $shardId - * - * @throws ShardingException - */ - public function connect($shardId = null) : void - { - if ($shardId === null && $this->_conn) { - return; - } - - if ($shardId !== null && $shardId === $this->activeShardId) { - return; - } - - if ($this->getTransactionNestingLevel() > 0) { - throw new ShardingException('Cannot switch shard when transaction is active.'); - } - - $activeShardId = $this->activeShardId = (int) $shardId; - - if (isset($this->activeConnections[$activeShardId])) { - $this->_conn = $this->activeConnections[$activeShardId]; - - return; - } - - $this->_conn = $this->activeConnections[$activeShardId] = $this->connectTo($activeShardId); - - if (! $this->_eventManager->hasListeners(Events::postConnect)) { - return; - } - - $eventArgs = new ConnectionEventArgs($this); - $this->_eventManager->dispatchEvent(Events::postConnect, $eventArgs); - } - - /** - * Connects to a specific connection. - * - * @param string|int $shardId - */ - protected function connectTo($shardId) : DriverConnection - { - $params = $this->getParams(); - - $driverOptions = $params['driverOptions'] ?? []; - - $connectionParams = $this->connectionParameters[$shardId]; - - $user = $connectionParams['user'] ?? ''; - $password = $connectionParams['password'] ?? ''; - - return $this->_driver->connect($connectionParams, $user, $password, $driverOptions); - } - - /** - * @param string|int|null $shardId - */ - public function isConnected($shardId = null) : bool - { - if ($shardId === null) { - return $this->_conn !== null; - } - - return isset($this->activeConnections[$shardId]); - } - - public function close() : void - { - $this->_conn = null; - $this->activeConnections = []; - $this->activeShardId = null; - } -} diff --git a/lib/Doctrine/DBAL/Sharding/PoolingShardManager.php b/lib/Doctrine/DBAL/Sharding/PoolingShardManager.php deleted file mode 100644 index 5a21944d303..00000000000 --- a/lib/Doctrine/DBAL/Sharding/PoolingShardManager.php +++ /dev/null @@ -1,103 +0,0 @@ -getParams(); - $this->conn = $conn; - $this->choser = $params['shardChoser']; - } - - /** - * {@inheritDoc} - */ - public function selectGlobal() : void - { - $this->conn->connect(0); - $this->currentDistributionValue = null; - } - - /** - * {@inheritDoc} - */ - public function selectShard($distributionValue) : void - { - $shardId = $this->choser->pickShard($distributionValue, $this->conn); - $this->conn->connect($shardId); - $this->currentDistributionValue = $distributionValue; - } - - /** - * {@inheritDoc} - */ - public function getCurrentDistributionValue() - { - return $this->currentDistributionValue; - } - - /** - * {@inheritDoc} - */ - public function getShards() : array - { - $params = $this->conn->getParams(); - $shards = []; - - foreach ($params['shards'] as $shard) { - $shards[] = ['id' => $shard['id']]; - } - - return $shards; - } - - /** - * {@inheritDoc} - * - * @throws RuntimeException - */ - public function queryAll(string $sql, array $params, array $types) : array - { - $shards = $this->getShards(); - if (! $shards) { - throw new RuntimeException('No shards found.'); - } - - $result = []; - $oldDistribution = $this->getCurrentDistributionValue(); - - foreach ($shards as $shard) { - $this->conn->connect($shard['id']); - foreach ($this->conn->fetchAll($sql, $params, $types) as $row) { - $result[] = $row; - } - } - - if ($oldDistribution === null) { - $this->selectGlobal(); - } else { - $this->selectShard($oldDistribution); - } - - return $result; - } -} diff --git a/lib/Doctrine/DBAL/Sharding/SQLAzure/SQLAzureFederationsSynchronizer.php b/lib/Doctrine/DBAL/Sharding/SQLAzure/SQLAzureFederationsSynchronizer.php deleted file mode 100644 index 343b941a6ec..00000000000 --- a/lib/Doctrine/DBAL/Sharding/SQLAzure/SQLAzureFederationsSynchronizer.php +++ /dev/null @@ -1,263 +0,0 @@ -shardManager = $shardManager; - $this->synchronizer = $sync ?: new SingleDatabaseSynchronizer($conn); - } - - /** - * {@inheritdoc} - */ - public function getCreateSchema(Schema $createSchema) : array - { - $sql = []; - - [$global, $federation] = $this->partitionSchema($createSchema); - - $globalSql = $this->synchronizer->getCreateSchema($global); - if ($globalSql) { - $sql[] = "-- Create Root Federation\n" . - 'USE FEDERATION ROOT WITH RESET;'; - $sql = array_merge($sql, $globalSql); - } - - $federationSql = $this->synchronizer->getCreateSchema($federation); - - if ($federationSql) { - $defaultValue = $this->getFederationTypeDefaultValue(); - - $sql[] = $this->getCreateFederationStatement(); - $sql[] = 'USE FEDERATION ' . $this->shardManager->getFederationName() . ' (' . $this->shardManager->getDistributionKey() . ' = ' . $defaultValue . ') WITH RESET, FILTERING = OFF;'; - $sql = array_merge($sql, $federationSql); - } - - return $sql; - } - - /** - * {@inheritdoc} - */ - public function getUpdateSchema(Schema $toSchema, bool $noDrops = false) : array - { - return $this->work($toSchema, static function ($synchronizer, $schema) use ($noDrops) { - return $synchronizer->getUpdateSchema($schema, $noDrops); - }); - } - - /** - * {@inheritdoc} - */ - public function getDropSchema(Schema $dropSchema) : array - { - return $this->work($dropSchema, static function ($synchronizer, $schema) { - return $synchronizer->getDropSchema($schema); - }); - } - - /** - * {@inheritdoc} - */ - public function createSchema(Schema $createSchema) : void - { - $this->processSql($this->getCreateSchema($createSchema)); - } - - /** - * {@inheritdoc} - */ - public function updateSchema(Schema $toSchema, bool $noDrops = false) : void - { - $this->processSql($this->getUpdateSchema($toSchema, $noDrops)); - } - - /** - * {@inheritdoc} - */ - public function dropSchema(Schema $dropSchema) : void - { - $this->processSqlSafely($this->getDropSchema($dropSchema)); - } - - /** - * {@inheritdoc} - */ - public function getDropAllSchema() : array - { - $this->shardManager->selectGlobal(); - $globalSql = $this->synchronizer->getDropAllSchema(); - - if ($globalSql) { - $sql[] = "-- Work on Root Federation\nUSE FEDERATION ROOT WITH RESET;"; - $sql = array_merge($sql, $globalSql); - } - - $shards = $this->shardManager->getShards(); - foreach ($shards as $shard) { - $this->shardManager->selectShard($shard['rangeLow']); - - $federationSql = $this->synchronizer->getDropAllSchema(); - if (! $federationSql) { - continue; - } - - $sql[] = '-- Work on Federation ID ' . $shard['id'] . "\n" . - 'USE FEDERATION ' . $this->shardManager->getFederationName() . ' (' . $this->shardManager->getDistributionKey() . ' = ' . $shard['rangeLow'] . ') WITH RESET, FILTERING = OFF;'; - $sql = array_merge($sql, $federationSql); - } - - $sql[] = 'USE FEDERATION ROOT WITH RESET;'; - $sql[] = 'DROP FEDERATION ' . $this->shardManager->getFederationName(); - - return $sql; - } - - /** - * {@inheritdoc} - */ - public function dropAllSchema() : void - { - $this->processSqlSafely($this->getDropAllSchema()); - } - - /** - * @return Schema[] - */ - private function partitionSchema(Schema $schema) : array - { - return [ - $this->extractSchemaFederation($schema, false), - $this->extractSchemaFederation($schema, true), - ]; - } - - /** - * @throws RuntimeException - */ - private function extractSchemaFederation(Schema $schema, bool $isFederation) : Schema - { - $partitionedSchema = clone $schema; - - foreach ($partitionedSchema->getTables() as $table) { - if ($isFederation) { - $table->addOption(self::FEDERATION_DISTRIBUTION_NAME, $this->shardManager->getDistributionKey()); - } - - if ($table->hasOption(self::FEDERATION_TABLE_FEDERATED) !== $isFederation) { - $partitionedSchema->dropTable($table->getName()); - } else { - foreach ($table->getForeignKeys() as $fk) { - $foreignTable = $schema->getTable($fk->getForeignTableName()); - if ($foreignTable->hasOption(self::FEDERATION_TABLE_FEDERATED) !== $isFederation) { - throw new RuntimeException('Cannot have foreign key between global/federation.'); - } - } - } - } - - return $partitionedSchema; - } - - /** - * Work on the Global/Federation based on currently existing shards and - * perform the given operation on the underlying schema synchronizer given - * the different partitioned schema instances. - * - * @return string[] - */ - private function work(Schema $schema, Closure $operation) : array - { - [$global, $federation] = $this->partitionSchema($schema); - $sql = []; - - $this->shardManager->selectGlobal(); - $globalSql = $operation($this->synchronizer, $global); - - if ($globalSql) { - $sql[] = "-- Work on Root Federation\nUSE FEDERATION ROOT WITH RESET;"; - $sql = array_merge($sql, $globalSql); - } - - $shards = $this->shardManager->getShards(); - - foreach ($shards as $shard) { - $this->shardManager->selectShard($shard['rangeLow']); - - $federationSql = $operation($this->synchronizer, $federation); - if (! $federationSql) { - continue; - } - - $sql[] = '-- Work on Federation ID ' . $shard['id'] . "\n" . - 'USE FEDERATION ' . $this->shardManager->getFederationName() . ' (' . $this->shardManager->getDistributionKey() . ' = ' . $shard['rangeLow'] . ') WITH RESET, FILTERING = OFF;'; - $sql = array_merge($sql, $federationSql); - } - - return $sql; - } - - private function getFederationTypeDefaultValue() : string - { - $federationType = Type::getType($this->shardManager->getDistributionType()); - - switch ($federationType->getName()) { - case Types::GUID: - $defaultValue = '00000000-0000-0000-0000-000000000000'; - break; - case Types::INTEGER: - case Types::SMALLINT: - case Types::BIGINT: - $defaultValue = '0'; - break; - default: - $defaultValue = ''; - break; - } - - return $defaultValue; - } - - private function getCreateFederationStatement() : string - { - $federationType = Type::getType($this->shardManager->getDistributionType()); - $federationTypeSql = $federationType->getSQLDeclaration([], $this->conn->getDatabasePlatform()); - - return "--Create Federation\n" . - 'CREATE FEDERATION ' . $this->shardManager->getFederationName() . ' (' . $this->shardManager->getDistributionKey() . ' ' . $federationTypeSql . ' RANGE)'; - } -} diff --git a/lib/Doctrine/DBAL/Sharding/SQLAzure/SQLAzureShardManager.php b/lib/Doctrine/DBAL/Sharding/SQLAzure/SQLAzureShardManager.php deleted file mode 100644 index 0b00e9ade13..00000000000 --- a/lib/Doctrine/DBAL/Sharding/SQLAzure/SQLAzureShardManager.php +++ /dev/null @@ -1,199 +0,0 @@ -conn = $conn; - $params = $conn->getParams(); - - if (! isset($params['sharding']['federationName'])) { - throw MissingDefaultFederationName::new(); - } - - if (! isset($params['sharding']['distributionKey'])) { - throw MissingDefaultDistributionKey::new(); - } - - if (! isset($params['sharding']['distributionType'])) { - throw MissingDistributionType::new(); - } - - $this->federationName = $params['sharding']['federationName']; - $this->distributionKey = $params['sharding']['distributionKey']; - $this->distributionType = $params['sharding']['distributionType']; - $this->filteringEnabled = (bool) ($params['sharding']['filteringEnabled'] ?? false); - } - - /** - * Gets the name of the federation. - */ - public function getFederationName() : string - { - return $this->federationName; - } - - /** - * Gets the distribution key. - */ - public function getDistributionKey() : string - { - return $this->distributionKey; - } - - /** - * Gets the Doctrine Type name used for the distribution. - */ - public function getDistributionType() : string - { - return $this->distributionType; - } - - /** - * Sets Enabled/Disable filtering on the fly. - */ - public function setFilteringEnabled(bool $flag) : void - { - $this->filteringEnabled = $flag; - } - - /** - * {@inheritDoc} - */ - public function selectGlobal() : void - { - if ($this->conn->isTransactionActive()) { - throw ActiveTransaction::new(); - } - - $sql = 'USE FEDERATION ROOT WITH RESET'; - $this->conn->exec($sql); - $this->currentDistributionValue = null; - } - - /** - * {@inheritDoc} - */ - public function selectShard($distributionValue) : void - { - if ($this->conn->isTransactionActive()) { - throw ActiveTransaction::new(); - } - - $platform = $this->conn->getDatabasePlatform(); - $sql = sprintf( - 'USE FEDERATION %s (%s = %s) WITH RESET, FILTERING = %s;', - $platform->quoteIdentifier($this->federationName), - $platform->quoteIdentifier($this->distributionKey), - $this->conn->quote($distributionValue), - ($this->filteringEnabled ? 'ON' : 'OFF') - ); - - $this->conn->exec($sql); - $this->currentDistributionValue = $distributionValue; - } - - /** - * {@inheritDoc} - */ - public function getCurrentDistributionValue() - { - return $this->currentDistributionValue; - } - - /** - * {@inheritDoc} - */ - public function getShards() : array - { - $sql = 'SELECT member_id as id, - distribution_name as distribution_key, - CAST(range_low AS CHAR) AS rangeLow, - CAST(range_high AS CHAR) AS rangeHigh - FROM sys.federation_member_distributions d - INNER JOIN sys.federations f ON f.federation_id = d.federation_id - WHERE f.name = ' . $this->conn->quote($this->federationName); - - return $this->conn->fetchAll($sql); - } - - /** - * {@inheritDoc} - */ - public function queryAll(string $sql, array $params = [], array $types = []) : array - { - $shards = $this->getShards(); - if (! $shards) { - throw new RuntimeException(sprintf('No shards found for "%s".', $this->federationName)); - } - - $result = []; - $oldDistribution = $this->getCurrentDistributionValue(); - - foreach ($shards as $shard) { - $this->selectShard($shard['rangeLow']); - foreach ($this->conn->fetchAll($sql, $params, $types) as $row) { - $result[] = $row; - } - } - - if ($oldDistribution === null) { - $this->selectGlobal(); - } else { - $this->selectShard($oldDistribution); - } - - return $result; - } - - /** - * Splits Federation at a given distribution value. - * - * @param mixed $splitDistributionValue - */ - public function splitFederation($splitDistributionValue) : void - { - $sql = 'ALTER FEDERATION ' . $this->getFederationName() . ' ' . - 'SPLIT AT (' . $this->getDistributionKey() . ' = ' . - $this->conn->quote($splitDistributionValue) . ')'; - $this->conn->exec($sql); - } -} diff --git a/lib/Doctrine/DBAL/Sharding/SQLAzure/Schema/MultiTenantVisitor.php b/lib/Doctrine/DBAL/Sharding/SQLAzure/Schema/MultiTenantVisitor.php deleted file mode 100644 index a14f26c2418..00000000000 --- a/lib/Doctrine/DBAL/Sharding/SQLAzure/Schema/MultiTenantVisitor.php +++ /dev/null @@ -1,145 +0,0 @@ -excludedTables = $excludedTables; - $this->tenantColumnName = $tenantColumnName; - $this->distributionName = $distributionName ?: $tenantColumnName; - } - - /** - * {@inheritdoc} - */ - public function acceptTable(Table $table) : void - { - if (in_array($table->getName(), $this->excludedTables)) { - return; - } - - $table->addColumn($this->tenantColumnName, $this->tenantColumnType, [ - 'default' => "federation_filtering_value('" . $this->distributionName . "')", - ]); - - $clusteredIndex = $this->getClusteredIndex($table); - - $indexColumns = $clusteredIndex->getColumns(); - $indexColumns[] = $this->tenantColumnName; - - if ($clusteredIndex->isPrimary()) { - $table->dropPrimaryKey(); - $table->setPrimaryKey($indexColumns); - } else { - $table->dropIndex($clusteredIndex->getName()); - $table->addIndex($indexColumns, $clusteredIndex->getName()); - $table->getIndex($clusteredIndex->getName())->addFlag('clustered'); - } - } - - /** - * @throws RuntimeException - */ - private function getClusteredIndex(Table $table) : Index - { - foreach ($table->getIndexes() as $index) { - if ($index->isPrimary() && ! $index->hasFlag('nonclustered')) { - return $index; - } - - if ($index->hasFlag('clustered')) { - return $index; - } - } - throw new RuntimeException(sprintf('No clustered index found on table "%s".', $table->getName())); - } - - /** - * {@inheritdoc} - */ - public function acceptSchema(Schema $schema) : void - { - } - - /** - * {@inheritdoc} - */ - public function acceptColumn(Table $table, Column $column) : void - { - } - - /** - * {@inheritdoc} - */ - public function acceptForeignKey(Table $localTable, ForeignKeyConstraint $fkConstraint) : void - { - } - - /** - * {@inheritdoc} - */ - public function acceptIndex(Table $table, Index $index) : void - { - } - - /** - * {@inheritdoc} - */ - public function acceptSequence(Sequence $sequence) : void - { - } -} diff --git a/lib/Doctrine/DBAL/Sharding/ShardChoser/MultiTenantShardChoser.php b/lib/Doctrine/DBAL/Sharding/ShardChoser/MultiTenantShardChoser.php deleted file mode 100644 index c6c2f3843f4..00000000000 --- a/lib/Doctrine/DBAL/Sharding/ShardChoser/MultiTenantShardChoser.php +++ /dev/null @@ -1,22 +0,0 @@ - 'pdo_mysql', - 'shardChoser' => MultiTenantShardChoser::class, - 'global' => ['url' => 'mysql://foo:bar@localhost:11211/baz'], - 'shards' => [ - [ - 'id' => 1, - 'url' => 'mysql://foo:bar@localhost:11211/baz_slave', - ], - ], - 'wrapperClass' => PoolingShardConnection::class, - ]; - - $conn = DriverManager::getConnection($options); - - $params = $conn->getParams(); - self::assertInstanceOf(PDOMySQLDriver::class, $conn->getDriver()); - - $expected = [ - 'user' => 'foo', - 'password' => 'bar', - 'host' => 'localhost', - 'port' => 11211, - ]; - - foreach ($expected as $key => $value) { - self::assertEquals($value, $params['global'][$key]); - self::assertEquals($value, $params['shards'][0][$key]); - } - - self::assertEquals('baz', $params['global']['dbname']); - self::assertEquals('baz_slave', $params['shards'][0]['dbname']); - } - /** * @param mixed $url * @param mixed $expected diff --git a/tests/Doctrine/Tests/DBAL/Platforms/SQLAzurePlatformTest.php b/tests/Doctrine/Tests/DBAL/Platforms/SQLAzurePlatformTest.php deleted file mode 100644 index 69eac4a127d..00000000000 --- a/tests/Doctrine/Tests/DBAL/Platforms/SQLAzurePlatformTest.php +++ /dev/null @@ -1,33 +0,0 @@ -platform = new SQLAzurePlatform(); - } - - public function testCreateFederatedOnTable() : void - { - $table = new Table('tbl'); - $table->addColumn('id', 'integer'); - $table->addOption('azure.federatedOnDistributionName', 'TblId'); - $table->addOption('azure.federatedOnColumnName', 'id'); - - self::assertEquals(['CREATE TABLE tbl (id INT NOT NULL) FEDERATED ON (TblId = id)'], $this->platform->getCreateTableSQL($table)); - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/PoolingShardConnectionTest.php b/tests/Doctrine/Tests/DBAL/Sharding/PoolingShardConnectionTest.php deleted file mode 100644 index 6716efe4260..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/PoolingShardConnectionTest.php +++ /dev/null @@ -1,260 +0,0 @@ -createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 1, 'memory' => true], - ['id' => 2, 'memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - - assert($conn instanceof PoolingShardConnection); - - self::assertFalse($conn->isConnected(0)); - $conn->connect(0); - self::assertEquals(1, $conn->fetchColumn('SELECT 1')); - self::assertTrue($conn->isConnected(0)); - - self::assertFalse($conn->isConnected(1)); - $conn->connect(1); - self::assertEquals(1, $conn->fetchColumn('SELECT 1')); - self::assertTrue($conn->isConnected(1)); - - self::assertFalse($conn->isConnected(2)); - $conn->connect(2); - self::assertEquals(1, $conn->fetchColumn('SELECT 1')); - self::assertTrue($conn->isConnected(2)); - - $conn->close(); - self::assertFalse($conn->isConnected(0)); - self::assertFalse($conn->isConnected(1)); - self::assertFalse($conn->isConnected(2)); - } - - public function testNoGlobalServerException() : void - { - $this->expectException(InvalidArgumentException::class); - $this->expectExceptionMessage('Connection Parameters require "global" and "shards" configurations.'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'shards' => [ - ['id' => 1, 'memory' => true], - ['id' => 2, 'memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - } - - public function testNoShardsServersException() : void - { - $this->expectException(InvalidArgumentException::class); - $this->expectExceptionMessage('Connection Parameters require "global" and "shards" configurations.'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - } - - public function testNoShardsChoserException() : void - { - $this->expectException(InvalidArgumentException::class); - $this->expectExceptionMessage('Missing Shard Choser configuration "shardChoser".'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 1, 'memory' => true], - ['id' => 2, 'memory' => true], - ], - ]); - } - - public function testShardChoserWrongInstance() : void - { - $this->expectException(InvalidArgumentException::class); - $this->expectExceptionMessage('The "shardChoser" configuration is not a valid instance of Doctrine\DBAL\Sharding\ShardChoser\ShardChoser'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 1, 'memory' => true], - ['id' => 2, 'memory' => true], - ], - 'shardChoser' => new stdClass(), - ]); - } - - public function testShardNonNumericId() : void - { - $this->expectException('InvalidArgumentException'); - $this->expectExceptionMessage('Shard Id has to be a non-negative number.'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 'foo', 'memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - } - - public function testShardMissingId() : void - { - $this->expectException(InvalidArgumentException::class); - $this->expectExceptionMessage('Missing "id" for one configured shard. Please specify a unique shard-id.'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - } - - public function testDuplicateShardId() : void - { - $this->expectException(InvalidArgumentException::class); - $this->expectExceptionMessage('Shard "1" is duplicated in the configuration.'); - - $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 1, 'memory' => true], - ['id' => 1, 'memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - } - - public function testSwitchShardWithOpenTransactionException() : void - { - $conn = $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 1, 'memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - - assert($conn instanceof PoolingShardConnection); - - $conn->beginTransaction(); - - $this->expectException(ShardingException::class); - $this->expectExceptionMessage('Cannot switch shard when transaction is active.'); - $conn->connect(1); - } - - public function testGetActiveShardId() : void - { - $conn = $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true], - 'shards' => [ - ['id' => 1, 'memory' => true], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - - assert($conn instanceof PoolingShardConnection); - - self::assertNull($conn->getActiveShardId()); - - $conn->connect(0); - self::assertEquals(0, $conn->getActiveShardId()); - - $conn->connect(1); - self::assertEquals(1, $conn->getActiveShardId()); - - $conn->close(); - self::assertNull($conn->getActiveShardId()); - } - - public function testGetParamsOverride() : void - { - $conn = $this->createConnection([ - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true, 'host' => 'localhost'], - 'shards' => [ - ['id' => 1, 'memory' => true, 'host' => 'foo'], - ], - 'shardChoser' => MultiTenantShardChoser::class, - ]); - - assert($conn instanceof PoolingShardConnection); - - self::assertEquals([ - 'wrapperClass' => PoolingShardConnection::class, - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true, 'host' => 'localhost'], - 'shards' => [ - ['id' => 1, 'memory' => true, 'host' => 'foo'], - ], - 'shardChoser' => new MultiTenantShardChoser(), - 'memory' => true, - 'host' => 'localhost', - ], $conn->getParams()); - - $conn->connect(1); - self::assertEquals([ - 'wrapperClass' => PoolingShardConnection::class, - 'driver' => 'pdo_sqlite', - 'global' => ['memory' => true, 'host' => 'localhost'], - 'shards' => [ - ['id' => 1, 'memory' => true, 'host' => 'foo'], - ], - 'shardChoser' => new MultiTenantShardChoser(), - 'id' => 1, - 'memory' => true, - 'host' => 'foo', - ], $conn->getParams()); - } - - /** - * @param array $parameters - */ - private function createConnection(array $parameters) : PoolingShardConnection - { - $connection = DriverManager::getConnection(array_merge( - ['wrapperClass' => PoolingShardConnection::class], - $parameters - )); - - self::assertInstanceOf(PoolingShardConnection::class, $connection); - - return $connection; - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/PoolingShardManagerTest.php b/tests/Doctrine/Tests/DBAL/Sharding/PoolingShardManagerTest.php deleted file mode 100644 index 114b27ca824..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/PoolingShardManagerTest.php +++ /dev/null @@ -1,153 +0,0 @@ -getMockBuilder(PoolingShardConnection::class) - ->setMethods(['connect', 'getParams', 'fetchAll']) - ->disableOriginalConstructor() - ->getMock(); - } - - private function createPassthroughShardChoser() : ShardChoser - { - $mock = $this->createMock(ShardChoser::class); - $mock->expects($this->any()) - ->method('pickShard') - ->will($this->returnCallback(static function ($value) { - return $value; - })); - - return $mock; - } - - private function createStaticShardChooser() : ShardChoser - { - $mock = $this->createMock(ShardChoser::class); - $mock->expects($this->any()) - ->method('pickShard') - ->willReturn(1); - - return $mock; - } - - public function testSelectGlobal() : void - { - $conn = $this->createConnectionMock(); - $conn->expects($this->at(0)) - ->method('getParams') - ->will( - $this->returnValue([ - 'shardChoser' => $this->createMock(ShardChoser::class), - ]) - ); - $conn->expects($this->at(1))->method('connect')->with($this->equalTo(0)); - - $shardManager = new PoolingShardManager($conn); - $shardManager->selectGlobal(); - - self::assertNull($shardManager->getCurrentDistributionValue()); - } - - public function testSelectShard() : void - { - $shardId = 10; - $conn = $this->createConnectionMock(); - $conn->expects($this->at(0))->method('getParams')->will($this->returnValue(['shardChoser' => $this->createPassthroughShardChoser()])); - $conn->expects($this->at(1))->method('connect')->with($this->equalTo($shardId)); - - $shardManager = new PoolingShardManager($conn); - $shardManager->selectShard($shardId); - - self::assertEquals($shardId, $shardManager->getCurrentDistributionValue()); - } - - public function testGetShards() : void - { - $conn = $this->createConnectionMock(); - $conn->expects($this->any())->method('getParams')->will( - $this->returnValue( - ['shards' => [ ['id' => 1], ['id' => 2] ], 'shardChoser' => $this->createPassthroughShardChoser()] - ) - ); - - $shardManager = new PoolingShardManager($conn); - $shards = $shardManager->getShards(); - - self::assertEquals([['id' => 1], ['id' => 2]], $shards); - } - - public function testQueryAll() : void - { - $sql = 'SELECT * FROM table'; - $params = [1]; - $types = [1]; - - $conn = $this->createConnectionMock(); - $conn->expects($this->at(0))->method('getParams')->will($this->returnValue( - ['shards' => [ ['id' => 1], ['id' => 2] ], 'shardChoser' => $this->createPassthroughShardChoser()] - )); - $conn->expects($this->at(1))->method('getParams')->will($this->returnValue( - ['shards' => [ ['id' => 1], ['id' => 2] ], 'shardChoser' => $this->createPassthroughShardChoser()] - )); - $conn->expects($this->at(2))->method('connect')->with($this->equalTo(1)); - $conn->expects($this->at(3)) - ->method('fetchAll') - ->with($this->equalTo($sql), $this->equalTo($params), $this->equalTo($types)) - ->will($this->returnValue([ ['id' => 1] ])); - $conn->expects($this->at(4))->method('connect')->with($this->equalTo(2)); - $conn->expects($this->at(5)) - ->method('fetchAll') - ->with($this->equalTo($sql), $this->equalTo($params), $this->equalTo($types)) - ->will($this->returnValue([ ['id' => 2] ])); - - $shardManager = new PoolingShardManager($conn); - $result = $shardManager->queryAll($sql, $params, $types); - - self::assertEquals([['id' => 1], ['id' => 2]], $result); - } - - public function testQueryAllWithStaticShardChoser() : void - { - $sql = 'SELECT * FROM table'; - $params = [1]; - $types = [1]; - - $conn = $this->createConnectionMock(); - $conn->expects($this->at(0))->method('getParams')->will($this->returnValue( - ['shards' => [ ['id' => 1], ['id' => 2] ], 'shardChoser' => $this->createStaticShardChooser()] - )); - $conn->expects($this->at(1))->method('getParams')->will($this->returnValue( - ['shards' => [ ['id' => 1], ['id' => 2] ], 'shardChoser' => $this->createStaticShardChooser()] - )); - $conn->expects($this->at(2))->method('connect')->with($this->equalTo(1)); - $conn->expects($this->at(3)) - ->method('fetchAll') - ->with($this->equalTo($sql), $this->equalTo($params), $this->equalTo($types)) - ->will($this->returnValue([ ['id' => 1] ])); - $conn->expects($this->at(4))->method('connect')->with($this->equalTo(2)); - $conn->expects($this->at(5)) - ->method('fetchAll') - ->with($this->equalTo($sql), $this->equalTo($params), $this->equalTo($types)) - ->will($this->returnValue([ ['id' => 2] ])); - - $shardManager = new PoolingShardManager($conn); - $result = $shardManager->queryAll($sql, $params, $types); - - self::assertEquals([['id' => 1], ['id' => 2]], $result); - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/AbstractTestCase.php b/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/AbstractTestCase.php deleted file mode 100644 index ef36ada364a..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/AbstractTestCase.php +++ /dev/null @@ -1,97 +0,0 @@ -markTestSkipped('No driver or sqlserver driver specified.'); - } - - $params = [ - 'driver' => $GLOBALS['db_type'], - 'dbname' => $GLOBALS['db_name'], - 'user' => $GLOBALS['db_username'], - 'password' => $GLOBALS['db_password'], - 'host' => $GLOBALS['db_host'], - 'sharding' => [ - 'federationName' => 'Orders_Federation', - 'distributionKey' => 'CustID', - 'distributionType' => 'integer', - 'filteringEnabled' => false, - ], - 'driverOptions' => ['MultipleActiveResultSets' => false], - ]; - $this->conn = DriverManager::getConnection($params); - - $serverEdition = $this->conn->fetchColumn("SELECT CONVERT(NVARCHAR(128), SERVERPROPERTY('Edition'))"); - - if (strpos($serverEdition, 'SQL Azure') !== 0) { - $this->markTestSkipped('SQL Azure only test.'); - } - - // assume database is created and schema is: - // Global products table - // Customers, Orders, OrderItems federation tables. - // See http://cloud.dzone.com/articles/using-sql-azure-federations - $this->sm = new SQLAzureShardManager($this->conn); - } - - protected function createShopSchema() : Schema - { - $schema = new Schema(); - - $products = $schema->createTable('Products'); - $products->addColumn('ProductID', 'integer'); - $products->addColumn('SupplierID', 'integer'); - $products->addColumn('ProductName', 'string'); - $products->addColumn('Price', 'decimal', ['scale' => 2, 'precision' => 12]); - $products->setPrimaryKey(['ProductID']); - $products->addOption('azure.federated', true); - - $customers = $schema->createTable('Customers'); - $customers->addColumn('CustomerID', 'integer'); - $customers->addColumn('CompanyName', 'string'); - $customers->addColumn('FirstName', 'string'); - $customers->addColumn('LastName', 'string'); - $customers->setPrimaryKey(['CustomerID']); - $customers->addOption('azure.federated', true); - $customers->addOption('azure.federatedOnColumnName', 'CustomerID'); - - $orders = $schema->createTable('Orders'); - $orders->addColumn('CustomerID', 'integer'); - $orders->addColumn('OrderID', 'integer'); - $orders->addColumn('OrderDate', 'datetime'); - $orders->setPrimaryKey(['CustomerID', 'OrderID']); - $orders->addOption('azure.federated', true); - $orders->addOption('azure.federatedOnColumnName', 'CustomerID'); - - $orderItems = $schema->createTable('OrderItems'); - $orderItems->addColumn('CustomerID', 'integer'); - $orderItems->addColumn('OrderID', 'integer'); - $orderItems->addColumn('ProductID', 'integer'); - $orderItems->addColumn('Quantity', 'integer'); - $orderItems->setPrimaryKey(['CustomerID', 'OrderID', 'ProductID']); - $orderItems->addOption('azure.federated', true); - $orderItems->addOption('azure.federatedOnColumnName', 'CustomerID'); - - return $schema; - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/FunctionalTest.php b/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/FunctionalTest.php deleted file mode 100644 index 67bf2369494..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/FunctionalTest.php +++ /dev/null @@ -1,47 +0,0 @@ -createShopSchema(); - - $synchronizer = new SQLAzureFederationsSynchronizer($this->conn, $this->sm); - $synchronizer->dropAllSchema(); - $synchronizer->createSchema($schema); - - $this->sm->selectShard(0); - - $this->conn->insert('Products', [ - 'ProductID' => 1, - 'SupplierID' => 2, - 'ProductName' => 'Test', - 'Price' => 10.45, - ]); - - $this->conn->insert('Customers', [ - 'CustomerID' => 1, - 'CompanyName' => 'Foo', - 'FirstName' => 'Benjamin', - 'LastName' => 'E.', - ]); - - $query = 'SELECT * FROM Products'; - $data = $this->conn->fetchAll($query); - self::assertGreaterThan(0, count($data)); - - $query = 'SELECT * FROM Customers'; - $data = $this->conn->fetchAll($query); - self::assertGreaterThan(0, count($data)); - - $data = $this->sm->queryAll('SELECT * FROM Customers'); - self::assertGreaterThan(0, count($data)); - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/MultiTenantVisitorTest.php b/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/MultiTenantVisitorTest.php deleted file mode 100644 index 019ef12a2a0..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/MultiTenantVisitorTest.php +++ /dev/null @@ -1,50 +0,0 @@ -createTable('foo'); - $foo->addColumn('id', 'string'); - $foo->setPrimaryKey(['id']); - $schema->visit($visitor); - - self::assertEquals(['id', 'tenant_id'], $foo->getPrimaryKey()->getColumns()); - self::assertTrue($foo->hasColumn('tenant_id')); - } - - public function testMultiTenantNonPrimaryKey() : void - { - $platform = new SQLAzurePlatform(); - $visitor = new MultiTenantVisitor(); - - $schema = new Schema(); - $foo = $schema->createTable('foo'); - $foo->addColumn('id', 'string'); - $foo->addColumn('created', 'datetime'); - $foo->setPrimaryKey(['id']); - $foo->addIndex(['created'], 'idx'); - - $foo->getPrimaryKey()->addFlag('nonclustered'); - $foo->getIndex('idx')->addFlag('clustered'); - - $schema->visit($visitor); - - self::assertEquals(['id'], $foo->getPrimaryKey()->getColumns()); - self::assertTrue($foo->hasColumn('tenant_id')); - self::assertEquals(['created', 'tenant_id'], $foo->getIndex('idx')->getColumns()); - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/SQLAzureFederationsSynchronizerTest.php b/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/SQLAzureFederationsSynchronizerTest.php deleted file mode 100644 index ce8ee7ba065..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/SQLAzureFederationsSynchronizerTest.php +++ /dev/null @@ -1,51 +0,0 @@ -createShopSchema(); - - $synchronizer = new SQLAzureFederationsSynchronizer($this->conn, $this->sm); - $sql = $synchronizer->getCreateSchema($schema); - - self::assertEquals([ - "--Create Federation\nCREATE FEDERATION Orders_Federation (CustID INT RANGE)", - 'USE FEDERATION Orders_Federation (CustID = 0) WITH RESET, FILTERING = OFF;', - 'CREATE TABLE Products (ProductID INT NOT NULL, SupplierID INT NOT NULL, ProductName NVARCHAR(255) NOT NULL, Price NUMERIC(12, 2) NOT NULL, PRIMARY KEY (ProductID))', - 'CREATE TABLE Customers (CustomerID INT NOT NULL, CompanyName NVARCHAR(255) NOT NULL, FirstName NVARCHAR(255) NOT NULL, LastName NVARCHAR(255) NOT NULL, PRIMARY KEY (CustomerID))', - 'CREATE TABLE Orders (CustomerID INT NOT NULL, OrderID INT NOT NULL, OrderDate DATETIME2(6) NOT NULL, PRIMARY KEY (CustomerID, OrderID))', - 'CREATE TABLE OrderItems (CustomerID INT NOT NULL, OrderID INT NOT NULL, ProductID INT NOT NULL, Quantity INT NOT NULL, PRIMARY KEY (CustomerID, OrderID, ProductID))', - ], $sql); - } - - public function testUpdateSchema() : void - { - $schema = $this->createShopSchema(); - - $synchronizer = new SQLAzureFederationsSynchronizer($this->conn, $this->sm); - $synchronizer->dropAllSchema(); - - $sql = $synchronizer->getUpdateSchema($schema); - - self::assertEquals([], $sql); - } - - public function testDropSchema() : void - { - $schema = $this->createShopSchema(); - - $synchronizer = new SQLAzureFederationsSynchronizer($this->conn, $this->sm); - $synchronizer->dropAllSchema(); - $synchronizer->createSchema($schema); - $sql = $synchronizer->getDropSchema($schema); - - self::assertCount(5, $sql); - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/SQLAzureShardManagerTest.php b/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/SQLAzureShardManagerTest.php deleted file mode 100644 index 5b850d3fdc5..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/SQLAzure/SQLAzureShardManagerTest.php +++ /dev/null @@ -1,100 +0,0 @@ -expectException(ShardingException::class); - $this->expectExceptionMessage('SQLAzure requires a federation name to be set during sharding configuration.'); - - $conn = $this->createConnection(['sharding' => ['distributionKey' => 'abc', 'distributionType' => 'integer']]); - new SQLAzureShardManager($conn); - } - - public function testNoDistributionKey() : void - { - $this->expectException(ShardingException::class); - $this->expectExceptionMessage('SQLAzure requires a distribution key to be set during sharding configuration.'); - - $conn = $this->createConnection(['sharding' => ['federationName' => 'abc', 'distributionType' => 'integer']]); - new SQLAzureShardManager($conn); - } - - public function testNoDistributionType() : void - { - $this->expectException(ShardingException::class); - - $conn = $this->createConnection(['sharding' => ['federationName' => 'abc', 'distributionKey' => 'foo']]); - new SQLAzureShardManager($conn); - } - - public function testGetDefaultDistributionValue() : void - { - $conn = $this->createConnection(['sharding' => ['federationName' => 'abc', 'distributionKey' => 'foo', 'distributionType' => 'integer']]); - - $sm = new SQLAzureShardManager($conn); - self::assertNull($sm->getCurrentDistributionValue()); - } - - public function testSelectGlobalTransactionActive() : void - { - $conn = $this->createConnection(['sharding' => ['federationName' => 'abc', 'distributionKey' => 'foo', 'distributionType' => 'integer']]); - $conn->expects($this->at(1))->method('isTransactionActive')->will($this->returnValue(true)); - - $this->expectException(ShardingException::class); - $this->expectExceptionMessage('Cannot switch shard during an active transaction.'); - - $sm = new SQLAzureShardManager($conn); - $sm->selectGlobal(); - } - - public function testSelectGlobal() : void - { - $conn = $this->createConnection(['sharding' => ['federationName' => 'abc', 'distributionKey' => 'foo', 'distributionType' => 'integer']]); - $conn->expects($this->at(1))->method('isTransactionActive')->will($this->returnValue(false)); - $conn->expects($this->at(2))->method('exec')->with($this->equalTo('USE FEDERATION ROOT WITH RESET')); - - $sm = new SQLAzureShardManager($conn); - $sm->selectGlobal(); - } - - public function testSelectShard() : void - { - $conn = $this->createConnection(['sharding' => ['federationName' => 'abc', 'distributionKey' => 'foo', 'distributionType' => 'integer']]); - $conn->expects($this->at(1))->method('isTransactionActive')->will($this->returnValue(true)); - - $this->expectException(ShardingException::class); - $this->expectExceptionMessage('Cannot switch shard during an active transaction.'); - - $sm = new SQLAzureShardManager($conn); - $sm->selectShard(1234); - - self::assertEquals(1234, $sm->getCurrentDistributionValue()); - } - - /** - * @param mixed[] $params - * - * @return Connection|MockObject - */ - private function createConnection(array $params) : Connection - { - $conn = $this->getMockBuilder(Connection::class) - ->setMethods(['getParams', 'exec', 'isTransactionActive']) - ->disableOriginalConstructor() - ->getMock(); - $conn->expects($this->at(0))->method('getParams')->will($this->returnValue($params)); - - return $conn; - } -} diff --git a/tests/Doctrine/Tests/DBAL/Sharding/ShardChoser/MultiTenantShardChoserTest.php b/tests/Doctrine/Tests/DBAL/Sharding/ShardChoser/MultiTenantShardChoserTest.php deleted file mode 100644 index 97e9e21ef33..00000000000 --- a/tests/Doctrine/Tests/DBAL/Sharding/ShardChoser/MultiTenantShardChoserTest.php +++ /dev/null @@ -1,29 +0,0 @@ -createConnectionMock(); - - self::assertEquals(1, $choser->pickShard(1, $conn)); - self::assertEquals(2, $choser->pickShard(2, $conn)); - } - - private function createConnectionMock() : PoolingShardConnection - { - return $this->getMockBuilder(PoolingShardConnection::class) - ->setMethods(['connect', 'getParams', 'fetchAll']) - ->disableOriginalConstructor() - ->getMock(); - } -}