Adding support clickhouse cluster (golang-migrate#568)

* added url query for creating schema_migrations table on cluster Signed-off-by: Ildar Valiullin <[email protected]> * changed documentation by clickhouse databases Signed-off-by: Ildar Valiullin <[email protected]> Co-authored-by: Dale Hui <[email protected]>
bodyguard-ai · Aug 25, 2021 · aa11941 · aa11941
1 parent df6106d
commit aa11941
Show file tree

Hide file tree

Showing 2 changed files with 22 additions and 7 deletions.
diff --git a/database/clickhouse/README.md b/database/clickhouse/README.md
@@ -6,6 +6,7 @@
 |------------|-------------|
 | `x-migrations-table`| Name of the migrations table |
 | `x-migrations-table-engine`| Engine to use for the migrations table, defaults to TinyLog |
+| `x-cluster-name` | Name of cluster for creating `schema_migrations` table cluster wide |
 | `database` | The name of the database to connect to |
 | `username` | The user to sign in as |
 | `password` | The user's password |
@@ -18,4 +19,7 @@
 * The Clickhouse driver does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
   * This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
   * The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
-* Using the default TinyLog table engine for the schema_versions table prevents backing up the table if using the [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) tool. If backing up the database with make sure the migrations are run with `x-migrations-table-engine=MergeTree`.
+* Using the default TinyLog table engine for the schema_versions table prevents backing up the table if using the [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) tool. If backing up the database with make sure the migrations are run with `x-migrations-table-engine=MergeTree`.
+* Clickhouse cluster mode not officially supported, because not covered by tests right now, but you can try enable `schema_migrations` table replication:
+  * When `x-cluster-name` specified, `x-migrations-table-engine` also should be specify. Read about [replicated table engines](https://clickhouse.tech/docs/en/engines/table-engines/mergetree-family/replication/#table_engines-replication).
+  * `x-cluster-name` param only specify `schema_migrations` table replication by given cluster. You should still write your migrations so that the application tables are replicated within the cluster.
diff --git a/database/clickhouse/clickhouse.go b/database/clickhouse/clickhouse.go
@@ -30,6 +30,7 @@ var (
 
 type Config struct {
 	DatabaseName          string
+	ClusterName           string
 	MigrationsTable       string
 	MigrationsTableEngine string
 	MultiStatementEnabled bool
@@ -98,6 +99,7 @@ func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
 			MigrationsTable:       purl.Query().Get("x-migrations-table"),
 			MigrationsTableEngine: migrationsTableEngine,
 			DatabaseName:          purl.Query().Get("database"),
+			ClusterName:           purl.Query().Get("x-cluster-name"),
 			MultiStatementEnabled: purl.Query().Get("x-multi-statement") == "true",
 			MultiStatementMaxSize: multiStatementMaxSize,
 		},
@@ -227,12 +229,21 @@ func (ch *ClickHouse) ensureVersionTable() (err error) {
 	}
 
 	// if not, create the empty migration table
-	query = fmt.Sprintf(`
-		CREATE TABLE %s (
-			version    Int64,
-			dirty      UInt8,
-			sequence   UInt64
-		) Engine=%s`, ch.config.MigrationsTable, ch.config.MigrationsTableEngine)
+	if len(ch.config.ClusterName) > 0 {
+		query = fmt.Sprintf(`
+			CREATE TABLE %s ON CLUSTER %s (
+				version    Int64,
+				dirty      UInt8,
+				sequence   UInt64
+			) Engine=%s`, ch.config.MigrationsTable, ch.config.ClusterName, ch.config.MigrationsTableEngine)
+	} else {
+		query = fmt.Sprintf(`
+			CREATE TABLE %s (
+				version    Int64,
+				dirty      UInt8,
+				sequence   UInt64
+			) Engine=%s`, ch.config.MigrationsTable, ch.config.MigrationsTableEngine)
+	}
 
 	if strings.HasSuffix(ch.config.MigrationsTableEngine, "Tree") {
 		query = fmt.Sprintf(`%s ORDER BY sequence`, query)