Skip to content

Commit

Permalink
Add spring properties for keyspace strategy and replications (patka#68)
Browse files Browse the repository at this point in the history
* feat: add spring properties for keyspace strategy and replications

* Fix typo

* improve properties

* update readme

* code cleanup

* test with 2 datacenters

* deprecate old keyspace-name property

* typo

* refactor similar to v3 branch feature

* update readme

* clean useless imports

* fix thrown exception
  • Loading branch information
rbleuse authored Apr 19, 2022
1 parent 182fa13 commit 875bcd5
Show file tree
Hide file tree
Showing 13 changed files with 292 additions and 83 deletions.
47 changes: 22 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ supposed to be used with this library. You can do this by adding the name to the
In order to make sure that this session will not be used by your application, you can
mark the application session as primary.
Here is an example for a programmatic configuration:
```
```java
@Bean
@Qualifier(CassandraMigrationAutoConfiguration.CQL_SESSION_BEAN_NAME)
public CqlSession cassandraMigrationCqlSession() {
Expand All @@ -46,27 +46,24 @@ In order to be sure to use the correct name, there is a public constant in
`CassandraMigrationAutoConfiguration` that is called `CQL_SESSION_BEAN_NAME`. You can use that
when declaring the session bean as shown in the example.

### Reactive Cassandra Driver
If you are using spring-data-cassandra or the reactive counterpart, you will not have easy access
to the ```CqlSession``` as it is created inside the ```CqlSessionFactoryBean``` and that class maintains
the ```CqlSession``` as a singleton.
### Spring Data Cassandra 3.X.X
If you are using spring-data-cassandra or the reactive counterpart, providing the CqlSession named `CQL_SESSION_BEAN_NAME` to be used by this library
will bypass the spring data session as it is annotated by [`@ConditionalOnMissingBean`](https://github.com/spring-projects/spring-boot/blob/fdb1010cbc75517f511d4ab82de7d8f0ee058849/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/cassandra/CassandraAutoConfiguration.java#L74).

As per my understanding the easiest solution would be to manually create another session in the manner
shown above so that ```cassandra-migration``` can pick it up. If you encounter any problems with the
wrong ```CqlSession``` ending up wired to your beans, you can mark the ```CqlSession``` created by Spring
as primary by using a ```BeanFactoryPostProcessor```:

```
import org.springframework.data.cassandra.config.DefaultCqlBeanNames;
The easiest solution is to provide both CqlSession and mark the one used by spring-data as `@Primary` :

@Component
public class CqlSessionFactoryPostProcessor implements BeanFactoryPostProcessor {
```java
@Bean(CassandraMigrationAutoConfiguration.CQL_SESSION_BEAN_NAME)
public CqlSession cassandraMigrationCqlSession() {
// migration session creation code here
}

@Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
BeanDefinition bd = beanFactory.getBeanDefinition(DefaultCqlBeanNames.SESSION);
bd.setPrimary(true);
}
@Bean(DefaultCqlBeanNames.SESSION)
@Primary
// ensure that the keyspace is created if needed before initializing spring-data session
@DependsOn(CassandraMigrationAutoConfiguration.MIGRATION_TASK_BEAN_NAME)
public CqlSession cassandraSession(CqlSessionBuilder cqlSessionBuilder) {
return cqlSessionBuilder.build();
}
```

Expand Down Expand Up @@ -127,7 +124,7 @@ Multi line comments are not supported.

## Migrations
Migrations are executed with the Quorum consistency level to make sure that always a majority of nodes share the same schema information.
Besides this after the scripts are executed, it wil be checked if the schema is in agreement by calling the
Besides this after the scripts are executed, it will be checked if the schema is in agreement by calling the
corresponding method on the metadata of the ResultSet. That call is blocking until either an agreement has been
reached or the configured `maxSchemaAgreementWaitSeconds` have been passed. This value can be configured on the Cluster
builder.
Expand All @@ -145,8 +142,8 @@ the actions or, the preferred approach, make use of Cassandras "IF EXISTS" or "I
ensure that the same script can be run multiple times without failing.

## More details
The library checks if there is a table inside the given keyspace that is called "schema_migration". If it
is not existing it will be created and it contains the following columns:
The library checks if there is a table inside the given keyspace that is called "schema_migration". It will be created if it
doesn't already exist and will contain the following columns:
* applied_successful (boolean)
* version (int)
* script_name varchar
Expand Down Expand Up @@ -176,8 +173,6 @@ just for the migration of the schema. In that case you can define a separate pro
used just for migrations. Have a look on the [Datastax documentation](https://docs.datastax.com/en/developer/java-driver/4.6/manual/core/configuration/)
on how to define such a profile.
Once defined, you can set the execution profile name in the `MigrationConfiguration` and it will be used during migration.
`execution-profile-name` is also available in the spring auto configuration and can be used in the `application.properties`
file.

## Version deprecation
Please be aware that the version 2 of this library that uses the old version 3 Datastax driver will be deprecated by end
Expand Down Expand Up @@ -209,9 +204,11 @@ the migration. You have to include the following dependency to make it work:

In your properties file you can set the following properties:
* cassandra.migration.keyspace-name Specifies the keyspace that should be migrated
* cassandra.migration.simple-strategy.replication-factor Specifies the simple strategy to use on the keyspace
* cassandra.migration.network-strategy.replications Specifies the network strategy to use on the keyspace with datacenters and factors
* cassandra.migration.script-locations Overrides the default script location
* cassandra.migration.strategy Can either be IGNORE_DUPLICATES or FAIL_ON_DUPLICATES
* cassandra.migration.consistency-level Provides the consistency level that will be used to execute migrations
* cassandra.migration.table-prefix Prefix for the migrations table name
* cassandra.migration.execution-profile-name the name for the execution profile
* cassandra.migration.with-consensus to prevent concurrent schema updates.
* cassandra.migration.with-consensus to prevent concurrent schema updates.
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
import org.cognitor.cassandra.migration.MigrationTask;
import org.cognitor.cassandra.migration.collector.FailOnDuplicatesCollector;
import org.cognitor.cassandra.migration.collector.IgnoreDuplicatesCollector;
import org.cognitor.cassandra.migration.keyspace.Keyspace;
import org.cognitor.cassandra.migration.keyspace.ReplicationStrategy;
import org.cognitor.cassandra.migration.scanner.ScannerRegistry;
import org.cognitor.cassandra.migration.spring.scanner.SpringBootLocationScanner;
import org.springframework.beans.factory.annotation.Autowired;
Expand All @@ -29,14 +31,15 @@
@ConditionalOnClass(CqlSession.class)
public class CassandraMigrationAutoConfiguration {
public static final String CQL_SESSION_BEAN_NAME = "cassandraMigrationCqlSession";
public static final String MIGRATION_TASK_BEAN_NAME = "migrationTask";
private final CassandraMigrationConfigurationProperties properties;

@Autowired
public CassandraMigrationAutoConfiguration(CassandraMigrationConfigurationProperties properties) {
this.properties = properties;
}

@Bean(initMethod = "migrate")
@Bean(name = MIGRATION_TASK_BEAN_NAME, initMethod = "migrate")
@ConditionalOnBean(value = CqlSession.class)
@ConditionalOnMissingBean(MigrationTask.class)
public MigrationTask migrationTask(@Qualifier(CQL_SESSION_BEAN_NAME) CqlSession cqlSession) {
Expand All @@ -46,16 +49,23 @@ public MigrationTask migrationTask(@Qualifier(CQL_SESSION_BEAN_NAME) CqlSession
}

MigrationRepository migrationRepository = createRepository();
MigrationConfiguration configuration = new MigrationConfiguration()
.withKeyspaceName(properties.getKeyspaceName())
.withTablePrefix(properties.getTablePrefix())
.withExecutionProfile(properties.getExecutionProfileName());
MigrationConfiguration configuration = createConfiguration();
return new MigrationTask(new Database(cqlSession, configuration)
.setConsistencyLevel(properties.getConsistencyLevel()),
migrationRepository,
properties.isWithConsensus());
}

private MigrationConfiguration createConfiguration() {
String keyspaceName = properties.getKeyspaceName();
ReplicationStrategy replicationStrategy = properties.getReplicationStrategy();

return new MigrationConfiguration()
.withKeyspace(new Keyspace(keyspaceName).with(replicationStrategy))
.withTablePrefix(properties.getTablePrefix())
.withExecutionProfile(properties.getExecutionProfileName());
}

private MigrationRepository createRepository() {
ScannerRegistry registry = new ScannerRegistry();
registry.register(ScannerRegistry.JAR_SCHEME, new SpringBootLocationScanner());
Expand All @@ -64,4 +74,4 @@ private MigrationRepository createRepository() {
}
return new MigrationRepository(properties.getScriptLocations(), new IgnoreDuplicatesCollector(), registry);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

import com.datastax.oss.driver.api.core.DefaultConsistencyLevel;
import org.cognitor.cassandra.migration.MigrationRepository;
import org.cognitor.cassandra.migration.keyspace.ReplicationStrategy;
import org.cognitor.cassandra.migration.spring.keyspace.KeyspaceReplicationStrategyDefinition;
import org.cognitor.cassandra.migration.spring.keyspace.NetworkStrategyDefinition;
import org.cognitor.cassandra.migration.spring.keyspace.SimpleStrategyDefinition;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.util.CollectionUtils;

Expand All @@ -20,6 +24,8 @@ public class CassandraMigrationConfigurationProperties {
private ScriptCollectorStrategy strategy = ScriptCollectorStrategy.FAIL_ON_DUPLICATES;
private List<String> scriptLocations = Collections.singletonList(MigrationRepository.DEFAULT_SCRIPT_PATH);
private String keyspaceName;
private KeyspaceReplicationStrategyDefinition simpleStrategy = new SimpleStrategyDefinition();
private KeyspaceReplicationStrategyDefinition networkStrategy;
private String tablePrefix = "";
private String executionProfileName = null;
private DefaultConsistencyLevel consistencyLevel = DefaultConsistencyLevel.QUORUM;
Expand Down Expand Up @@ -82,6 +88,43 @@ public void setKeyspaceName(String keyspaceName) {
this.keyspaceName = keyspaceName;
}

/**
* @return true if a keyspace name was provided, false otherwise
*/
public boolean hasKeyspaceName() {
return this.keyspaceName != null && !this.keyspaceName.isEmpty();
}

/**
* @return the strategy to use for creating keyspace. network strategy if is present,
* simple strategy otherwise.
*/
public ReplicationStrategy getReplicationStrategy() {
if (networkStrategy == null) {
return simpleStrategy.getStrategy();
}

return networkStrategy.getStrategy();
}

/**
* Sets the simple replication strategy that should be used to create keyspace
*
* @param simpleStrategy the simple strategy. This setting is optional.
*/
public void setSimpleStrategy(SimpleStrategyDefinition simpleStrategy) {
this.simpleStrategy = simpleStrategy;
}

/**
* Sets the network replication strategy that should be used to create keyspace
*
* @param networkStrategy the network strategy. This setting is optional.
*/
public void setNetworkStrategy(NetworkStrategyDefinition networkStrategy) {
this.networkStrategy = networkStrategy;
}

/**
* @return the strategy to use for collecting scripts inside the repository.
*/
Expand All @@ -99,13 +142,6 @@ public void setStrategy(ScriptCollectorStrategy strategy) {
this.strategy = strategy;
}

/**
* @return true if a keyspace name was provided, false otherwise
*/
public boolean hasKeyspaceName() {
return this.keyspaceName != null && !this.keyspaceName.isEmpty();
}

/**
* @return the consistency level to be used for schema migrations
*/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package org.cognitor.cassandra.migration.spring.keyspace;

import org.cognitor.cassandra.migration.keyspace.ReplicationStrategy;

public interface KeyspaceReplicationStrategyDefinition {

ReplicationStrategy getStrategy();
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package org.cognitor.cassandra.migration.spring.keyspace;

import org.cognitor.cassandra.migration.keyspace.NetworkStrategy;
import org.cognitor.cassandra.migration.keyspace.ReplicationStrategy;

import java.util.Map;

public class NetworkStrategyDefinition implements KeyspaceReplicationStrategyDefinition {

private Map<String, Integer> replications;

public Map<String, Integer> getReplications() {
return replications;
}

public void setReplications(Map<String, Integer> replications) {
this.replications = replications;
}

@Override
public ReplicationStrategy getStrategy() {
NetworkStrategy networkStrategy = new NetworkStrategy();
replications.forEach(networkStrategy::with);
return networkStrategy;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package org.cognitor.cassandra.migration.spring.keyspace;

import org.cognitor.cassandra.migration.keyspace.ReplicationStrategy;
import org.cognitor.cassandra.migration.keyspace.SimpleStrategy;

public class SimpleStrategyDefinition implements KeyspaceReplicationStrategyDefinition {

private int replicationFactor = 1;

public int getReplicationFactor() {
return replicationFactor;
}

public void setReplicationFactor(int replicationFactor) {
this.replicationFactor = replicationFactor;
}

@Override
public ReplicationStrategy getStrategy() {
return new SimpleStrategy(replicationFactor);
}
}
Loading

0 comments on commit 875bcd5

Please sign in to comment.