Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infinispan cross-site replication failover configuration #34363

Merged
merged 1 commit into from
Jun 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 50 additions & 1 deletion docs/src/main/asciidoc/infinispan-client-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,56 @@ quarkus.infinispan-client.client-intelligence=BASIC <2>
Use Infinispan Dev Services to run a server and connect without configuration.
====

=== Configuring backup clusters in Cross-Site Replication
In High Availability production deployments, it is common to have multiple Infinispan Clusters that are
distributed across various Data Centers worldwide. Infinispan offers the capability to connect these clusters and
configure backups between them. This enables seamless switching between clusters through both automated and manual
methods using a single connection. To achieve this, it is necessary to configure the client to direct to the backup
clusters.

[source,properties]
----
quarkus.infinispan-client.hosts=host1:11222,host2:3122 <1>
quarkus.infinispan-client.username=admin
quarkus.infinispan-client.password=password
quarkus.infinispan-client.backup-cluster.nyc-site.hosts=nyc1:11222,nyc2:21222,nyc3:31222 <2>
quarkus.infinispan-client.backup-cluster.nyc-site.client-intelligence=BASIC <3>
quarkus.infinispan-client.backup-cluster.lon-site.hosts=lon1:11222,lon2:21222,lon3:31222 <4>
----
<1> Sets Infinispan Server address list, separated with commas. This is the default cluster.
<2> Configures a backup site 'nyc-site' with the provided address list
<3> Configures a backup site 'nyc-site' with the provided client intelligence
<4> Configures an additional backup site 'lon-site' with the provided address list

Based on the provided configuration, in the event of the default cluster becoming unavailable, the client will
seamlessly transition to one of the accessible backup clusters.
Additionally, there is also the option to manually switch the client to an alternate cluster:

[source,java]
----
@ApplicationScoped
public class InfinispanExample {
@Inject
RemoteCacheManager cacheManager;

public void doSomething() {
cacheManager.switchToCluster("nyc-site"); //<1>
cacheManager.switchToCluster("lon-site"); //<2>
cacheManager.switchToDefaultCluster(); //<3>
}
}
----
<1> The client connects to the 'nyc-site'.
<2> The client connects to the 'lon-site'.
<3> The client connects to the default site.

[NOTE]
====
Cross-site replication is a powerful feature offered by Infinispan that facilitates data backup between clusters
situated in geographically diverse data centers, even spanning across various cloud providers.
Learn more in the link:https://infinispan.org/docs/stable/titles/xsite/xsite.html[Infinispan documentation].
====

=== Default and named connections
This extension lets you configure a _default_ Infinispan client connections and _named_ ones.
Named connections are essential to connect to multiple Infinispan clusters.
Expand Down Expand Up @@ -163,7 +213,6 @@ the traces from the Infinispan Client to the Server.
This behavior can be disabled via the property `quarkus.infinispan-client.tracing.propagation.enabled`

=== Creating caches from the client

When a cache is accessed from the client, if the cache does not exist in the Infinispan Server and you want
to create it on first access, use one of the following properties:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package org.quarkus.infinispan.client.deployment;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.tuple;

import jakarta.inject.Inject;

Expand Down Expand Up @@ -29,9 +30,7 @@ public class InfinispanConfigurationSetupTest {
public void infinispanConnectionConfiguration() {
assertThat(remoteCacheManager).isNotNull();
Configuration configuration = remoteCacheManager.getConfiguration();
assertThat(configuration.servers().size()).isEqualTo(1);
assertThat(configuration.servers().get(0).host()).isEqualTo("cluster1");
assertThat(configuration.servers().get(0).port()).isEqualTo(31000);
assertThat(configuration.servers()).extracting("host", "port").containsExactly(tuple("cluster1", 31000));
assertThat(configuration.tracingPropagationEnabled()).isFalse();
assertThat(configuration.clientIntelligence()).isEqualTo(ClientIntelligence.BASIC);
assertThat(configuration.security().authentication().enabled()).isTrue();
Expand All @@ -44,6 +43,11 @@ public void infinispanConnectionConfiguration() {
assertThat(configuration.security().ssl().provider()).isEqualTo("SSL_prov");
assertThat(configuration.security().ssl().protocol()).isEqualTo("SSL_protocol");
assertThat(configuration.security().ssl().ciphers()).containsExactlyInAnyOrder("SSL_cipher1", "SSL_cipher2");
assertThat(configuration.clusters()).extracting("clusterName", "clientIntelligence")
.containsExactly(tuple("bsite", ClientIntelligence.BASIC));
assertThat(configuration.clusters()).hasSize(1);
assertThat(configuration.clusters().get(0).getCluster()).extracting("host", "port")
.containsExactly(tuple("bsite1", 32111));

assertThat(configuration.remoteCaches().get("cache1").configuration()).isEqualTo("<replicated-cache/>");
assertThat(configuration.remoteCaches().get("cache1").nearCacheBloomFilter()).isTrue();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ quarkus.infinispan-client.trust-store-type=JCEKS
quarkus.infinispan-client.ssl-provider=SSL_prov
quarkus.infinispan-client.ssl-protocol=SSL_protocol
quarkus.infinispan-client.ssl-ciphers=SSL_cipher1,SSL_cipher2
quarkus.infinispan-client.backup-cluster.bsite.hosts=bsite1:32111
quarkus.infinispan-client.backup-cluster.bsite.client-intelligence=BASIC

# cache 1 config
quarkus.infinispan-client.cache.cache1.configuration=<replicated-cache/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
import org.infinispan.client.hotrod.RemoteCache;
import org.infinispan.client.hotrod.RemoteCacheManager;
import org.infinispan.client.hotrod.RemoteCounterManagerFactory;
import org.infinispan.client.hotrod.configuration.ClientIntelligence;
import org.infinispan.client.hotrod.configuration.ClusterConfigurationBuilder;
import org.infinispan.client.hotrod.configuration.ConfigurationBuilder;
import org.infinispan.client.hotrod.impl.ConfigurationProperties;
import org.infinispan.client.hotrod.logging.Log;
Expand Down Expand Up @@ -289,6 +291,17 @@ private ConfigurationBuilder builderFromProperties(String infinispanClientName,
}
}

for (Map.Entry<String, InfinispanClientRuntimeConfig.BackupClusterConfig> backupCluster : infinispanClientRuntimeConfig.backupCluster
.entrySet()) {
InfinispanClientRuntimeConfig.BackupClusterConfig backupClusterConfig = backupCluster.getValue();
ClusterConfigurationBuilder clusterConfigurationBuilder = builder.addCluster(backupCluster.getKey());
clusterConfigurationBuilder.addClusterNodes(backupClusterConfig.hosts);
if (backupClusterConfig.clientIntelligence.isPresent()) {
clusterConfigurationBuilder.clusterClientIntelligence(
ClientIntelligence.valueOf(backupClusterConfig.clientIntelligence.get()));
}
}

return builder;
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package io.quarkus.infinispan.client.runtime;

import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
Expand Down Expand Up @@ -178,7 +177,20 @@ public class InfinispanClientRuntimeConfig {
* Configures caches from the client with the provided configuration.
*/
@ConfigItem
public Map<String, InfinispanClientRuntimeConfig.RemoteCacheConfig> cache = new HashMap<>();
public Map<String, InfinispanClientRuntimeConfig.RemoteCacheConfig> cache;

/**
* // @formatter:off
* Configures cross site replication from the client with the provided configuration.
* This allows automatic failover when a site is down, as well as switching manual with methods like:
* <code>
* cacheManager.switchToDefaultCluster();
* cacheManager.switchToCluster('clusterName')
* </code>
* // @formatter:on
*/
@ConfigItem
public Map<String, InfinispanClientRuntimeConfig.BackupClusterConfig> backupCluster;

@ConfigGroup
public static class RemoteCacheConfig {
Expand Down Expand Up @@ -233,6 +245,33 @@ public static class RemoteCacheConfig {
public Optional<Boolean> nearCacheUseBloomFilter;
}

@ConfigGroup
public static class BackupClusterConfig {
// @formatter:off
/**
* Sets the host name/port to connect to. Each one is separated by a semicolon (eg. hostA:11222;hostB:11222).
*/
// @formatter:on
@ConfigItem
public String hosts;

// @formatter:off
/**
* Sets client intelligence used by authentication
* Available values:
* * `BASIC` - Means that the client doesn't handle server topology changes and therefore will only use the list
* of servers supplied at configuration time.
* * `TOPOLOGY_AWARE` - Use this provider if you don't want the client to present any certificates to the
* remote TLS host.
* * `HASH_DISTRIBUTION_AWARE` - Like `TOPOLOGY_AWARE` but with the additional advantage that each request
* involving keys will be routed to the server who is the primary owner which improves performance
* greatly. This is the default.
*/
// @formatter:on
@ConfigItem(defaultValue = "HASH_DISTRIBUTION_AWARE")
Optional<String> clientIntelligence;
}

@Override
public String toString() {
return "InfinispanClientRuntimeConfig{" +
Expand Down