Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use quarkus with Google Cloud Run And CloudSQL #6634

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions docs/src/main/asciidoc/deploying-to-google-cloud.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/master/docs/src/main/asciidoc
////
= Quarkus - Deploying on Google cloud platform

include::./attributes.adoc[]

This guide covers:

* Connecting to a CloudSQL postgresql instance with quarkus-agroal and service account using CloudRun
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved
== Prerequisites

For this guide you need:

* roughly 1 hour
* having access to an Google cloud platform project with owner rights.

This guide will take as input an application developed in the link:datasource.adoc[datasource guide].
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved

Make sure you have the application at hand working locally.


== Solution

We recommend to follow the instructions in the next sections and build the application step by step.

== Connecting to CloudSQL

Google CloudSQL managed service allows 4 kinds of connection :

. Using public IP
. Using private IP
. Using Cloud SQL Proxy
. Using service account

The first two don't need anymore details, simply connect to your database following the link:datasource.adoc[datasource guide].
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved
The third allows you to connect as if locally, but is not available for AppEngine or CloudRun where you only deploy one artifact.
Please check link:https://cloud.google.com/sql/docs/postgres/external-connection-methods?hl=en[Connection options for external applications].

This guide will help you through the fourth possibility : connecting using service account.

== Adding necessary dependencies to your application.

You will need to add the _Cloud SQL Postgres Socket Factory_ and postgresql driver dependencies.
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved

With maven :
[source,xml, subs="attributes"]
----
<!-- https://mvnrepository.com/artifact/com.google.cloud.sql/postgres-socket-factory -->
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>postgres-socket-factory</artifactId>
<version>1.0.15</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.quarkus/quarkus-jdbc-postgresql -->
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-jdbc-postgresql</artifactId>
</dependency>
----

With Gradle :
[source,groovy, subs="attributes"]
----
// https://mvnrepository.com/artifact/com.google.cloud.sql/postgres-socket-factory
compile group: 'com.google.cloud.sql', name: 'postgres-socket-factory', version: '1.0.15'
// https://mvnrepository.com/artifact/io.quarkus/quarkus-jdbc-postgresql
compile group: 'io.quarkus', name: 'quarkus-jdbc-postgresql', version: '1.1.1.Final'
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved
----

== Configuring the CloudSQL instance

If you haven't already please follow the link:https://cloud.google.com/sql/docs/postgres/create-instance[Creating instances guide].

Ensure you have a service account with _Cloud SQL Client_ role (minimum necessary role), and get your instance connection name (`PROJECT_ID:REGION:INSTANCE_ID`).

== Configuring the application

Configure your quarkus application as follows :

[source,properties]
--
quarkus.datasource.url=jdbc:postgresql:///${DB_NAME}
quarkus.datasource.driver=org.postgresql.Driver
quarkus.datasource.username = ${DB_USER}
quarkus.datasource.password = ${DB_PASSWORD}
quarkus.datasource.socket-factory=com.google.cloud.sql.postgres.SocketFactory
quarkus.datasource.additional-jdbc-properties=cloud-sql-instance=project:zone:db-name
--

NOTE: To set the environment variables(DB_NAME, DB_USER, DB_PASSWORD) when deploying to CloudRun there is an option `ADD VARIABLE`
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved

WARNING: Don't forget to add your service account key to the deployed image, and to accept requests on the expected port

[source,docker]
--
FROM gcr.io/distroless/java:11
COPY target/*.jar /app/application-runner.jar
COPY <your_service_account_key>.json /app
WORKDIR /app
CMD ["application-runner.jar","-Dquarkus.http.host=0.0.0.0", "-Dquarkus.http.port=$PORT"]
--

NOTE: `$PORT` value will be injected automatically by CloudRun and is to be used for HealthCheck


You should now be able to connect to your instance.
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package io.quarkus.agroal.test;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

import java.sql.SQLException;
import java.time.Duration;

import javax.inject.Inject;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.agroal.api.AgroalDataSource;
import io.agroal.api.configuration.AgroalConnectionFactoryConfiguration;
import io.agroal.api.configuration.AgroalConnectionPoolConfiguration;
import io.agroal.narayana.NarayanaTransactionIntegration;
import io.quarkus.test.QuarkusUnitTest;

public class GcpDataSourceConfigTest {
gastaldi marked this conversation as resolved.
Show resolved Hide resolved

//tag::injection[]
@Inject
AgroalDataSource defaultDataSource;
//end::injection[]

@RegisterExtension
static final QuarkusUnitTest config = new QuarkusUnitTest()
.withConfigurationResource("application-gcp-datasource.properties");

@Test
public void testGCPDataSourceInjection() throws SQLException {
testDataSource(defaultDataSource, "username-default", 3, 13, 7, Duration.ofSeconds(53), Duration.ofSeconds(54),
Duration.ofSeconds(55), Duration.ofSeconds(56), Duration.ofSeconds(57),
"create schema if not exists schema_default");
}

private static void testDataSource(AgroalDataSource dataSource, String username, int minSize, int maxSize,
int initialSize, Duration backgroundValidationInterval, Duration acquisitionTimeout, Duration leakDetectionInterval,
Duration idleRemovalInterval, Duration maxLifetime, String newConnectionSql) {
AgroalConnectionPoolConfiguration configuration = dataSource.getConfiguration().connectionPoolConfiguration();
AgroalConnectionFactoryConfiguration agroalConnectionFactoryConfiguration = configuration
.connectionFactoryConfiguration();

assertTrue(agroalConnectionFactoryConfiguration.jdbcProperties().containsKey("socketFactory"));
assertEquals(agroalConnectionFactoryConfiguration.jdbcProperties().getProperty("socketFactory"),
"com.google.cloud.sql.postgres.SocketFactory");
assertTrue(agroalConnectionFactoryConfiguration.jdbcProperties().containsKey("randomProperty"));
assertEquals(agroalConnectionFactoryConfiguration.jdbcProperties().getProperty("randomProperty"), "myProperty");
assertEquals(username, agroalConnectionFactoryConfiguration.principal().getName());
assertEquals(minSize, configuration.minSize());
assertEquals(maxSize, configuration.maxSize());
assertEquals(initialSize, configuration.initialSize());
assertEquals(backgroundValidationInterval, configuration.validationTimeout());
assertEquals(acquisitionTimeout, configuration.acquisitionTimeout());
assertEquals(leakDetectionInterval, configuration.leakTimeout());
assertEquals(idleRemovalInterval, configuration.reapTimeout());
assertEquals(maxLifetime, configuration.maxLifetime());
assertTrue(configuration.transactionIntegration() instanceof NarayanaTransactionIntegration);
assertEquals(AgroalConnectionFactoryConfiguration.TransactionIsolation.SERIALIZABLE,
agroalConnectionFactoryConfiguration.jdbcTransactionIsolation());
assertTrue(agroalConnectionFactoryConfiguration.trackJdbcResources());
assertTrue(dataSource.getConfiguration().metricsEnabled());
assertEquals(newConnectionSql, agroalConnectionFactoryConfiguration.initialSql());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#tag::basic[]
quarkus.datasource.url=jdbc:h2:tcp://localhost/mem:default
quarkus.datasource.driver=org.h2.Driver
quarkus.datasource.username=username-default
quarkus.datasource.min-size=3
quarkus.datasource.max-size=13
quarkus.datasource.enable-metrics=true
#end::basic[]
quarkus.datasource.initial-size=7
quarkus.datasource.background-validation-interval=53
quarkus.datasource.acquisition-timeout=54
quarkus.datasource.leak-detection-interval=55
quarkus.datasource.idle-removal-interval=56
quarkus.datasource.max-lifetime=57
quarkus.datasource.transaction-isolation-level=serializable
quarkus.datasource.new-connection-sql=create schema if not exists schema_default
quarkus.datasource.socket-factory=com.google.cloud.sql.postgres.SocketFactory
quarkus.datasource.additional-jdbc-properties=cloud-sql-instance=project:zone:db-name,randomProperty=myProperty
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import java.sql.Driver;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;

Expand Down Expand Up @@ -201,6 +202,24 @@ public boolean isValid(Connection connection) {
poolConfiguration.maxLifetime(dataSourceRuntimeConfig.maxLifetime.get());
}

if (dataSourceRuntimeConfig.socketFactory.isPresent()) {
agroalConnectionFactoryConfigurationSupplier.jdbcProperty("socketFactory",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since socketFactory is a JDBC property, the best is to have it declared in the additionalJdbcProperties, like in:

quarkus.datasource.additional-jdbc-properties.socketFactory=foobar

dataSourceRuntimeConfig.socketFactory.get());
}

if (dataSourceRuntimeConfig.additionalJdbcProperties.isPresent()) {
String generalProperties = dataSourceRuntimeConfig.additionalJdbcProperties.get();
List<String> properties = Arrays.asList(generalProperties.split(","));
for (String property : properties) {
String[] propertyData = property.split("=");
if (propertyData.length == 2) {
agroalConnectionFactoryConfigurationSupplier.jdbcProperty(propertyData[0], propertyData[1]);
} else {
log.warnv("Property {0} is not correctly formatted", property);
}
}
}

// SSL support: we should push the driver specific code to the driver extensions but it will have to do for now
if (disableSslSupport) {
switch (driverName) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -124,4 +124,18 @@ public class DataSourceRuntimeConfig {
*/
@ConfigItem
public Optional<String> validationQuerySql;

/**
* The socket factory driver that should be appended as a datasource url property
*/
@ConfigItem
public Optional<String> socketFactory;
Comment on lines +128 to +133
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed since the configuration is set as a JDBC property and could live in the additionalJdbcProperties Map below


/**
* General JDBC properties that can configured for a specific JDBC connection
*
* Should be comma separated, example: <property_name1>=<property_value1>,<property_name2>=<property_value2>
*/
@ConfigItem
public Optional<String> additionalJdbcProperties;
Mihai-B marked this conversation as resolved.
Show resolved Hide resolved
}