Skip to content

Commit

Permalink
Add PostgreSQL connection pooling sample. (#926)
Browse files Browse the repository at this point in the history
* Add Postgresql connection pooling sample.

* Lint.

* Address review comments.

* Address additional review comments.

* Rename cloudsql directory to cloud-sql.
  • Loading branch information
jsimonweb authored and kurtisvg committed Dec 3, 2018
1 parent 2c9a920 commit cea8b0d
Show file tree
Hide file tree
Showing 10 changed files with 860 additions and 0 deletions.
13 changes: 13 additions & 0 deletions .kokoro/postgres-knex.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Format: //devtools/kokoro/config/proto/build.proto

# Set the folder in which the tests are run
env_vars: {
key: "PROJECT"
value: "cloudsql/postgres/knex"
}

# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/nodejs-docs-samples/.kokoro/build.sh"
}
116 changes: 116 additions & 0 deletions cloud-sql/postgres/knex/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Connecting to Cloud SQL - PostgreSQL

## Before you begin

1. If you haven't already, set up a Node.js Development Environment by following the [Node.js setup guide](https://cloud.google.com/nodejs/docs/setup) and
[create a project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#creating_a_project).

2. Create a Cloud SQL for PostgreSQL instance by following these
[instructions](https://cloud.google.com/sql/docs/postgres/create-instance). Note the instance name that you create,
and password that you specify for the default 'postgres' user.

* If you don't want to use the default user to connect, [create a user](https://cloud.google.com/sql/docs/postgres/create-manage-users#creating).

3. Create a database for your application by following these [instructions](https://cloud.google.com/sql/docs/postgres/create-manage-databases). Note the database name.

4. Create a service account with the 'Cloud SQL Client' permissions by following these
[instructions](https://cloud.google.com/sql/docs/mysql/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account).
Download a JSON key to use to authenticate your connection.


5. Use the information noted in the previous steps:
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>'
export DB_USER='my-db-user'
export DB_PASS='my-db-pass'
export DB_NAME='my_db'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe.

## Running locally

1. To run this application locally, download and install the `cloud_sql_proxy` by
following the instructions [here](https://cloud.google.com/sql/docs/postgres/sql-proxy#install).

Once the proxy is ready, use the following command to start the proxy in the
background:
```bash
./cloud_sql_proxy -dir=/cloudsql -instances=$CLOUD_SQL_CONNECTION_NAME -credential_file=$GOOGLE_APPLICATION_CREDENTIALS
```
Note: Make sure to run the command under a user with write access in the
`/cloudsql` directory. This proxy will use this folder to create a unix socket
the application will use to connect to Cloud SQL.

2. Next, install the Node.js packages necessary to run the app locally by running the following command:

```
npm install
```

3. Run `createTable.js` to create the database table the app requires and to ensure that the database is properly configured.
With the Cloud SQL proxy running, run the following command to create the sample app's table in your Cloud SQL PostgreSQL database:

```
node createTable.js
```

4. Run the sample app locally with the following command:

```
npm start
```

Navigate towards `http://127.0.0.1:8080` to verify your application is running correctly.

## Deploy to Google App Engine Standard

1. To allow your app to connect to your Cloud SQL instance when the app is deployed, add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the `app.standard.yaml` file. The deployed application will connect via unix sockets.

```
env_variables:
DB_USER: MY_DB_USER
DB_PASS: MY_DB_PASSWORD
DB_NAME: MY_DATABASE
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
```

2. To deploy to App Engine Standard, run the following command:

```
gcloud app deploy app.standard.yaml
```

3. To launch your browser and view the app at https://[YOUR_PROJECT_ID].appspot.com, run the following command:

```
gcloud app browse
```

## Deploy to Google App Engine Flexible

1. Add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the `app.flexible.yaml` file. The deployed application will connect via unix sockets.

```
env_variables:
DB_USER: MY_DB_USER
DB_PASS: MY_DB_PASSWORD
DB_NAME: MY_DATABASE
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
```

2. To deploy to App Engine Node.js Flexible Environment, run the following command:

```
gcloud app deploy app.flexible.yaml
```

3. To launch your browser and view the app at https://[YOUR_PROJECT_ID].appspot.com, run the following command:

```
gcloud app browse
```

30 changes: 30 additions & 0 deletions cloud-sql/postgres/knex/app.flexible.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Copyright 2018, Google, Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

runtime: nodejs
env: flex

# The following env variables may contain sensitive information that grants
# anyone access to your database. Do not add this file to your source control.
env_variables:
DB_USER: MY_DB_USER
DB_PASS: MY_DB_PASSWORD
DB_NAME: MY_DATABASE
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance
CLOUD_SQL_INSTANCE_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>

beta_settings:
# The connection name of your instance, available by using
# 'gcloud beta sql instances describe [INSTANCE_NAME]' or from
# the Instance details page in the Google Cloud Platform Console.
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
29 changes: 29 additions & 0 deletions cloud-sql/postgres/knex/app.standard.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Copyright 2018, Google, Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

runtime: nodejs8

# The following env variables may contain sensitive information that grants
# anyone access to your database. Do not add this file to your source control.
env_variables:
DB_USER: MY_DB_USER
DB_PASS: MY_DB_PASSWORD
DB_NAME: MY_DATABASE
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance
CLOUD_SQL_INSTANCE_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>

beta_settings:
# The connection name of your instance, available by using
# 'gcloud beta sql instances describe [INSTANCE_NAME]' or from
# the Instance details page in the Google Cloud Platform Console.
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
52 changes: 52 additions & 0 deletions cloud-sql/postgres/knex/createTable.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/**
* Copyright 2018 Google LLC.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

'use strict';

const Knex = require('knex');
const prompt = require('prompt');

const FIELDS = ['user', 'password', 'database'];

prompt.start();

// Prompt the user for connection details
prompt.get(FIELDS, (err, config) => {
if (err) {
console.error(err);
return;
}

// Connect to the database
const knex = Knex({client: 'pg', connection: config});

// Create the "votes" table
knex.schema
.createTable('votes', table => {
table.bigIncrements('vote_id').notNull();
table.timestamp('time_cast').notNull();
table.specificType('candidate', 'CHAR(6) NOT NULL');
})
.then(() => {
console.log(`Successfully created 'votes' table.`);
return knex.destroy();
})
.catch(err => {
console.error(`Failed to create 'votes' table:`, err);
if (knex) {
knex.destroy();
}
});
});
66 changes: 66 additions & 0 deletions cloud-sql/postgres/knex/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
{
"name": "appengine-cloudsql-postgres",
"description": "Node.js PostgreSQL sample for Cloud SQL on App Engine.",
"version": "0.0.1",
"private": true,
"license": "Apache-2.0",
"author": "Google Inc.",
"repository": {
"type": "git",
"url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
},
"engines": {
"node": ">=8"
},
"scripts": {
"unit-test": "ava --verbose test/*.test.js",
"start-proxy": "! pgrep cloud_sql_proxy > /dev/null && cloud_sql_proxy -dir=/cloudsql -instances=$CLOUD_SQL_INSTANCE_NAME &",
"system-test": "repo-tools test app -- server.js",
"system-test-proxy": "npm run start-proxy; npm run system-test",
"all-test": "npm run unit-test && npm run system-test",
"test": "repo-tools test run --cmd npm -- run all-test"
},
"dependencies": {
"async": "2.6.0",
"body-parser": "1.18.2",
"express": "4.16.2",
"knex": "0.14.4",
"pg": "7.4.1",
"prompt": "1.0.0",
"pug": "2.0.0-rc.4",
"@google-cloud/logging-winston": "^0.10.2",
"winston": "^3.1.0"
},
"devDependencies": {
"@google-cloud/nodejs-repo-tools": "3.0.0",
"ava": "0.25.0",
"proxyquire": "^2.1.0",
"supertest": "^3.3.0",
"sinon": "^7.1.1"
},
"cloud-repo-tools": {
"requiresKeyFile": true,
"requiresProjectId": true,
"test": {
"app": {
"requiredEnvVars": [
"DB_USER",
"DB_PASS",
"DB_NAME",
"CLOUD_SQL_INSTANCE_NAME"
],
"args": [
"server.js"
]
},
"build": {
"requiredEnvVars": [
"DB_USER",
"DB_PASS",
"DB_NAME",
"CLOUD_SQL_INSTANCE_NAME"
]
}
}
}
}
Loading

0 comments on commit cea8b0d

Please sign in to comment.