-
Notifications
You must be signed in to change notification settings - Fork 2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add PostgreSQL connection pooling sample. (#926)
* Add Postgresql connection pooling sample. * Lint. * Address review comments. * Address additional review comments. * Rename cloudsql directory to cloud-sql.
- Loading branch information
Showing
10 changed files
with
860 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# Format: //devtools/kokoro/config/proto/build.proto | ||
|
||
# Set the folder in which the tests are run | ||
env_vars: { | ||
key: "PROJECT" | ||
value: "cloudsql/postgres/knex" | ||
} | ||
|
||
# Tell the trampoline which build file to use. | ||
env_vars: { | ||
key: "TRAMPOLINE_BUILD_FILE" | ||
value: "github/nodejs-docs-samples/.kokoro/build.sh" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
# Connecting to Cloud SQL - PostgreSQL | ||
|
||
## Before you begin | ||
|
||
1. If you haven't already, set up a Node.js Development Environment by following the [Node.js setup guide](https://cloud.google.com/nodejs/docs/setup) and | ||
[create a project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#creating_a_project). | ||
|
||
2. Create a Cloud SQL for PostgreSQL instance by following these | ||
[instructions](https://cloud.google.com/sql/docs/postgres/create-instance). Note the instance name that you create, | ||
and password that you specify for the default 'postgres' user. | ||
|
||
* If you don't want to use the default user to connect, [create a user](https://cloud.google.com/sql/docs/postgres/create-manage-users#creating). | ||
|
||
3. Create a database for your application by following these [instructions](https://cloud.google.com/sql/docs/postgres/create-manage-databases). Note the database name. | ||
|
||
4. Create a service account with the 'Cloud SQL Client' permissions by following these | ||
[instructions](https://cloud.google.com/sql/docs/mysql/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account). | ||
Download a JSON key to use to authenticate your connection. | ||
|
||
|
||
5. Use the information noted in the previous steps: | ||
```bash | ||
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json | ||
export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>' | ||
export DB_USER='my-db-user' | ||
export DB_PASS='my-db-pass' | ||
export DB_NAME='my_db' | ||
``` | ||
Note: Saving credentials in environment variables is convenient, but not secure - consider a more | ||
secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe. | ||
|
||
## Running locally | ||
|
||
1. To run this application locally, download and install the `cloud_sql_proxy` by | ||
following the instructions [here](https://cloud.google.com/sql/docs/postgres/sql-proxy#install). | ||
|
||
Once the proxy is ready, use the following command to start the proxy in the | ||
background: | ||
```bash | ||
./cloud_sql_proxy -dir=/cloudsql -instances=$CLOUD_SQL_CONNECTION_NAME -credential_file=$GOOGLE_APPLICATION_CREDENTIALS | ||
``` | ||
Note: Make sure to run the command under a user with write access in the | ||
`/cloudsql` directory. This proxy will use this folder to create a unix socket | ||
the application will use to connect to Cloud SQL. | ||
|
||
2. Next, install the Node.js packages necessary to run the app locally by running the following command: | ||
|
||
``` | ||
npm install | ||
``` | ||
|
||
3. Run `createTable.js` to create the database table the app requires and to ensure that the database is properly configured. | ||
With the Cloud SQL proxy running, run the following command to create the sample app's table in your Cloud SQL PostgreSQL database: | ||
|
||
``` | ||
node createTable.js | ||
``` | ||
|
||
4. Run the sample app locally with the following command: | ||
|
||
``` | ||
npm start | ||
``` | ||
|
||
Navigate towards `http://127.0.0.1:8080` to verify your application is running correctly. | ||
|
||
## Deploy to Google App Engine Standard | ||
|
||
1. To allow your app to connect to your Cloud SQL instance when the app is deployed, add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the `app.standard.yaml` file. The deployed application will connect via unix sockets. | ||
|
||
``` | ||
env_variables: | ||
DB_USER: MY_DB_USER | ||
DB_PASS: MY_DB_PASSWORD | ||
DB_NAME: MY_DATABASE | ||
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
``` | ||
|
||
2. To deploy to App Engine Standard, run the following command: | ||
|
||
``` | ||
gcloud app deploy app.standard.yaml | ||
``` | ||
|
||
3. To launch your browser and view the app at https://[YOUR_PROJECT_ID].appspot.com, run the following command: | ||
|
||
``` | ||
gcloud app browse | ||
``` | ||
|
||
## Deploy to Google App Engine Flexible | ||
|
||
1. Add the user, password, database, and instance connection name variables from Cloud SQL to the related environment variables in the `app.flexible.yaml` file. The deployed application will connect via unix sockets. | ||
|
||
``` | ||
env_variables: | ||
DB_USER: MY_DB_USER | ||
DB_PASS: MY_DB_PASSWORD | ||
DB_NAME: MY_DATABASE | ||
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
``` | ||
|
||
2. To deploy to App Engine Node.js Flexible Environment, run the following command: | ||
|
||
``` | ||
gcloud app deploy app.flexible.yaml | ||
``` | ||
|
||
3. To launch your browser and view the app at https://[YOUR_PROJECT_ID].appspot.com, run the following command: | ||
|
||
``` | ||
gcloud app browse | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Copyright 2018, Google, Inc. | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
runtime: nodejs | ||
env: flex | ||
|
||
# The following env variables may contain sensitive information that grants | ||
# anyone access to your database. Do not add this file to your source control. | ||
env_variables: | ||
DB_USER: MY_DB_USER | ||
DB_PASS: MY_DB_PASSWORD | ||
DB_NAME: MY_DATABASE | ||
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
CLOUD_SQL_INSTANCE_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
|
||
beta_settings: | ||
# The connection name of your instance, available by using | ||
# 'gcloud beta sql instances describe [INSTANCE_NAME]' or from | ||
# the Instance details page in the Google Cloud Platform Console. | ||
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Copyright 2018, Google, Inc. | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
runtime: nodejs8 | ||
|
||
# The following env variables may contain sensitive information that grants | ||
# anyone access to your database. Do not add this file to your source control. | ||
env_variables: | ||
DB_USER: MY_DB_USER | ||
DB_PASS: MY_DB_PASSWORD | ||
DB_NAME: MY_DATABASE | ||
# e.g. my-awesome-project:us-central1:my-cloud-sql-instance | ||
CLOUD_SQL_INSTANCE_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> | ||
|
||
beta_settings: | ||
# The connection name of your instance, available by using | ||
# 'gcloud beta sql instances describe [INSTANCE_NAME]' or from | ||
# the Instance details page in the Google Cloud Platform Console. | ||
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
/** | ||
* Copyright 2018 Google LLC. | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
'use strict'; | ||
|
||
const Knex = require('knex'); | ||
const prompt = require('prompt'); | ||
|
||
const FIELDS = ['user', 'password', 'database']; | ||
|
||
prompt.start(); | ||
|
||
// Prompt the user for connection details | ||
prompt.get(FIELDS, (err, config) => { | ||
if (err) { | ||
console.error(err); | ||
return; | ||
} | ||
|
||
// Connect to the database | ||
const knex = Knex({client: 'pg', connection: config}); | ||
|
||
// Create the "votes" table | ||
knex.schema | ||
.createTable('votes', table => { | ||
table.bigIncrements('vote_id').notNull(); | ||
table.timestamp('time_cast').notNull(); | ||
table.specificType('candidate', 'CHAR(6) NOT NULL'); | ||
}) | ||
.then(() => { | ||
console.log(`Successfully created 'votes' table.`); | ||
return knex.destroy(); | ||
}) | ||
.catch(err => { | ||
console.error(`Failed to create 'votes' table:`, err); | ||
if (knex) { | ||
knex.destroy(); | ||
} | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
{ | ||
"name": "appengine-cloudsql-postgres", | ||
"description": "Node.js PostgreSQL sample for Cloud SQL on App Engine.", | ||
"version": "0.0.1", | ||
"private": true, | ||
"license": "Apache-2.0", | ||
"author": "Google Inc.", | ||
"repository": { | ||
"type": "git", | ||
"url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git" | ||
}, | ||
"engines": { | ||
"node": ">=8" | ||
}, | ||
"scripts": { | ||
"unit-test": "ava --verbose test/*.test.js", | ||
"start-proxy": "! pgrep cloud_sql_proxy > /dev/null && cloud_sql_proxy -dir=/cloudsql -instances=$CLOUD_SQL_INSTANCE_NAME &", | ||
"system-test": "repo-tools test app -- server.js", | ||
"system-test-proxy": "npm run start-proxy; npm run system-test", | ||
"all-test": "npm run unit-test && npm run system-test", | ||
"test": "repo-tools test run --cmd npm -- run all-test" | ||
}, | ||
"dependencies": { | ||
"async": "2.6.0", | ||
"body-parser": "1.18.2", | ||
"express": "4.16.2", | ||
"knex": "0.14.4", | ||
"pg": "7.4.1", | ||
"prompt": "1.0.0", | ||
"pug": "2.0.0-rc.4", | ||
"@google-cloud/logging-winston": "^0.10.2", | ||
"winston": "^3.1.0" | ||
}, | ||
"devDependencies": { | ||
"@google-cloud/nodejs-repo-tools": "3.0.0", | ||
"ava": "0.25.0", | ||
"proxyquire": "^2.1.0", | ||
"supertest": "^3.3.0", | ||
"sinon": "^7.1.1" | ||
}, | ||
"cloud-repo-tools": { | ||
"requiresKeyFile": true, | ||
"requiresProjectId": true, | ||
"test": { | ||
"app": { | ||
"requiredEnvVars": [ | ||
"DB_USER", | ||
"DB_PASS", | ||
"DB_NAME", | ||
"CLOUD_SQL_INSTANCE_NAME" | ||
], | ||
"args": [ | ||
"server.js" | ||
] | ||
}, | ||
"build": { | ||
"requiredEnvVars": [ | ||
"DB_USER", | ||
"DB_PASS", | ||
"DB_NAME", | ||
"CLOUD_SQL_INSTANCE_NAME" | ||
] | ||
} | ||
} | ||
} | ||
} |
Oops, something went wrong.