Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Spanner copy backup #1530

Merged
merged 9 commits into from
Mar 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-spanner/tre
| Sample | Source Code | Try it |
| --------------------------- | --------------------------------- | ------ |
| Backups-cancel | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-cancel.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-cancel.js,samples/README.md) |
| Copies a source backup | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-copy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-copy.js,samples/README.md) |
| Backups-create-with-encryption-key | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-create-with-encryption-key.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-create-with-encryption-key.js,samples/README.md) |
| Backups-create | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-create.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-create.js,samples/README.md) |
| Backups-delete | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-delete.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-delete.js,samples/README.md) |
Expand Down
18 changes: 18 additions & 0 deletions samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ and automatic, synchronous replication for high availability.
* [Before you begin](#before-you-begin)
* [Samples](#samples)
* [Backups-cancel](#backups-cancel)
* [Copies a source backup](#copies-a-source-backup)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rest of the links are "-" separated. Maybe change to [Copy-source-backup]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is because this is in the new format of test. As you can see on line 34-36, this is acceptable

* [Backups-create-with-encryption-key](#backups-create-with-encryption-key)
* [Backups-create](#backups-create)
* [Backups-delete](#backups-delete)
Expand Down Expand Up @@ -95,6 +96,23 @@ __Usage:__



### Copies a source backup

View the [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-copy.js).

[![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-copy.js,samples/README.md)

__Usage:__


`node spannerCopyBackup <INSTANCE_ID> <COPY_BACKUP_ID> <SOURCE_BACKUP_ID> <PROJECT_ID>`


-----




### Backups-create-with-encryption-key

View the [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-create-with-encryption-key.js).
Expand Down
97 changes: 97 additions & 0 deletions samples/backups-copy.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// sample-metadata:
// title: Copies a source backup
// usage: node spannerCopyBackup <INSTANCE_ID> <COPY_BACKUP_ID> <SOURCE_BACKUP_ID> <PROJECT_ID>

'use strict';

function main(
instanceId = 'my-instance',
backupId = 'my-backup',
sourceBackupPath = 'projects/my-project-id/instances/my-source-instance/backups/my-source-backup',
projectId = 'my-project-id'
) {
// [START spanner_copy_backup]
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// const instanceId = 'my-instance';
// const backupId = 'my-backup',
// const sourceBackupPath = 'projects/my-project-id/instances/my-source-instance/backups/my-source-backup',
// const projectId = 'my-project-id';

// Imports the Google Cloud Spanner client library
const {Spanner} = require('@google-cloud/spanner');
const {PreciseDate} = require('@google-cloud/precise-date');

// Instantiates a client
const spanner = new Spanner({
projectId: projectId,
});

async function spannerCopyBackup() {
// Gets a reference to a Cloud Spanner instance and backup
const instance = spanner.instance(instanceId);

// Expire copy backup 14 days in the future
const expireTime = Spanner.timestamp(
Date.now() + 1000 * 60 * 60 * 24 * 14
).toStruct();

// Copy the source backup
try {
console.log(`Creating copy of the source backup ${sourceBackupPath}.`);
const [, operation] = await instance.copyBackup(
sourceBackupPath,
backupId,
{
expireTime: expireTime,
}
);

console.log(
`Waiting for backup copy ${
instance.backup(backupId).formattedName_
} to complete...`
);
await operation.promise();

// Verify the copy backup is ready
const copyBackup = instance.backup(backupId);
const [copyBackupInfo] = await copyBackup.getMetadata();
if (copyBackupInfo.state === 'READY') {
console.log(
`Backup copy ${copyBackupInfo.name} of size ` +
`${copyBackupInfo.sizeBytes} bytes was created at ` +
`${new PreciseDate(copyBackupInfo.createTime).toISOString()} ` +
'with version time ' +
`${new PreciseDate(copyBackupInfo.versionTime).toISOString()}`
);
} else {
console.error('ERROR: Copy of backup is not ready.');
}
} catch (err) {
console.error('ERROR:', err);
}
}
spannerCopyBackup();
// [END spanner_copy_backup]
}
process.on('unhandledRejection', err => {
console.error(err.message);
process.exitCode = 1;
});
main(...process.argv.slice(2));
36 changes: 34 additions & 2 deletions samples/backups-get-operations.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@

'use strict';

async function getBackupOperations(instanceId, databaseId, projectId) {
async function getBackupOperations(
instanceId,
databaseId,
backupId,
projectId
) {
// [START spanner_list_backup_operations]
// Imports the Google Cloud client library
const {Spanner, protos} = require('@google-cloud/spanner');
Expand All @@ -25,6 +30,7 @@ async function getBackupOperations(instanceId, databaseId, projectId) {
*/
// const projectId = 'my-project-id';
// const databaseId = 'my-database';
// const backupId = 'my-backup';
// const instanceId = 'my-instance';

// Creates a client
Expand All @@ -35,7 +41,7 @@ async function getBackupOperations(instanceId, databaseId, projectId) {
// Gets a reference to a Cloud Spanner instance
const instance = spanner.instance(instanceId);

// List backup operations
// List create backup operations
try {
const [backupOperations] = await instance.getBackupOperations({
filter:
Expand All @@ -56,6 +62,32 @@ async function getBackupOperations(instanceId, databaseId, projectId) {
} catch (err) {
console.error('ERROR:', err);
}

// List copy backup operations
try {
console.log(
'(metadata.@type:type.googleapis.com/google.spanner.admin.database.v1.CopyBackupMetadata) ' +
`AND (metadata.source_backup:${backupId})`
);
const [backupOperations] = await instance.getBackupOperations({
filter:
'(metadata.@type:type.googleapis.com/google.spanner.admin.database.v1.CopyBackupMetadata) ' +
`AND (metadata.source_backup:${backupId})`,
});
console.log('Copy Backup Operations:');
backupOperations.forEach(backupOperation => {
const metadata =
protos.google.spanner.admin.database.v1.CopyBackupMetadata.decode(
backupOperation.metadata.value
);
console.log(
`Backup ${metadata.name} copied from source backup ${metadata.sourceBackup} is ` +
`${metadata.progress.progressPercent}% complete.`
);
});
} catch (err) {
console.error('ERROR:', err);
}
// [END spanner_list_backup_operations]
}

Expand Down
9 changes: 7 additions & 2 deletions samples/backups-update.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,13 @@ async function updateBackup(instanceId, backupId, projectId) {
// Read backup metadata and update expiry time
try {
const currentExpireTime = await backup.getExpireTime();
const newExpireTime = new PreciseDate(currentExpireTime);
newExpireTime.setDate(newExpireTime.getDate() + 30);
const maxExpireTime = backup.metadata.maxExpireTime;
const wantExpireTime = new PreciseDate(currentExpireTime);
wantExpireTime.setDate(wantExpireTime.getDate() + 1);
// New expire time should be less than the max expire time
const min = (currentExpireTime, maxExpireTime) =>
currentExpireTime < maxExpireTime ? currentExpireTime : maxExpireTime;
const newExpireTime = new PreciseDate(min(wantExpireTime, maxExpireTime));
console.log(
`Backup ${backupId} current expire time: ${currentExpireTime.toISOString()}`
);
Expand Down
9 changes: 7 additions & 2 deletions samples/backups.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,16 @@ require('yargs')
)
)
.command(
'getBackupOperations <instanceName> <databaseName> <projectId>',
'getBackupOperations <instanceName> <databaseName> <backupName> <projectId>',
'Lists all backup operations in the instance.',
{},
opts =>
getBackupOperations(opts.instanceName, opts.databaseName, opts.projectId)
getBackupOperations(
opts.instanceName,
opts.databaseName,
opts.backupName,
opts.projectId
)
)
.command(
'getDatabaseOperations <instanceName> <projectId>',
Expand Down
22 changes: 21 additions & 1 deletion samples/system-test/spanner.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ const VERSION_RETENTION_DATABASE_ID = `test-database-${CURRENT_TIME}-v`;
const ENCRYPTED_DATABASE_ID = `test-database-${CURRENT_TIME}-enc`;
const DEFAULT_LEADER_DATABASE_ID = `test-database-${CURRENT_TIME}-dl`;
const BACKUP_ID = `test-backup-${CURRENT_TIME}`;
const COPY_BACKUP_ID = `test-copy-backup-${CURRENT_TIME}`;
const ENCRYPTED_BACKUP_ID = `test-backup-${CURRENT_TIME}-enc`;
const CANCELLED_BACKUP_ID = `test-backup-${CURRENT_TIME}-c`;
const LOCATION_ID = 'regional-us-west1';
Expand Down Expand Up @@ -214,6 +215,7 @@ describe('Spanner', () => {
await Promise.all([
instance.backup(BACKUP_ID).delete(GAX_OPTIONS),
instance.backup(ENCRYPTED_BACKUP_ID).delete(GAX_OPTIONS),
instance.backup(COPY_BACKUP_ID).delete(GAX_OPTIONS),
instance.backup(CANCELLED_BACKUP_ID).delete(GAX_OPTIONS),
]);
await instance.delete(GAX_OPTIONS);
Expand All @@ -223,6 +225,7 @@ describe('Spanner', () => {
instance.database(RESTORE_DATABASE_ID).delete(),
instance.database(ENCRYPTED_RESTORE_DATABASE_ID).delete(),
instance.backup(BACKUP_ID).delete(GAX_OPTIONS),
instance.backup(COPY_BACKUP_ID).delete(GAX_OPTIONS),
instance.backup(ENCRYPTED_BACKUP_ID).delete(GAX_OPTIONS),
instance.backup(CANCELLED_BACKUP_ID).delete(GAX_OPTIONS),
]);
Expand Down Expand Up @@ -1021,6 +1024,18 @@ describe('Spanner', () => {
assert.include(output, `using encryption key ${key.name}`);
});

// copy_backup
it('should create a copy of a backup', async () => {
const sourceBackupPath = `projects/${PROJECT_ID}/instances/${INSTANCE_ID}/backups/${BACKUP_ID}`;
const output = execSync(
`node backups-copy.js ${INSTANCE_ID} ${COPY_BACKUP_ID} ${sourceBackupPath} ${PROJECT_ID}`
);
assert.match(
output,
new RegExp(`(.*)Backup copy(.*)${COPY_BACKUP_ID} of size(.*)`)
);
});

// cancel_backup
it('should cancel a backup of the database', async () => {
const output = execSync(
Expand Down Expand Up @@ -1048,13 +1063,18 @@ describe('Spanner', () => {
// list_backup_operations
it('should list backup operations in the instance', async () => {
const output = execSync(
`${backupsCmd} getBackupOperations ${INSTANCE_ID} ${DATABASE_ID} ${PROJECT_ID}`
`${backupsCmd} getBackupOperations ${INSTANCE_ID} ${DATABASE_ID} ${BACKUP_ID} ${PROJECT_ID}`
);
assert.match(output, /Create Backup Operations:/);
assert.match(
output,
new RegExp(`Backup (.+)${BACKUP_ID} (.+) is 100% complete`)
);
assert.match(output, /Copy Backup Operations:/);
assert.match(
output,
new RegExp(`Backup (.+)${COPY_BACKUP_ID} (.+) is 100% complete`)
);
});

// update_backup_expire_time
Expand Down
Loading