Skip to content

Commit

Permalink
feat: instrument src/ with traces using OpenTelemetry
Browse files Browse the repository at this point in the history
This change implements tracing of RPC calls using
OpenTelemetry to aid in providing observability.

Fixes googleapis#2079

observability-test: more tests for startTrace
  • Loading branch information
odeke-em committed Aug 30, 2024
1 parent 992baaa commit a1576ca
Show file tree
Hide file tree
Showing 16 changed files with 3,473 additions and 1,549 deletions.
97 changes: 97 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Google APIs Client Libraries, in [Client Libraries Explained][explained].
* [Before you begin](#before-you-begin)
* [Installing the client library](#installing-the-client-library)
* [Using the client library](#using-the-client-library)
* [Observability](#observability)
* [Samples](#samples)
* [Versioning](#versioning)
* [Contributing](#contributing)
Expand Down Expand Up @@ -82,6 +83,101 @@ rows.forEach(row => console.log(row));
```


## Observability

This package has been instrumented with [OpenTelemetry](https://opentelemetry.io/docs/languages/js/) for tracing. Make sure to firstly import and enable
OpenTelemetry before importing this Spanner library.

Please use a tracer named "nodejs-spanner".

> :warning: **Make sure that the OpenTelemetry imports are the first, before importing the Spanner library**
> :warning: **In order for your spans to be annotated with SQL, you MUST opt-in by setting environment variable
`SPANNER_ENABLE_EXTENDED_TRACING=true`, this is because SQL statements can be
sensitive personally-identifiable-information (PII).**

To test out trace examination, you can use Google Cloud Trace like this.

```javascript
function exportSpans(instanceId, databaseId, projectId) {
// Firstly initiate OpenTelemetry
const {Resource} = require('@opentelemetry/resources');
const {NodeSDK} = require('@opentelemetry/sdk-node');
const {trace} = require('@opentelemetry/api');
const {
NodeTracerProvider,
TraceIdRatioBasedSampler,
} = require('@opentelemetry/sdk-trace-node');
const {BatchSpanProcessor} = require('@opentelemetry/sdk-trace-base');
const {
SEMRESATTRS_SERVICE_NAME,
SEMRESATTRS_SERVICE_VERSION,
} = require('@opentelemetry/semantic-conventions');

const resource = Resource.default().merge(
new Resource({
[SEMRESATTRS_SERVICE_NAME]: 'spanner-sample',
[SEMRESATTRS_SERVICE_VERSION]: 'v1.0.0', // Whatever version of your app is running.,
})
);

const {TraceExporter} = require('@google-cloud/opentelemetry-cloud-trace-exporter');
const exporter = new TraceExporter({});

const sdk = new NodeSDK({
resource: resource,
traceExporter: exporter,
// Trace every single request to ensure that we generate
// enough traffic for proper examination of traces.
sampler: new TraceIdRatioBasedSampler(1.0),
});
sdk.start();

const provider = new NodeTracerProvider({resource: resource});
provider.addSpanProcessor(new BatchSpanProcessor(exporter));
provider.register();

// OpenTelemetry MUST be imported much earlier than the cloud-spanner package.
const tracer = trace.getTracer('nodejs-spanner');

const {Spanner} = require('@google-cloud/spanner');

tracer.startActiveSpan('deleteAndCreateDatabase', span => {
// Creates a client
const spanner = new Spanner({
projectId: projectId,
});

// Gets a reference to a Cloud Spanner instance and database
const instance = spanner.instance(instanceId);
const database = instance.database(databaseId);
const databaseAdminClient = spanner.getDatabaseAdminClient();

const databasePath = databaseAdminClient.databasePath(
projectId,
instanceId,
databaseId
);

deleteDatabase(databaseAdminClient, databasePath, () => {
createDatabase(
databaseAdminClient,
projectId,
instanceId,
databaseId,
() => {
span.end();
console.log('main span.end');
setTimeout(() => {
console.log('finished delete and creation of the database');
}, 5000);
}
);
});
});
}
```


## Samples

Expand All @@ -90,6 +186,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-spanner/tre
| Sample | Source Code | Try it |
| --------------------------- | --------------------------------- | ------ |
| Add and drop new database role | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/add-and-drop-new-database-role.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/add-and-drop-new-database-role.js,samples/README.md) |
| Export traces & observability from this library | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/observability-traces.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/observability-traces.js,samples/README.md) |
| Backups-cancel | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-cancel.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-cancel.js,samples/README.md) |
| Copies a source backup | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-copy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-copy.js,samples/README.md) |
| Backups-create-with-encryption-key | [source code](https://github.com/googleapis/nodejs-spanner/blob/main/samples/backups-create-with-encryption-key.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-spanner&page=editor&open_in_editor=samples/backups-create-with-encryption-key.js,samples/README.md) |
Expand Down
155 changes: 155 additions & 0 deletions observability-test/grpc-instrumentation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
/*!
* Copyright 2024 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

const projectId = process.env.SPANNER_TEST_PROJECTID || 'test-project';

describe('gRPC instrumentation with sampling on', () => {
const assert = require('assert');
const {registerInstrumentations} = require('@opentelemetry/instrumentation');
const {SimpleSpanProcessor} = require('@opentelemetry/sdk-trace-base');
const {
AlwaysOnSampler,
InMemorySpanExporter,
NodeTracerProvider,
} = require('@opentelemetry/sdk-trace-node');
const {
AsyncHooksContextManager,
} = require('@opentelemetry/context-async-hooks');
const {GrpcInstrumentation} = require('@opentelemetry/instrumentation-grpc');
const fini = registerInstrumentations({
instrumentations: [new GrpcInstrumentation()],
});
const {startTrace} = require('../src/instrument');
const {
disableContextAndManager,
setGlobalContextManager,
} = require('./helper');
const {ContextManager} = require('@opentelemetry/api');

const {SEMATTRS_DB_SYSTEM} = require('@opentelemetry/semantic-conventions');

const exporter = new InMemorySpanExporter();
const sampler = new AlwaysOnSampler();

const {Database, Spanner} = require('../src');

let provider: typeof NodeTracerProvider;
let contextManager: typeof ContextManager;
let spanner: typeof Spanner;
let database: typeof Database;

beforeEach(async () => {
provider = new NodeTracerProvider({
sampler: sampler,
exporter: exporter,
});
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();

contextManager = new AsyncHooksContextManager();
setGlobalContextManager(contextManager);

spanner = new Spanner({
projectId: projectId,
});

const instance = spanner.instance('test-instance');
database = instance.database('test-db');

// Warm up session creation.
await database.run('SELECT 2');
// Mimick usual customer usage in which at setup time, the
// Spanner and Database handles are created once then sit
// and wait until they service HTTP or gRPC calls that
// come in say 5+ seconds after the service is fully started.
// This gives time for the batch session creation to be be completed.
await new Promise((resolve, reject) => setTimeout(resolve, 100));
});

afterEach(async () => {
spanner.close();
exporter.forceFlush();
exporter.reset();
await provider.shutdown();
disableContextAndManager(contextManager);
});

after(() => {
fini();
});

it('Invoking database methods creates spans: gRPC enabled', () => {
const tracer = provider.getTracer();

tracer.startActiveSpan('test', async span => {
const query = {sql: 'SELECT * FROM INFORMATION_SCHEMA.TABLES'};
const [rows] = await database.run(query);
assert.ok(rows.length > 1);

// Read from the rows until completion.
for (const row of rows) {
const _ = row.toJSON();
}

span.end();

await new Promise((resolve, reject) => setTimeout(resolve, 600));
await exporter.forceFlush();

// We need to ensure that spans were generated and exported.
const spans = exporter.getFinishedSpans();
assert.ok(spans.length > 0, 'at least 1 span must have been created');

// Sort the spans by duration, in the natural
// trace view order by longer duration first.
spans.sort((spanA, spanB) => {
return spanA.duration > spanB.duration;
});

const got: string[] = [];
spans.forEach(span => {
got.push(span.name);
});

const want = [
'cloud.google.com/nodejs/spanner/Database.run',
'cloud.google.com/nodejs/spanner/Database.runStream',
'grpc.google.spanner.v1.Spanner/ExecuteStreamingSql',
];

assert.deepEqual(
want,
got,
'The spans order by duration has been violated:\n\tGot: ' +
got.toString() +
'\n\tWant: ' +
want.toString()
);

// Ensure that each span has the attribute
// SEMATTRS_DB_SYSTEM, set to 'spanner'
spans.forEach(span => {
if (span.name.startsWith('cloud.google.com')) {
assert.equal(
span.attributes[SEMATTRS_DB_SYSTEM],
'spanner',
'Invalid DB_SYSTEM attribute'
);
}
});
});
});
});
Loading

0 comments on commit a1576ca

Please sign in to comment.