Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(neptune): enable cloudwatch logs exports #22004

Merged
merged 9 commits into from
Sep 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions packages/@aws-cdk/aws-neptune/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,38 @@ new neptune.DatabaseCluster(this, 'Cluster', {
});
```

## Logging

Neptune supports various methods for monitoring performance and usage. One of those methods is logging

1. Neptune provides logs e.g. audit logs which can be viewed or downloaded via the AWS Console. Audit logs can be enabled using the `neptune_enable_audit_log` parameter in `ClusterParameterGroup` or `ParameterGroup`
2. Neptune provides the ability to export those logs to CloudWatch Logs

```ts
// Cluster parameter group with the neptune_enable_audit_log param set to 1
const clusterParameterGroup = new neptune.ClusterParameterGroup(this, 'ClusterParams', {
description: 'Cluster parameter group',
parameters: {
neptune_enable_audit_log: '1'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be cool if we could somehow do this automatically, but it doesn't look like we have that capability in these constructs currently. I'll consider it out of scope, but just wanted to leave a note.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely, it'd be gr8 if that's handled by default. But to achieve that in a nice fashion, I think

  1. Maybe the cluster needs to create default cluster and instance parameter groups
  2. Expose methods to allow setting parameters on those groups - while also refactoring parameter groups to support exposing what parameters are there and adding/overriding the existing ones

All in all, maybe an idea for a separate PR

},
});

const cluster = new neptune.DatabaseCluster(this, 'Database', {
vpc,
instanceType: neptune.InstanceType.R5_LARGE,
// Audit logs are enabled via the clusterParameterGroup
clusterParameterGroup,
// Optionally configuring audit logs to be exported to CloudWatch Logs
cloudwatchLogsExports: [neptune.LogType.AUDIT],
// Optionally set a retention period on exported CloudWatch Logs
cloudwatchLogsRetention: logs.RetentionDays.ONE_MONTH,
});
```

For more information on monitoring, refer to https://docs.aws.amazon.com/neptune/latest/userguide/monitoring.html.
For more information on audit logs, refer to https://docs.aws.amazon.com/neptune/latest/userguide/auditing.html.
For more information on exporting logs to CloudWatch Logs, refer to https://docs.aws.amazon.com/neptune/latest/userguide/cloudwatch-logs.html.

## Metrics

Both `DatabaseCluster` and `DatabaseInstance` provide a `metric()` method to help with cluster-level and instance-level monitoring.
Expand Down
63 changes: 63 additions & 0 deletions packages/@aws-cdk/aws-neptune/lib/cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import * as cloudwatch from '@aws-cdk/aws-cloudwatch';
import * as ec2 from '@aws-cdk/aws-ec2';
import * as iam from '@aws-cdk/aws-iam';
import * as kms from '@aws-cdk/aws-kms';
import * as logs from '@aws-cdk/aws-logs';
import { Aws, Duration, IResource, Lazy, RemovalPolicy, Resource, Token } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { Endpoint } from './endpoint';
Expand Down Expand Up @@ -71,6 +72,26 @@ export class EngineVersion {
public constructor(public readonly version: string) {}
}

/**
* Neptune log types that can be exported to CloudWatch logs
*
* @see https://docs.aws.amazon.com/neptune/latest/userguide/cloudwatch-logs.html
*/
export class LogType {
/**
* Audit logs
*
* @see https://docs.aws.amazon.com/neptune/latest/userguide/auditing.html
*/
public static readonly AUDIT = new LogType('audit');

/**
* Constructor for specifying a custom log type
* @param value the log type
*/
public constructor(public readonly value: string) {}
}

/**
* Properties for a new database cluster
*/
Expand Down Expand Up @@ -243,6 +264,34 @@ export interface DatabaseClusterProps {
* @default - false
*/
readonly autoMinorVersionUpgrade?: boolean;

/**
* The list of log types that need to be enabled for exporting to
* CloudWatch Logs.
*
* @see https://docs.aws.amazon.com/neptune/latest/userguide/cloudwatch-logs.html
* @see https://docs.aws.amazon.com/neptune/latest/userguide/auditing.html#auditing-enable
*
* @default - no log exports
*/
readonly cloudwatchLogsExports?: LogType[];

/**
* The number of days log events are kept in CloudWatch Logs. When updating
* this property, unsetting it doesn't remove the log retention policy. To
* remove the retention policy, set the value to `Infinity`.
*
* @default - logs never expire
*/
readonly cloudwatchLogsRetention?: logs.RetentionDays;

/**
* The IAM role for the Lambda function associated with the custom resource
* that sets the retention policy.
*
* @default - a new role is created.
*/
readonly cloudwatchLogsRetentionRole?: iam.IRole;
}

/**
Expand Down Expand Up @@ -529,6 +578,8 @@ export class DatabaseCluster extends DatabaseClusterBase implements IDatabaseClu
preferredMaintenanceWindow: props.preferredMaintenanceWindow,
// Encryption
kmsKeyId: props.kmsKey?.keyArn,
// CloudWatch Logs exports
enableCloudwatchLogsExports: props.cloudwatchLogsExports?.map(logType => logType.value),
storageEncrypted,
});

Expand All @@ -543,6 +594,18 @@ export class DatabaseCluster extends DatabaseClusterBase implements IDatabaseClu
this.clusterEndpoint = new Endpoint(cluster.attrEndpoint, port);
this.clusterReadEndpoint = new Endpoint(cluster.attrReadEndpoint, port);

// Log retention
const retention = props.cloudwatchLogsRetention;
if (retention) {
props.cloudwatchLogsExports?.forEach(logType => {
new logs.LogRetention(this, `${logType}LogRetention`, {
logGroupName: `/aws/neptune/${this.clusterIdentifier}/${logType.value}`,
role: props.cloudwatchLogsRetentionRole,
retention,
});
});
}

// Create the instances
const instanceCount = props.instances ?? DatabaseCluster.DEFAULT_NUM_INSTANCES;
if (instanceCount < 1) {
Expand Down
2 changes: 2 additions & 0 deletions packages/@aws-cdk/aws-neptune/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@
"@aws-cdk/aws-ec2": "0.0.0",
"@aws-cdk/aws-iam": "0.0.0",
"@aws-cdk/aws-kms": "0.0.0",
"@aws-cdk/aws-logs": "0.0.0",
"@aws-cdk/core": "0.0.0",
"constructs": "^10.0.0"
},
Expand All @@ -102,6 +103,7 @@
"@aws-cdk/aws-ec2": "0.0.0",
"@aws-cdk/aws-iam": "0.0.0",
"@aws-cdk/aws-kms": "0.0.0",
"@aws-cdk/aws-logs": "0.0.0",
"@aws-cdk/core": "0.0.0",
"constructs": "^10.0.0"
},
Expand Down
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-neptune/rosetta/default.ts-fixture
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { Duration, Stack } from '@aws-cdk/core';
import { Construct } from 'constructs';
import * as iam from '@aws-cdk/aws-iam';
import * as ec2 from '@aws-cdk/aws-ec2';
import * as logs from '@aws-cdk/aws-logs';
import * as neptune from '@aws-cdk/aws-neptune';

class Fixture extends Stack {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"path": "Tree",
"constructInfo": {
"fqn": "constructs.Construct",
"version": "10.1.92"
"version": "10.1.95"
}
},
"aws-cdk-neptune-integ": {
Expand Down Expand Up @@ -1026,7 +1026,7 @@
"path": "ClusterTest/DefaultTest/Default",
"constructInfo": {
"fqn": "constructs.Construct",
"version": "10.1.92"
"version": "10.1.95"
}
},
"DeployAssert": {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export declare function handler(event: AWSLambda.CloudFormationCustomResourceEvent, context: AWSLambda.Context): Promise<void>;

Large diffs are not rendered by default.

Loading