diff --git a/docs/admin-guide.md b/docs/admin-guide.md index 37c81c357..388616810 100644 --- a/docs/admin-guide.md +++ b/docs/admin-guide.md @@ -331,7 +331,7 @@ Tag Policies are available only in an organization that has [all features enable ## Integrating Slack - +### Integrating with Slack using Lambda The ADF allows alternate *notification_endpoint* values that can be used to notify the status of a specific pipeline *(in deployment_map.yml)*. You can specify an email address in the deployment map and notifications will be emailed directly to that address. However, if you specify a slack channel name *(eg team-bugs)* as the value, the notifications will be forwarded to that channel. In order to setup this integration you will need to create a [Slack App](https://api.slack.com/apps). When you create your Slack app, you can create multiple Webhook URL's *(Incoming Webhook)* that are each associated with their own channel. Create a webhook for each channel you plan on using throughout your Organization. Once created, copy the webhook URL and create a new secret in Secrets Manager on the Deployment Account: 1. In AWS Console, click _Store a new secret_ and select type 'Other type of secrets' *(eg API Key)*. @@ -362,6 +362,22 @@ pipelines: name: omg_production ``` +### Integrating with Slack with AWS ChatBot +The ADF also supports integrating pipeline notifications with Slack via the AWS ChatBot. This allows pipeline notifications to scale and provides a consistent Slack notification across different AWS services. + +In order to use AWS ChatBot, first you must configure an (AWS ChatBot Client)[https://us-east-2.console.aws.amazon.com/chatbot/home?region=eu-west-1#/chat-clients] for your desired Slack workspace. Once the client has been created. You will need to manually create a channel configuration that will be used by the ADF. + +Currently, dynamically creating channel configurations is not supported. In the deployment map, you can configure a unique channel via the notification endpoint parameter for each pipeline separately. Add the `params` section if that is missing and add the following configuration to the pipeline: +``` +pipelines: + - name: some-pipeline + # ... + params: + notification_endpoint: + type: chat_bot + target: my_channel_config +``` + ## Check Current Version To determine the current version, follow these steps: diff --git a/docs/user-guide.md b/docs/user-guide.md index 0d238e646..667b72baf 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -195,8 +195,16 @@ Pipelines also have parameters that don't relate to a specific stage but rather The following are the available pipeline parameters: -- *notification_endpoint* *(String)* defaults to none. - > Can either be a valid email address or a string that represents the name of a Slack Channel. In order to integrate ADF with Slack see [Integrating with Slack](./admin-guide.md) in the admin guide. By Default, Notifications will be sent when pipelines Start, Complete or Fail. +- *notification_endpoint* *(String) | (Dict) * defaults to none. + > Can either be a valid email address or a string that represents the name of a Slack Channel. + > A more complex configuration can be provided to integrate with Slack via AWS ChatBot. + > ```yaml + > notification_endpoint: + > type: chat_bot + > target: example_slack_channel # This is the name of an slack channel configuration you created within the AWS Chat Bot service. This needs to be created before you apply the changes to the deployment map. + > ``` + > + > In order to integrate ADF with Slack see [Integrating with Slack](./admin-guide.md#integrating-with-slack-with-aws-chatbot) in the admin guide. By default, notifications will be sent when pipelines Start, Complete, or Fail. - *schedule* *(String)* defaults to none. > If the Pipeline should execute on a specific Schedule. Schedules are defined by using a Rate or an Expression. See [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#RateExpressions) for more information on how to define Rate or an Expression. diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_chatbot.py b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_chatbot.py new file mode 100644 index 000000000..2db87476d --- /dev/null +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_chatbot.py @@ -0,0 +1,68 @@ +# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT-0 + +"""Construct related to Notifications Codepipeline Input +""" + +import os +from aws_cdk import ( + aws_codestarnotifications as cp_notifications, + aws_codepipeline as codepipeline, + core, +) +from logger import configure_logger + +ADF_DEPLOYMENT_REGION = os.environ["AWS_REGION"] +ADF_DEPLOYMENT_ACCOUNT_ID = os.environ["ACCOUNT_ID"] + +LOGGER = configure_logger(__name__) + +EVENT_TYPE_IDS = [ + "codepipeline-pipeline-stage-execution-succeeded", + "codepipeline-pipeline-stage-execution-failed", + "codepipeline-pipeline-pipeline-execution-started", + "codepipeline-pipeline-pipeline-execution-failed", + "codepipeline-pipeline-pipeline-execution-succeeded", + "codepipeline-pipeline-manual-approval-needed", + "codepipeline-pipeline-manual-approval-succeeded", +] + + +class PipelineNotifications(core.Construct): + def __init__( + self, + scope: core.Construct, + id: str, + pipeline: codepipeline.CfnPipeline, + notification_config, + **kwargs, + ): # pylint: disable=W0622 + super().__init__(scope, id, **kwargs) + slack_channel_arn = f"arn:aws:chatbot::{ADF_DEPLOYMENT_ACCOUNT_ID}:chat-configuration/slack-channel/{notification_config.get('target')}" + pipeline_arn = f"arn:aws:codepipeline:{ADF_DEPLOYMENT_REGION}:{ADF_DEPLOYMENT_ACCOUNT_ID}:{pipeline.ref}" + cp_notifications.CfnNotificationRule( + scope, + "pipeline-notification", + detail_type="FULL", + event_type_ids=EVENT_TYPE_IDS, + name=pipeline.ref, + resource=pipeline_arn, + targets=[ + cp_notifications.CfnNotificationRule.TargetProperty( + target_type=PipelineNotifications.get_target_type_from_config( + scope, notification_config + ), + target_address=slack_channel_arn, + ) + ], + ) + + @staticmethod + def get_target_type_from_config(scope, config): + target_type = config.get("type", "chat_bot") + if target_type == "chat_bot": + return "AWSChatbotSlack" + scope.node.add_error( + f"{target_type} is not supported for CodePipeline notifications." + ) + return None diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_notifications.py b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_notifications.py index ce44dfb7d..ef5ef1390 100644 --- a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_notifications.py +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_notifications.py @@ -21,7 +21,9 @@ class Notifications(core.Construct): - def __init__(self, scope: core.Construct, id: str, map_params: dict, **kwargs): #pylint: disable=W0622 + def __init__( + self, scope: core.Construct, id: str, map_params: dict, **kwargs + ): # pylint: disable=W0622 super().__init__(scope, id, **kwargs) LOGGER.debug('Notification configuration required for %s', map_params['name']) stack = core.Stack.of(self) @@ -32,42 +34,36 @@ def __init__(self, scope: core.Construct, id: str, map_params: dict, **kwargs): f'arn:{stack.partition}:lambda:{ADF_DEPLOYMENT_REGION}:' f'{ADF_DEPLOYMENT_ACCOUNT_ID}:function:SendSlackNotification' ) - _topic = _sns.Topic(self, 'PipelineTopic') + _topic = _sns.Topic(self, "PipelineTopic") _statement = _iam.PolicyStatement( actions=["sns:Publish"], effect=_iam.Effect.ALLOW, principals=[ - _iam.ServicePrincipal( - 'sns.amazonaws.com' - ), - _iam.ServicePrincipal( - 'codecommit.amazonaws.com' - ), - _iam.ServicePrincipal( - 'events.amazonaws.com' - ) + _iam.ServicePrincipal("sns.amazonaws.com"), + _iam.ServicePrincipal("codecommit.amazonaws.com"), + _iam.ServicePrincipal("events.amazonaws.com"), ], - resources=["*"] + resources=["*"], ) _topic.add_to_resource_policy(_statement) - _lambda.CfnPermission( - self, - 'slack_notification_sns_permissions', - principal='sns.amazonaws.com', - action='lambda:InvokeFunction', - source_arn=_topic.topic_arn, - function_name='SendSlackNotification' - ) - _endpoint = map_params.get('params', {}).get('notification_endpoint', '') + _endpoint = map_params.get("params", {}).get("notification_endpoint", "") _sub = _sns.Subscription( self, - 'sns_subscription', + "sns_subscription", topic=_topic, - endpoint=_endpoint if '@' in _endpoint else _slack_func.function_arn, - protocol=_sns.SubscriptionProtocol.EMAIL if '@' in _endpoint else _sns.SubscriptionProtocol.LAMBDA + endpoint=_endpoint if "@" in _endpoint else _slack_func.function_arn, + protocol=_sns.SubscriptionProtocol.EMAIL + if "@" in _endpoint + else _sns.SubscriptionProtocol.LAMBDA, ) - if '@' not in _endpoint: - _slack_func.add_event_source( - source=_event_sources.SnsEventSource(_topic) + if "@" not in _endpoint: + _lambda.CfnPermission( + self, + "slack_notification_sns_permissions", + principal="sns.amazonaws.com", + action="lambda:InvokeFunction", + source_arn=_topic.topic_arn, + function_name="SendSlackNotification", ) + _slack_func.add_event_source(source=_event_sources.SnsEventSource(_topic)) self.topic_arn = _topic.topic_arn diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/adf_default_pipeline.py b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/adf_default_pipeline.py index 6ca6889c3..1d0457162 100644 --- a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/adf_default_pipeline.py +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/adf_default_pipeline.py @@ -16,6 +16,7 @@ from cdk_constructs import adf_s3 from cdk_constructs import adf_cloudformation from cdk_constructs import adf_notifications +from cdk_constructs import adf_chatbot from logger import configure_logger ADF_DEPLOYMENT_REGION = os.environ["AWS_REGION"] @@ -29,10 +30,10 @@ def generate_adf_default_pipeline(scope: core.Stack, stack_input): _stages = [] - if stack_input["input"].get("params", {}).get("notification_endpoint"): - stack_input["input"]["topic_arn"] = adf_notifications.Notifications( - scope, "adf_notifications", stack_input["input"] - ).topic_arn + notification_config = stack_input["input"].get("params", {}).get("notification_endpoint", {}) + + if isinstance(notification_config, str) or notification_config.get('type', '') == "lambda": + stack_input["input"]["topic_arn"] = adf_notifications.Notifications(scope, "adf_notifications", stack_input["input"]).topic_arn _source_name = generate_source_stage_for_pipeline(_stages, scope, stack_input) generate_build_stage_for_pipeline(_stages, scope, stack_input) @@ -41,9 +42,12 @@ def generate_adf_default_pipeline(scope: core.Stack, stack_input): _pipeline = adf_codepipeline.Pipeline( scope, "code_pipeline", stack_input["input"], stack_input["ssm_params"], _stages ) + if "github" in _source_name: adf_github.GitHub.create_webhook_when_required(scope, _pipeline.cfn, stack_input["input"]) + if isinstance(notification_config, dict) and notification_config.get('type', '') == 'chat_bot': + adf_chatbot.PipelineNotifications(scope, "adf_chatbot_notifications", _pipeline.cfn, notification_config) def generate_source_stage_for_pipeline(_stages, scope, stack_input): _source_name = stack_input["input"]["default_providers"]["source"][ diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_pipeline_creation.py b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_pipeline_creation.py index aa080fc43..c1c421f9a 100644 --- a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_pipeline_creation.py +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_pipeline_creation.py @@ -140,3 +140,50 @@ def test_pipeline_creation_outputs_as_expected_when_source_is_codecommit_and_bui assert build_stage_action['ActionTypeId']['Provider'] == "CodeBuild" assert len(build_stage['Actions']) == 1 + +def test_pipeline_creation_outputs_as_expected_when_notification_endpoint_is_chatbot(): + region_name = "eu-central-1" + acount_id = "123456789012" + + stack_input = { + "input": {"params": {"notification_endpoint": {"target": "fake-config", "type": "chat_bot"}}, "default_providers": {}, "regions": {}, }, + "ssm_params": {"fake-region": {}}, + } + + stack_input["input"]["name"] = "test-stack" + + stack_input["input"]["default_providers"]["source"] = { + "provider": "codecommit", + "properties": {"account_id": "123456789012"}, + } + stack_input["input"]["default_providers"]["build"] = { + "provider": "codebuild", + "properties": {"account_id": "123456789012"}, + } + + stack_input["ssm_params"][region_name] = { + "modules": "fake-bucket-name", + "kms": f"arn:aws:kms:{region_name}:{acount_id}:key/my-unique-kms-key-id", + } + app = core.App() + PipelineStack(app, stack_input) + + cloud_assembly = app.synth() + resources = {k[0:-8]: v for k, v in cloud_assembly.stacks[0].template['Resources'].items()} + pipeline_notification = resources['pipelinenoti']['Properties'] + + target = pipeline_notification["Targets"][0] + + assert resources["pipelinenoti"]["Type"] == "AWS::CodeStarNotifications::NotificationRule" + assert target["TargetAddress"] == "arn:aws:chatbot::111111111111:chat-configuration/slack-channel/fake-config" + assert target["TargetType"] == "AWSChatbotSlack" + assert pipeline_notification["EventTypeIds"] == [ + "codepipeline-pipeline-stage-execution-succeeded", + "codepipeline-pipeline-stage-execution-failed", + "codepipeline-pipeline-pipeline-execution-started", + "codepipeline-pipeline-pipeline-execution-failed", + "codepipeline-pipeline-pipeline-execution-succeeded", + "codepipeline-pipeline-manual-approval-needed", + "codepipeline-pipeline-manual-approval-succeeded" + ] + assert pipeline_notification["DetailType"] == "FULL" diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/requirements.txt b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/requirements.txt index 9ca942cf8..d5ad17136 100644 --- a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/requirements.txt +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/requirements.txt @@ -6,6 +6,7 @@ aws-cdk.aws-autoscaling==1.114 aws-cdk.aws-autoscaling_common==1.114 aws-cdk.aws-autoscaling_hooktargets==1.114 aws-cdk.aws-certificatemanager==1.114 +aws-cdk.aws-chatbot==1.114 aws-cdk.aws-cloudformation==1.114 aws-cdk.aws-cloudfront==1.114 aws-cdk.aws-cloudwatch==1.114 diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/schema_validation.py b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/schema_validation.py index 90d22dad9..e62450468 100644 --- a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/schema_validation.py +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/schema_validation.py @@ -10,9 +10,14 @@ LOGGER = configure_logger(__name__) +NOTIFICATION_PROPS = { + Optional("target"): str, + Optional("type") : Or("lambda", "chat_bot") +} + # Pipeline Params PARAM_SCHEMA = { - Optional("notification_endpoint"): str, + Optional("notification_endpoint"): Or(str, NOTIFICATION_PROPS), Optional("schedule"): str, Optional("restart_execution_on_update"): bool, Optional("pipeline_type", default="default"): Or("default"),