Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PRMP-1188] Create Lambda to handle MNS notifications #212

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
0b59e79
[PRMP-1185] create infra, permissions and policies for mns sqs
steph-torres-nhs Nov 14, 2024
3f86c33
[prmp-1185] format
NogaNHS Nov 15, 2024
dfb62dc
[prmp-1185] add ssm
NogaNHS Nov 15, 2024
f28e837
[prmp-1185] terraform errors fix
NogaNHS Nov 15, 2024
0000650
[prmp-1185] terraform missing condtion
NogaNHS Nov 15, 2024
b0fcb12
PRMP-1185 remove enbale deduplications
NogaNHS Nov 15, 2024
cd8667a
PRMP-1185 resoulving conflict
NogaNHS Nov 15, 2024
4fd72e7
PRMP-1185 wrong index
NogaNHS Nov 15, 2024
9bfef62
PRMP-1185 change false to null
NogaNHS Nov 15, 2024
874234e
PRMP-1185 change policy to aws_sqs_queue_policy
NogaNHS Nov 15, 2024
50c4dd5
[prmp-1185] change source to a list
NogaNHS Nov 15, 2024
a9b3318
[prmp-1185] change source to a list in kms
NogaNHS Nov 15, 2024
262151c
[prmp-1185] remove spaces
NogaNHS Nov 15, 2024
bf781b3
[PRMP-1188] inital commit
steph-torres-nhs Nov 20, 2024
84bc2f5
[PRMP-1188] setup mns-notification-lambda
steph-torres-nhs Nov 20, 2024
ffa1d13
[PRMP-1188] add kms policy to mns notification lambda
steph-torres-nhs Nov 25, 2024
b65e762
[PRMP-1188] correct attribute on lambda kms access policy
steph-torres-nhs Nov 25, 2024
dd93696
[PRMP-1188] resolve PR comments
steph-torres-nhs Nov 25, 2024
4cdc9ba
[PRMP-1188] fix typo
steph-torres-nhs Nov 26, 2024
69c7623
[PRMP-1188] remove sns ref after mns change
NogaNHS Nov 27, 2024
cce2a55
merge in main
steph-torres-nhs Nov 27, 2024
b5cf198
[PRMP-1188] address PR comments
steph-torres-nhs Nov 28, 2024
a133f57
[PRMP-1188] update lambda iam role policies following refactor
steph-torres-nhs Nov 29, 2024
201bebe
Merge branch 'refs/heads/main' into PRMP-1188
NogaNHS Dec 10, 2024
921dc06
[PRMP-1188] change iam policies
NogaNHS Dec 10, 2024
6f84dbb
[PRMP-1188] change iam_role_policy to iam_role_policy_documents
NogaNHS Dec 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion infrastructure/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.62.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.66.0 |

## Modules

Expand Down Expand Up @@ -84,6 +84,7 @@
| <a name="module_logout_alarm"></a> [logout\_alarm](#module\_logout\_alarm) | ./modules/lambda_alarms | n/a |
| <a name="module_logout_alarm_topic"></a> [logout\_alarm\_topic](#module\_logout\_alarm\_topic) | ./modules/sns | n/a |
| <a name="module_logout_lambda"></a> [logout\_lambda](#module\_logout\_lambda) | ./modules/lambda | n/a |
| <a name="module_mns_encryption_key"></a> [mns\_encryption\_key](#module\_mns\_encryption\_key) | ./modules/kms | n/a |
| <a name="module_ndr-app-config"></a> [ndr-app-config](#module\_ndr-app-config) | ./modules/app_config | n/a |
| <a name="module_ndr-bulk-staging-store"></a> [ndr-bulk-staging-store](#module\_ndr-bulk-staging-store) | ./modules/s3/ | n/a |
| <a name="module_ndr-docker-ecr-ui"></a> [ndr-docker-ecr-ui](#module\_ndr-docker-ecr-ui) | ./modules/ecr/ | n/a |
Expand Down Expand Up @@ -116,6 +117,7 @@
| <a name="module_sns_encryption_key"></a> [sns\_encryption\_key](#module\_sns\_encryption\_key) | ./modules/kms | n/a |
| <a name="module_sqs-lg-bulk-upload-invalid-queue"></a> [sqs-lg-bulk-upload-invalid-queue](#module\_sqs-lg-bulk-upload-invalid-queue) | ./modules/sqs | n/a |
| <a name="module_sqs-lg-bulk-upload-metadata-queue"></a> [sqs-lg-bulk-upload-metadata-queue](#module\_sqs-lg-bulk-upload-metadata-queue) | ./modules/sqs | n/a |
| <a name="module_sqs-mns-notification-queue"></a> [sqs-mns-notification-queue](#module\_sqs-mns-notification-queue) | ./modules/sqs | n/a |
| <a name="module_sqs-nems-queue"></a> [sqs-nems-queue](#module\_sqs-nems-queue) | ./modules/sqs | n/a |
| <a name="module_sqs-splunk-queue"></a> [sqs-splunk-queue](#module\_sqs-splunk-queue) | ./modules/sqs | n/a |
| <a name="module_statistical-report-alarm"></a> [statistical-report-alarm](#module\_statistical-report-alarm) | ./modules/lambda_alarms | n/a |
Expand Down Expand Up @@ -233,6 +235,7 @@
| [aws_security_group.ndr_mesh_sg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_sns_topic.alarm_notifications_topic](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) | resource |
| [aws_sns_topic_subscription.alarm_notifications_sns_topic_subscription](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription) | resource |
| [aws_sqs_queue_policy.mns_sqs_access_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) | resource |
| [aws_sqs_queue_policy.nems_events_subscription](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) | resource |
| [aws_ssm_parameter.nems_events_observability](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource |
| [aws_ssm_parameter.nems_events_topic_arn](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) | resource |
Expand Down Expand Up @@ -263,6 +266,8 @@
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
| [aws_ssm_parameter.backup_target_account](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) | data source |
| [aws_ssm_parameter.cloud_security_notification_email_list](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) | data source |
| [aws_ssm_parameter.mns_lambda_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) | data source |
| [aws_ssm_parameter.mns_sns_resources](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) | data source |
| [aws_ssm_parameter.splunk_trusted_principal](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) | data source |
| [aws_ssm_parameter.target_backup_vault_arn](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) | data source |

Expand Down
2 changes: 1 addition & 1 deletion infrastructure/kms_sns.tf
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ module "sns_encryption_key" {
current_account_id = data.aws_caller_identity.current.account_id
environment = var.environment
owner = var.owner
identifiers = ["sns.amazonaws.com", "cloudwatch.amazonaws.com"]
service_identifiers = ["sns.amazonaws.com", "cloudwatch.amazonaws.com"]
}
93 changes: 93 additions & 0 deletions infrastructure/lamba-mns-notification.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
module "mns-notification-lambda" {
source = "./modules/lambda"
name = "MNSNotificationLambda"
handler = "handlers.mns_notification_handler.lambda_handler"
iam_role_policies = [
oliverbeumkes-nhs marked this conversation as resolved.
Show resolved Hide resolved
"arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
"arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy",
module.sqs-mns-notification-queue.sqs_policy,
module.lloyd_george_reference_dynamodb_table.dynamodb_policy,
aws_iam_policy.ssm_access_policy.arn,
module.ndr-app-config.app_config_policy_arn
]
rest_api_id = null
api_execution_arn = null

lambda_environment_variables = {
APPCONFIG_APPLICATION = module.ndr-app-config.app_config_application_id
APPCONFIG_ENVIRONMENT = module.ndr-app-config.app_config_environment_id
APPCONFIG_CONFIGURATION = module.ndr-app-config.app_config_configuration_profile_id
WORKSPACE = terraform.workspace
LLOYD_GEORGE_DYNAMODB_NAME = "${terraform.workspace}_${var.lloyd_george_dynamodb_table_name}"
MNS_NOTIFICATION_QUEUE_URL = module.sqs-mns-notification-queue.sqs_url
PDS_FHIR_IS_STUBBED = local.is_sandbox
}

is_gateway_integration_needed = false
is_invoked_from_gateway = false
lambda_timeout = 900
reserved_concurrent_executions = local.mns_notification_lambda_concurrent_limit

depends_on = [
oliverbeumkes-nhs marked this conversation as resolved.
Show resolved Hide resolved
module.lloyd_george_reference_dynamodb_table,
aws_iam_policy.ssm_access_policy,
module.ndr-app-config,
module.sqs-mns-notification-queue
]
}

resource "aws_lambda_event_source_mapping" "mns_notification_lambda" {
event_source_arn = module.sqs-mns-notification-queue.endpoint
function_name = module.mns-notification-lambda.lambda_arn

scaling_config {
maximum_concurrency = local.mns_notification_lambda_concurrent_limit
}

depends_on = [
oliverbeumkes-nhs marked this conversation as resolved.
Show resolved Hide resolved
module.mns-notification-lambda,
module.sqs-mns-notification-queue
]
}

module "mns-notification-alarm" {
source = "./modules/lambda_alarms"
lambda_function_name = module.mns-notification-lambda.function_name
lambda_timeout = module.mns-notification-lambda.timeout
lambda_name = "mns_notification_handler"
namespace = "AWS/Lambda"
alarm_actions = [module.mns-notification-alarm-topic.arn]
ok_actions = [module.mns-notification-alarm-topic.arn]
depends_on = [module.mns-notification-lambda, module.mns-notification-alarm-topic]
oliverbeumkes-nhs marked this conversation as resolved.
Show resolved Hide resolved
}

module "mns-notification-alarm-topic" {
source = "./modules/sns"
sns_encryption_key_id = module.sns_encryption_key.id
current_account_id = data.aws_caller_identity.current.account_id
topic_name = "mns-notification-topic"
topic_protocol = "lambda"
topic_endpoint = module.mns-notification-lambda.lambda_arn
delivery_policy = jsonencode({
"Version" : "2012-10-17",
"Statement" : [
{
"Effect" : "Allow",
"Principal" : {
"Service" : "cloudwatch.amazonaws.com"
},
"Action" : [
"SNS:Publish",
],
"Condition" : {
"ArnLike" : {
"aws:SourceArn" : "arn:aws:cloudwatch:eu-west-2:${data.aws_caller_identity.current.account_id}:alarm:*"
}
}
"Resource" : "*"
}
]
})

depends_on = [module.mns-notification-lambda, module.sns_encryption_key]
oliverbeumkes-nhs marked this conversation as resolved.
Show resolved Hide resolved
}
68 changes: 68 additions & 0 deletions infrastructure/mns.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
data "aws_ssm_parameter" "mns_lambda_role" {
name = "/ndr/${var.environment}/mns/lambda_role"
}

data "aws_ssm_parameter" "mns_sns_resources" {
name = "/ndr/${var.environment}/mns/sns_resource"
}

locals {
mns_sns_source_arns = split(",", data.aws_ssm_parameter.mns_sns_resources.value)
}

module "mns_encryption_key" {
source = "./modules/kms"
kms_key_name = "alias/mns-notification-encryption-key-kms-${terraform.workspace}"
kms_key_description = "Custom KMS Key to enable server side encryption for mns subscriptions"
current_account_id = data.aws_caller_identity.current.account_id
environment = var.environment
owner = var.owner
service_identifiers = ["sns.amazonaws.com"]
aws_identifiers = [data.aws_ssm_parameter.mns_lambda_role.value]
allow_decrypt_for_arn = true
allowed_arn = local.mns_sns_source_arns
}

module "sqs-mns-notification-queue" {
source = "./modules/sqs"
name = "mns-notification-queue"
max_size_message = 256 * 1024 # allow message size up to 256 KB
message_retention = 60 * 60 * 24 * 14 # 14 days
environment = var.environment
owner = var.owner
max_visibility = 1020
delay = 60
enable_sse = null
kms_master_key_id = module.mns_encryption_key.id
}

resource "aws_sqs_queue_policy" "mns_sqs_access_policy" {
oliverbeumkes-nhs marked this conversation as resolved.
Show resolved Hide resolved
queue_url = module.sqs-mns-notification-queue.sqs_url

policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow",
Principal = {
Service = "sns.amazonaws.com"
},
Action = "SQS:SendMessage",
Resource = module.sqs-mns-notification-queue.sqs_arn,
Condition = {
"StringEquals" = {
"aws:SourceArn" = local.mns_sns_source_arns
}
}
},
{
Effect = "Allow",
Principal = {
AWS = data.aws_ssm_parameter.mns_lambda_role.value
},
Action = "SQS:SendMessage",
Resource = module.sqs-mns-notification-queue.sqs_arn
}
]
})
}
9 changes: 7 additions & 2 deletions infrastructure/modules/kms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,24 @@ No modules.
|------|------|
| [aws_kms_alias.encryption_key_alias](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_alias) | resource |
| [aws_kms_key.encryption_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
| [aws_iam_policy_document.kms_key_policy_doc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.combined_policy_documents](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.kmn_key_base](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.kms_key_generate](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_allow_decrypt_for_arn"></a> [allow\_decrypt\_for\_arn](#input\_allow\_decrypt\_for\_arn) | n/a | `bool` | `false` | no |
| <a name="input_allowed_arn"></a> [allowed\_arn](#input\_allowed\_arn) | n/a | `list(string)` | `[]` | no |
| <a name="input_aws_identifiers"></a> [aws\_identifiers](#input\_aws\_identifiers) | n/a | `list(string)` | `[]` | no |
| <a name="input_current_account_id"></a> [current\_account\_id](#input\_current\_account\_id) | n/a | `string` | n/a | yes |
| <a name="input_environment"></a> [environment](#input\_environment) | n/a | `string` | n/a | yes |
| <a name="input_identifiers"></a> [identifiers](#input\_identifiers) | n/a | `list(string)` | n/a | yes |
| <a name="input_kms_key_description"></a> [kms\_key\_description](#input\_kms\_key\_description) | n/a | `string` | n/a | yes |
| <a name="input_kms_key_name"></a> [kms\_key\_name](#input\_kms\_key\_name) | n/a | `string` | n/a | yes |
| <a name="input_kms_key_rotation_enabled"></a> [kms\_key\_rotation\_enabled](#input\_kms\_key\_rotation\_enabled) | n/a | `bool` | `true` | no |
| <a name="input_owner"></a> [owner](#input\_owner) | n/a | `string` | n/a | yes |
| <a name="input_service_identifiers"></a> [service\_identifiers](#input\_service\_identifiers) | n/a | `list(string)` | n/a | yes |

## Outputs

Expand Down
37 changes: 33 additions & 4 deletions infrastructure/modules/kms/main.tf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
resource "aws_kms_key" "encryption_key" {
description = var.kms_key_description
policy = data.aws_iam_policy_document.kms_key_policy_doc.json
policy = data.aws_iam_policy_document.combined_policy_documents.json
enable_key_rotation = var.kms_key_rotation_enabled

tags = {
Expand All @@ -17,7 +17,7 @@ resource "aws_kms_alias" "encryption_key_alias" {
}


data "aws_iam_policy_document" "kms_key_policy_doc" {
data "aws_iam_policy_document" "kmn_key_base" {
steph-torres-nhs marked this conversation as resolved.
Show resolved Hide resolved
statement {
effect = "Allow"
principals {
Expand All @@ -30,13 +30,42 @@ data "aws_iam_policy_document" "kms_key_policy_doc" {
statement {
effect = "Allow"
principals {
identifiers = var.identifiers
identifiers = var.service_identifiers
type = "Service"
}
actions = [
"kms:Decrypt",
"kms:GenerateDataKey*"
]
resources = ["*"]
dynamic "condition" {
for_each = var.allow_decrypt_for_arn ? [1] : []
content {
test = "ArnEquals"
values = var.allowed_arn
variable = "aws:SourceArn"
}
}
}
}

data "aws_iam_policy_document" "kms_key_generate" {
count = length(var.aws_identifiers) > 0 ? 1 : 0
statement {
effect = "Allow"
principals {
identifiers = var.aws_identifiers
type = "AWS"
}
actions = ["kms:GenerateDataKey"]
resources = ["*"]
}
}
}

data "aws_iam_policy_document" "combined_policy_documents" {
source_policy_documents = flatten([
data.aws_iam_policy_document.kmn_key_base.json,
length(var.aws_identifiers) > 0 ? [data.aws_iam_policy_document.kms_key_generate[0].json] : []
])
}

17 changes: 16 additions & 1 deletion infrastructure/modules/kms/variable.tf
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,25 @@ variable "owner" {
type = string
}

variable "identifiers" {
variable "service_identifiers" {
type = list(string)
}

variable "aws_identifiers" {
type = list(string)
default = []
}

variable "allow_decrypt_for_arn" {
type = bool
default = false
}

variable "allowed_arn" {
type = list(string)
default = []
}

output "kms_arn" {
value = aws_kms_key.encryption_key.arn
}
Expand Down
1 change: 0 additions & 1 deletion infrastructure/queues.tf
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,3 @@ module "sqs-lg-bulk-upload-invalid-queue" {
owner = var.owner
max_visibility = 1020
}

3 changes: 2 additions & 1 deletion infrastructure/variable.tf
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,8 @@ locals {
is_force_destroy = contains(["ndr-dev", "ndra", "ndrb", "ndrc", "ndrd", "ndr-test"], terraform.workspace)
is_sandbox_or_test = contains(["ndra", "ndrb", "ndrc", "ndrd", "ndr-test"], terraform.workspace)

bulk_upload_lambda_concurrent_limit = 5
bulk_upload_lambda_concurrent_limit = 5
mns_notification_lambda_concurrent_limit = 3

api_gateway_subdomain_name = contains(["prod"], terraform.workspace) ? "${var.certificate_subdomain_name_prefix}" : "${var.certificate_subdomain_name_prefix}${terraform.workspace}"
api_gateway_full_domain_name = contains(["prod"], terraform.workspace) ? "${var.certificate_subdomain_name_prefix}${var.domain}" : "${var.certificate_subdomain_name_prefix}${terraform.workspace}.${var.domain}"
Expand Down
2 changes: 1 addition & 1 deletion virusscanner/terraform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.46.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | 5.43.0 |

## Modules

Expand Down