Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Development Work) Initial Creation of DMS Endpoints for Audit #7441

Closed
wants to merge 184 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
184 commits
Select commit Hold shift + click to select a range
e3f82fd
Add new role
bill-buchan Jul 12, 2024
a60be3e
Merge branch 'main' into DBA-699
bill-buchan Jul 12, 2024
2054168
Noop
bill-buchan Jul 12, 2024
823b263
Correct database for use as source endpoint for user data
bill-buchan Jul 12, 2024
252d8d3
Add Oracle Port
bill-buchan Jul 12, 2024
4302ed3
Add privileges to list and add objects to the DMS target bucket
bill-buchan Jul 12, 2024
aa174d9
Fix role name
bill-buchan Jul 12, 2024
6313ba1
Must index the role due to count
bill-buchan Jul 12, 2024
18be86f
Policy must also be indexed
bill-buchan Jul 12, 2024
006de7d
Grant privileges on destination s3 bucket
bill-buchan Jul 15, 2024
8c36d83
New policyto allow assuming DMS Client Bucket Role
bill-buchan Jul 15, 2024
6b331aa
Add debug facility
bill-buchan Jul 15, 2024
6710e4f
Quotes are required
bill-buchan Jul 15, 2024
d51ef3a
Who is the terraform user
bill-buchan Jul 15, 2024
5360b50
Allow terraform to assume role to list buckets
bill-buchan Jul 15, 2024
4c802e0
Remove use of external provider
bill-buchan Jul 23, 2024
fffca04
Merge branch 'main' into DBA-699
bill-buchan Jul 23, 2024
7726f81
Merge branch 'main' into DBA-699
bill-buchan Jul 26, 2024
d168acc
Merge branch 'main' into DBA-699
bill-buchan Jul 30, 2024
68a46c8
Add new SSM Parameter for storing location of the DMS S3 target bucket
bill-buchan Jul 30, 2024
52031b1
Store bucket name
bill-buchan Jul 31, 2024
83797ae
Add Resource Share for SSM Parameter
bill-buchan Jul 31, 2024
20a62d0
The repository account ID is a singleton list
bill-buchan Jul 31, 2024
6f598c4
Convert SSM parameter to secret
bill-buchan Jul 31, 2024
c77022c
Merge branch 'main' into DBA-699
bill-buchan Jul 31, 2024
f423f8a
Do not apply policy to non-existing principals
bill-buchan Jul 31, 2024
967fa50
Remove null elements from the list
bill-buchan Jul 31, 2024
35e5529
Client Account IDs are not sensitive
bill-buchan Jul 31, 2024
43fe6c0
Replace client ARNs with IDs
bill-buchan Jul 31, 2024
05c5eb9
Interim
bill-buchan Aug 2, 2024
b6897be
Exclude null elements
bill-buchan Aug 5, 2024
fc318a5
Principal is role and not a service
bill-buchan Aug 5, 2024
2749b5a
Require the bucket name
bill-buchan Aug 5, 2024
79867b3
DMS must assume local role
bill-buchan Aug 5, 2024
ee4c073
Consolidate DMS related outputs
bill-buchan Aug 5, 2024
0355ef5
Do not create s3 endpoint if target role does not exist
bill-buchan Aug 5, 2024
e86696a
DMS S3 writer role should be environment (not account) specific
bill-buchan Aug 6, 2024
a43d6f4
Development
bill-buchan Aug 6, 2024
cfec427
Capture the environment name
bill-buchan Aug 6, 2024
1c79660
Use the environment name
bill-buchan Aug 6, 2024
08954b4
Unnest data
bill-buchan Aug 6, 2024
b70eeb3
Merge role exists maps
bill-buchan Aug 6, 2024
96d185b
Exclude non-matches
bill-buchan Aug 6, 2024
7220def
Collect writer roles from other accounts
bill-buchan Aug 6, 2024
f5e9be5
Only include writer accounts
bill-buchan Aug 6, 2024
95bc742
add ldap rbac version parameter
Jul 22, 2024
ab9a9ba
allow ecs to retrieve ssm params
Jul 31, 2024
1422998
Merge branch 'main' into DBA-699
bill-buchan Aug 6, 2024
958074f
Merge branch 'DBA-699' of https://github.com/ministryofjustice/modern…
bill-buchan Aug 6, 2024
87df66f
Merge branch 'main' into DBA-699
bill-buchan Aug 6, 2024
de76581
Merge branch 'main' into DBA-699
bill-buchan Aug 6, 2024
4c844a5
Merge branch 'main' into DBA-699
bill-buchan Aug 6, 2024
7d33d90
Revert "allow ecs to retrieve ssm params"
Aug 7, 2024
0373af8
Create DMS S3 Endpoint
bill-buchan Aug 7, 2024
a950233
Merge branch 'DBA-699' of https://github.com/ministryofjustice/modern…
bill-buchan Aug 7, 2024
673d689
Do not change bucket policy unless we have at least one writer role d…
bill-buchan Aug 7, 2024
e517059
Bucket policy must be supplied to the module
bill-buchan Aug 8, 2024
6bc4b91
Single Principal
bill-buchan Aug 8, 2024
9d1b09e
Simplify bucket policy
bill-buchan Aug 8, 2024
a36bb50
Fix incorrect principals
bill-buchan Aug 8, 2024
938bb99
Add outbound replication task
bill-buchan Aug 8, 2024
6ff8250
Use primary database name as client
bill-buchan Aug 8, 2024
997a053
Task Recovery Table Not Available in S3
bill-buchan Aug 8, 2024
ccea9c5
Add the table mapping
bill-buchan Aug 8, 2024
517f143
Ignore Audit Read Database in Repository Environment (Where it does n…
bill-buchan Aug 12, 2024
077722e
Force SSL
bill-buchan Aug 12, 2024
c48631b
SSL Mode for endpoint
bill-buchan Aug 12, 2024
5ae256d
Required additional minimum privileges
bill-buchan Aug 12, 2024
ed8c0a1
SSL not supported for S3 endpoint
bill-buchan Aug 12, 2024
ff6c93e
Create and Attach DMS Bucket writer policy
bill-buchan Aug 12, 2024
61c6edc
Add Bucket ARNs
bill-buchan Aug 12, 2024
71e1861
ARN should be ARN
bill-buchan Aug 12, 2024
8036370
Allow DMS to write to local bucket
bill-buchan Aug 12, 2024
86285fd
Append asterisk to bucket ARN
bill-buchan Aug 13, 2024
2272982
Add asterisk at resource stage
bill-buchan Aug 13, 2024
277a92d
Swap Resources to Resource
bill-buchan Aug 13, 2024
635d817
Action instead of Actions
bill-buchan Aug 13, 2024
695ecb7
Remove principal
bill-buchan Aug 13, 2024
d82d6c9
Merge branch 'main' into DBA-699
bill-buchan Aug 13, 2024
0f7505c
Allow List Bucket
bill-buchan Aug 13, 2024
c5b8c8b
Add missing closing bracket
bill-buchan Aug 13, 2024
ca55b21
Add VPC Endpoint for S3
bill-buchan Aug 13, 2024
01d09b7
Add Security Group rule to allow access to S3 Endpoint
bill-buchan Aug 14, 2024
1d85c77
Add privilege to list bucket
bill-buchan Aug 14, 2024
19ace1b
Replace deprecated s3 resource
bill-buchan Aug 14, 2024
981b0f0
Update table mapping
bill-buchan Aug 14, 2024
e5312d7
Remove column needs object locator
bill-buchan Aug 14, 2024
dab4d20
Missing comma
bill-buchan Aug 14, 2024
7df413b
Add User S3 Endpoint
bill-buchan Aug 14, 2024
94c2a0a
Add comments
bill-buchan Aug 15, 2024
52cc879
Simplify file hierarchy
bill-buchan Aug 15, 2024
43f127f
Use local role for user endpoint
bill-buchan Aug 15, 2024
2954468
Add User Replication task
bill-buchan Aug 15, 2024
7653ccc
Add inbound endpoint
bill-buchan Aug 15, 2024
54f9762
Skip new endpoint temporarily
bill-buchan Aug 15, 2024
c370c8e
Add CDC path
bill-buchan Aug 15, 2024
97efb84
Add S3 Source Endpoint
bill-buchan Aug 16, 2024
2ac0405
Add External Table Definition for USER_ and PROBATION_AREA_USER tables
bill-buchan Aug 16, 2024
ae2a23e
Whitespace
bill-buchan Aug 16, 2024
f5c08ad
Date format
bill-buchan Aug 16, 2024
9f0d10f
Date format
bill-buchan Aug 16, 2024
fab80e3
Add timestamp column name
bill-buchan Aug 16, 2024
7d4c8ba
Add DB target endpoint
bill-buchan Aug 16, 2024
3d3f58e
Avoid duplicate name
bill-buchan Aug 16, 2024
b68c6ff
Add User inbound replication
bill-buchan Aug 16, 2024
da4d824
Disable versioning on the DMS staging bucket
bill-buchan Aug 16, 2024
e7d1428
Enable debugging
bill-buchan Aug 16, 2024
bdf6a47
Include the auto-generated timestamp column
bill-buchan Aug 16, 2024
a9302b2
Add fractional seconds
bill-buchan Aug 16, 2024
da2c2e0
Missing column added
bill-buchan Aug 16, 2024
16d3dee
Increase column tally
bill-buchan Aug 16, 2024
4d9c04f
Is Timestamp a column or not?
bill-buchan Aug 16, 2024
53b77ab
Put the timestamp column back
bill-buchan Aug 16, 2024
97b7570
Increase column count to match
bill-buchan Aug 16, 2024
fbed2c2
Create environment -> source database map
bill-buchan Aug 16, 2024
e93ea89
Add definition for AUDITED_INTERACTION external table
bill-buchan Aug 16, 2024
5abec2e
Merge branch 'main' into DBA-699
bill-buchan Aug 27, 2024
575782b
running plan but needs changes
Aug 20, 2024
f5c2439
add ldap container vars
Aug 22, 2024
29d5d9a
comment out ldap policies temporarily
Aug 27, 2024
33de302
running plan without ldap folder
Aug 28, 2024
92104cb
revert commits
Aug 28, 2024
e183dd0
Revert "running plan but needs changes"
Aug 28, 2024
2c23ee1
running plan
Aug 28, 2024
af4e886
add ldap microservice config
Aug 28, 2024
d5d2347
slapd log level
Aug 28, 2024
84477a1
pass in task role
Aug 28, 2024
a325c2d
secrets in one place only
Aug 28, 2024
e69437c
params have moved
Aug 28, 2024
663da89
keep secret
Aug 28, 2024
afca878
ancillary not frontend
Aug 29, 2024
1e73a4d
Merge branch 'main' into DBA-699
bill-buchan Aug 29, 2024
89fcd74
Map of environments to audit source database names
bill-buchan Aug 29, 2024
ae1e658
Update to AUDITED_INTERACTION table
bill-buchan Aug 29, 2024
c14252a
missing alb_listener_rule_host_header
Aug 30, 2024
c5ba4df
Merge branch 'main' into DBA-699
bill-buchan Sep 3, 2024
2386c8c
Merge remote-tracking branch 'origin/NIT-1408' into DBA-699
bill-buchan Sep 3, 2024
0759e19
Revert "Merge remote-tracking branch 'origin/NIT-1408' into DBA-699"
bill-buchan Sep 3, 2024
4867b1d
running plan but needs changes
Aug 20, 2024
4d346b2
add ldap container vars
Aug 22, 2024
5e89acc
comment out ldap policies temporarily
Aug 27, 2024
a6b3a25
running plan without ldap folder
Aug 28, 2024
7a09baa
revert commits
Aug 28, 2024
b9b69ae
Revert "running plan but needs changes"
Aug 28, 2024
852c595
running plan
Aug 28, 2024
4ff2eb6
add ldap microservice config
Aug 28, 2024
5ace61e
slapd log level
Aug 28, 2024
c476af4
pass in task role
Aug 28, 2024
0276993
secrets in one place only
Aug 28, 2024
844060e
params have moved
Aug 28, 2024
9b158e5
keep secret
Aug 28, 2024
3e05995
ancillary not frontend
Aug 29, 2024
0c83fe1
missing alb_listener_rule_host_header
Aug 30, 2024
1f279f0
conflicting dns entry
Sep 3, 2024
e291097
Add Business Interaction outbound
bill-buchan Sep 3, 2024
7241d68
Task Recovery Table not available for S3
bill-buchan Sep 3, 2024
7fc6bd2
swap old nlb refs
Sep 3, 2024
7cfe30f
remove r53 record for now
Sep 3, 2024
546f07b
Revert "remove r53 record for now"
Sep 3, 2024
da063cb
Merge remote-tracking branch 'origin/NIT-1408' into DBA-699
bill-buchan Sep 3, 2024
73b9546
Business Interaction using template
bill-buchan Sep 4, 2024
8afad84
Add Audit Target Database Endpoint
bill-buchan Sep 4, 2024
66cbcb4
mount efs for ldap
Sep 4, 2024
f743323
add efs vars
Sep 4, 2024
542c2e1
desired count 1 for ldap
Sep 4, 2024
28c5481
ldap mount points
Sep 4, 2024
c1e560f
Merge remote-tracking branch 'origin/NIT-1408' into DBA-699
bill-buchan Sep 5, 2024
d980846
Revert "Merge remote-tracking branch 'origin/NIT-1408' into DBA-699"
bill-buchan Sep 5, 2024
b79e9be
Revert "Merge remote-tracking branch 'origin/NIT-1408' into DBA-699"
bill-buchan Sep 5, 2024
ef8e32a
Add Business Interaction Inbound Replication Task
bill-buchan Sep 5, 2024
c5f30aa
Task Recovery Table Not Available for S3
bill-buchan Sep 6, 2024
ce69f45
Remove unwanted rules
bill-buchan Sep 6, 2024
b1108e6
Add definition for the BUSINESS_INTERACTION table.
bill-buchan Sep 6, 2024
8d2967d
Allow AUDIT_INTERACTION_PARAMETER to be nullable
bill-buchan Sep 6, 2024
062971a
Add Audited Interaction Inbound Replication Task
bill-buchan Sep 9, 2024
3e73049
Rule ID must be numeric
bill-buchan Sep 9, 2024
654e633
add temporary config for higher envs
Sep 9, 2024
921726d
CLIENT_BUSINESS_INTERACT_CODE does not exist in external table
bill-buchan Sep 9, 2024
281ea60
Make task names consistent
bill-buchan Sep 9, 2024
1c30f1d
Add new outbound replication task for AUDITED_INTERACTION_CHECKSUM
bill-buchan Sep 9, 2024
03e354f
Reapply "Merge remote-tracking branch 'origin/NIT-1408' into DBA-699"
bill-buchan Sep 9, 2024
4bc784f
Reapply "Merge remote-tracking branch 'origin/NIT-1408' into DBA-699"
bill-buchan Sep 9, 2024
9fa3daa
Merge remote-tracking branch 'origin/NIT-1408' into DBA-699
bill-buchan Sep 9, 2024
962cd96
Add inbound task for AUDITED_INTERACTION_CHECKSUM
bill-buchan Sep 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into DBA-699
bill-buchan committed Aug 27, 2024
commit 5abec2e66e642c8bf7ce621e2bbf548ab87a523d
8 changes: 4 additions & 4 deletions .devcontainer/devcontainer-lock.json
Original file line number Diff line number Diff line change
@@ -16,9 +16,9 @@
"integrity": "sha256:e81d52725655c8ffb861605feac7ad155b447d51af65f6c3a03cab32d59f1e16"
},
"ghcr.io/ministryofjustice/devcontainer-feature/terraform:1": {
"version": "1.0.0",
"resolved": "ghcr.io/ministryofjustice/devcontainer-feature/terraform@sha256:af3b3891cf31ff373df29998c690257d6f21f2ee4536bc4d692856408ef0c83a",
"integrity": "sha256:af3b3891cf31ff373df29998c690257d6f21f2ee4536bc4d692856408ef0c83a"
"version": "1.1.0",
"resolved": "ghcr.io/ministryofjustice/devcontainer-feature/terraform@sha256:34eb8c510a11fc44abb8173519215fcb6b82715b94e647c69089ee23773c6dc8",
"integrity": "sha256:34eb8c510a11fc44abb8173519215fcb6b82715b94e647c69089ee23773c6dc8"
}
}
}
}
8 changes: 4 additions & 4 deletions .github/workflows/code-scanning.yml
Original file line number Diff line number Diff line change
@@ -38,7 +38,7 @@ jobs:
run: tflint --disable-rule=terraform_unused_declarations --format sarif > tflint.sarif
- name: Upload SARIF file
if: success() || failure()
uses: github/codeql-action/upload-sarif@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # v3.26.0
uses: github/codeql-action/upload-sarif@2c779ab0d087cd7fe7b826087247c2c81f27bfa6 # v3.26.5
with:
sarif_file: tflint.sarif
trivy:
@@ -63,7 +63,7 @@ jobs:

- name: Upload Trivy scan results to GitHub Security tab
if: success() || failure()
uses: github/codeql-action/upload-sarif@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # v3.26.0
uses: github/codeql-action/upload-sarif@2c779ab0d087cd7fe7b826087247c2c81f27bfa6 # v3.26.5
with:
sarif_file: 'trivy-results.sarif'
checkov:
@@ -81,7 +81,7 @@ jobs:
fetch-depth: 0
- name: Run Checkov action
id: checkov
uses: bridgecrewio/checkov-action@5fa28e9c4db2c0920ade6ae453c0e91745c6378a # v12.2847.0
uses: bridgecrewio/checkov-action@4fa90328619ebe2a5396c7f16308c17a7a4b5dc3 # v12.2858.0
with:
directory: ./
framework: terraform
@@ -90,6 +90,6 @@ jobs:
skip_check: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
- name: Upload SARIF file
if: success() || failure()
uses: github/codeql-action/upload-sarif@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # v3.26.0
uses: github/codeql-action/upload-sarif@2c779ab0d087cd7fe7b826087247c2c81f27bfa6 # v3.26.5
with:
sarif_file: ./checkov.sarif
2 changes: 1 addition & 1 deletion .github/workflows/format-code.yml
Original file line number Diff line number Diff line change
@@ -40,7 +40,7 @@ jobs:
id: ml
# You can override MegaLinter flavor used to have faster performances
# More info at https://megalinter.io/flavors/
uses: oxsecurity/megalinter/flavors/terraform@bacb5f8674e3730b904ca4d20c8bd477bc51b1a7 #v7.13.0
uses: oxsecurity/megalinter/flavors/terraform@c217fe8f7bc9207062a084e989bd97efd56e7b9a #v8.0.0
env:
# All available variables are described in documentation
# https://megalinter.io/configuration/#shared-variables
2 changes: 1 addition & 1 deletion .github/workflows/nuke-redeploy.yml
Original file line number Diff line number Diff line change
@@ -68,7 +68,7 @@ jobs:
aws-region: ${{ env.AWS_REGION }}

- name: Load and Configure Terraform
uses: hashicorp/setup-terraform@651471c36a6092792c552e8b1bef71e592b462d8 # v3.1.1
uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: "~1"
terraform_wrapper: false
4 changes: 2 additions & 2 deletions .github/workflows/reusable_terraform_plan_apply.yml
Original file line number Diff line number Diff line change
@@ -112,7 +112,7 @@ jobs:
aws-region: "eu-west-2"

- name: Setup Terraform
uses: hashicorp/setup-terraform@651471c36a6092792c552e8b1bef71e592b462d8 # v3.1.1
uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: "${{ inputs.terraform_version }}"
terraform_wrapper: false
@@ -294,7 +294,7 @@ jobs:
aws-region: "eu-west-2"

- name: Setup Terraform
uses: hashicorp/setup-terraform@651471c36a6092792c552e8b1bef71e592b462d8 # v3.1.1
uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: "${{ inputs.terraform_version }}"
terraform_wrapper: false
4 changes: 2 additions & 2 deletions .github/workflows/reusable_terraform_plan_apply_test.yml
Original file line number Diff line number Diff line change
@@ -108,7 +108,7 @@ jobs:
aws-region: "eu-west-2"

- name: Setup Terraform
uses: hashicorp/setup-terraform@651471c36a6092792c552e8b1bef71e592b462d8 # v3.1.1
uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: "${{ inputs.terraform_version }}"
terraform_wrapper: false
@@ -257,7 +257,7 @@ jobs:
aws-region: "eu-west-2"

- name: Setup Terraform
uses: hashicorp/setup-terraform@651471c36a6092792c552e8b1bef71e592b462d8 # v3.1.1
uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3.1.2
with:
terraform_version: "${{ inputs.terraform_version }}"
terraform_wrapper: false
2 changes: 1 addition & 1 deletion .github/workflows/scorecards.yml
Original file line number Diff line number Diff line change
@@ -67,6 +67,6 @@ jobs:

# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # v3.26.0
uses: github/codeql-action/upload-sarif@2c779ab0d087cd7fe7b826087247c2c81f27bfa6 # v3.26.5
with:
sarif_file: results.sarif
Original file line number Diff line number Diff line change
@@ -3,7 +3,7 @@ module "eks_log_group" {
#checkov:skip=CKV_TF_2:Module registry does not support tags for versions

source = "terraform-aws-modules/cloudwatch/aws//modules/log-group"
version = "5.4.0"
version = "5.5.0"

name = local.eks_cloudwatch_log_group_name
kms_key_id = module.eks_cluster_logs_kms.key_arn
@@ -17,7 +17,7 @@ module "managed_prometheus_log_group" {
#checkov:skip=CKV_TF_2:Module registry does not support tags for versions

source = "terraform-aws-modules/cloudwatch/aws//modules/log-group"
version = "5.3.1"
version = "5.5.0"

name = local.amp_cloudwatch_log_group_name
kms_key_id = module.managed_prometheus_logs_kms.key_arn
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ module "eks" {
#checkov:skip=CKV_TF_2:Module registry does not support tags for versions

source = "terraform-aws-modules/eks/aws"
version = "20.20.0"
version = "20.24.0"

cluster_name = local.eks_cluster_name
cluster_version = local.environment_configuration.eks_cluster_version
@@ -172,7 +172,7 @@ module "karpenter" {
#checkov:skip=CKV_TF_2:Module registry does not support tags for versions

source = "terraform-aws-modules/eks/aws//modules/karpenter"
version = "20.20.0"
version = "20.24.0"

cluster_name = module.eks.cluster_name

@@ -190,6 +190,7 @@ module "karpenter" {
iam_role_policies = {
KarpenterSQSKMSAccess = module.karpenter_sqs_kms_access_iam_policy.arn
}
enable_v1_permissions = true

node_iam_role_name = "karpenter"
node_iam_role_additional_policies = {
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ module "aws_cloudwatch_metrics_pod_identity" {
#checkov:skip=CKV_TF_2:Module registry does not support tags for versions

source = "terraform-aws-modules/eks-pod-identity/aws"
version = "1.3.0"
version = "1.4.0"

name = "aws-cloudwatch-metrics"

Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ locals {
eks_cloudwatch_log_group_retention_in_days = 400

/* Kube Prometheus Stack */
prometheus_operator_crd_version = "v0.75.1"
prometheus_operator_crd_version = "v0.76.0"

/* Environment Configuration */
environment_configuration = local.environment_configurations[local.environment]
@@ -47,15 +47,15 @@ locals {
/* EKS */
eks_sso_access_role = "modernisation-platform-sandbox"
eks_cluster_version = "1.30"
eks_node_version = "1.20.4-b6163b2a"
eks_node_version = "1.21.0-4d43022e"
eks_cluster_addon_versions = {
coredns = "v1.11.1-eksbuild.9"
kube_proxy = "v1.30.0-eksbuild.3"
aws_ebs_csi_driver = "v1.32.0-eksbuild.1"
aws_efs_csi_driver = "v2.0.5-eksbuild.1"
coredns = "v1.11.1-eksbuild.11"
kube_proxy = "v1.30.3-eksbuild.2"
aws_ebs_csi_driver = "v1.33.0-eksbuild.1"
aws_efs_csi_driver = "v2.0.6-eksbuild.2"
aws_guardduty_agent = "v1.6.1-eksbuild.1"
eks_pod_identity_agent = "v1.3.0-eksbuild.1"
vpc_cni = "v1.18.2-eksbuild.1"
vpc_cni = "v1.18.3-eksbuild.2"
}

/* Data Engineering Airflow */
@@ -98,15 +98,15 @@ locals {
/* EKS */
eks_sso_access_role = "modernisation-platform-developer"
eks_cluster_version = "1.30"
eks_node_version = "1.20.4-b6163b2a"
eks_node_version = "1.21.0-4d43022e"
eks_cluster_addon_versions = {
coredns = "v1.11.1-eksbuild.9"
kube_proxy = "v1.30.0-eksbuild.3"
aws_ebs_csi_driver = "v1.32.0-eksbuild.1"
aws_efs_csi_driver = "v2.0.5-eksbuild.1"
coredns = "v1.11.1-eksbuild.11"
kube_proxy = "v1.30.3-eksbuild.2"
aws_ebs_csi_driver = "v1.33.0-eksbuild.1"
aws_efs_csi_driver = "v2.0.6-eksbuild.2"
aws_guardduty_agent = "v1.6.1-eksbuild.1"
eks_pod_identity_agent = "v1.3.0-eksbuild.1"
vpc_cni = "v1.18.2-eksbuild.1"
vpc_cni = "v1.18.3-eksbuild.2"
}

/* Observability Platform */
@@ -148,15 +148,15 @@ locals {
/* EKS */
eks_sso_access_role = "modernisation-platform-developer"
eks_cluster_version = "1.30"
eks_node_version = "1.20.4-b6163b2a"
eks_node_version = "1.21.0-4d43022e"
eks_cluster_addon_versions = {
coredns = "v1.11.1-eksbuild.9"
kube_proxy = "v1.30.0-eksbuild.3"
aws_ebs_csi_driver = "v1.32.0-eksbuild.1"
aws_efs_csi_driver = "v2.0.5-eksbuild.1"
coredns = "v1.11.1-eksbuild.11"
kube_proxy = "v1.30.3-eksbuild.2"
aws_ebs_csi_driver = "v1.33.0-eksbuild.1"
aws_efs_csi_driver = "v2.0.6-eksbuild.2"
aws_guardduty_agent = "v1.6.1-eksbuild.1"
eks_pod_identity_agent = "v1.3.0-eksbuild.1"
vpc_cni = "v1.18.2-eksbuild.1"
vpc_cni = "v1.18.3-eksbuild.2"
}

/* Data Engineering Airflow */
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ resource "helm_release" "actions_runner_mojas_create_a_derived_table" {
/* https://github.com/ministryofjustice/analytical-platform-actions-runner */
name = "actions-runner-mojas-create-a-derived-table"
repository = "oci://ghcr.io/ministryofjustice/analytical-platform-charts"
version = "2.318.0"
version = "2.319.1"
chart = "actions-runner"
namespace = kubernetes_namespace.actions_runners[0].metadata[0].name
values = [
@@ -35,7 +35,7 @@ resource "helm_release" "actions_runner_mojas_create_a_derived_table_dpr" {
/* https://github.com/ministryofjustice/analytical-platform-actions-runner */
name = "actions-runner-mojas-create-a-derived-table-dpr"
repository = "oci://ghcr.io/ministryofjustice/analytical-platform-charts"
version = "2.318.0"
version = "2.319.1"
chart = "actions-runner"
namespace = kubernetes_namespace.actions_runners[0].metadata[0].name
values = [
@@ -66,7 +66,7 @@ resource "helm_release" "actions_runner_mojas_airflow" {
/* https://github.com/ministryofjustice/analytical-platform-actions-runner */
name = "actions-runner-mojas-airflow"
repository = "oci://ghcr.io/ministryofjustice/analytical-platform-charts"
version = "2.318.0"
version = "2.319.1"
chart = "actions-runner"
namespace = kubernetes_namespace.actions_runners[0].metadata[0].name
values = [
@@ -97,7 +97,7 @@ resource "helm_release" "actions_runner_mojas_airflow_create_a_pipeline" {
/* https://github.com/ministryofjustice/analytical-platform-actions-runner */
name = "actions-runner-mojas-airflow-create-a-pipeline"
repository = "oci://ghcr.io/ministryofjustice/analytical-platform-charts"
version = "2.318.0"
version = "2.319.1"
chart = "actions-runner"
namespace = kubernetes_namespace.actions_runners[0].metadata[0].name
values = [
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@ resource "helm_release" "ui" {
/* https://github.com/ministryofjustice/analytical-platform-ui */
name = "ui"
repository = "oci://ghcr.io/ministryofjustice/analytical-platform-charts"
version = "0.0.0-rc1"
version = "0.1.6"
chart = "analytical-platform-ui"
namespace = kubernetes_namespace.ui.metadata[0].name
values = [
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@ resource "helm_release" "mlflow" {
/* https://github.com/ministryofjustice/analytical-platform-mlflow */
name = "mlflow"
repository = "oci://ghcr.io/ministryofjustice/analytical-platform-charts"
version = "2.15.1-rc1"
version = "2.15.1-rc2"
chart = "mlflow"
namespace = kubernetes_namespace.mlflow.metadata[0].name
values = [
Original file line number Diff line number Diff line change
@@ -64,11 +64,14 @@ resource "helm_release" "aws_for_fluent_bit" {

resource "helm_release" "amazon_prometheus_proxy" {
/* https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack */
/* If you are upgrading this chart, check whether the CRD version needs updating */
/*
If you are upgrading this chart, check whether the CRD version needs updating
https://github.com/prometheus-operator/prometheus-operator/releases
*/
name = "amazon-prometheus-proxy"
repository = "https://prometheus-community.github.io/helm-charts"
chart = "kube-prometheus-stack"
version = "61.3.2"
version = "61.9.0"
namespace = kubernetes_namespace.aws_observability.metadata[0].name
values = [
templatefile(
@@ -111,12 +114,34 @@ resource "helm_release" "cluster_autoscaler" {
}

/* Karpenter */
resource "helm_release" "karpenter_crd" {
/* https://github.com/aws/karpenter-provider-aws/releases */
name = "karpenter-crd"
repository = "oci://public.ecr.aws/karpenter"
chart = "karpenter-crd"
version = "1.0.0"
namespace = kubernetes_namespace.karpenter.metadata[0].name

values = [
templatefile(
"${path.module}/src/helm/values/karpenter-crd/values.yml.tftpl",
{
service_namespace = kubernetes_namespace.karpenter.metadata[0].name
}
)
]
depends_on = [
aws_iam_service_linked_role.spot,
module.karpenter
]
}

resource "helm_release" "karpenter" {
/* https://github.com/aws/karpenter-provider-aws/releases */
name = "karpenter"
repository = "oci://public.ecr.aws/karpenter"
chart = "karpenter"
version = "0.37.0"
version = "1.0.0"
namespace = kubernetes_namespace.karpenter.metadata[0].name

values = [
@@ -132,7 +157,8 @@ resource "helm_release" "karpenter" {
]
depends_on = [
aws_iam_service_linked_role.spot,
module.karpenter
module.karpenter,
helm_release.karpenter_crd
]
}

@@ -183,7 +209,7 @@ resource "helm_release" "cert_manager" {
name = "cert-manager"
repository = "https://charts.jetstack.io"
chart = "cert-manager"
version = "v1.15.1"
version = "v1.15.3"
namespace = kubernetes_namespace.cert_manager.metadata[0].name
values = [
templatefile(
@@ -236,7 +262,7 @@ resource "helm_release" "ingress_nginx" {
name = "ingress-nginx"
repository = "https://kubernetes.github.io/ingress-nginx"
chart = "ingress-nginx"
version = "4.11.1"
version = "4.11.2"
namespace = kubernetes_namespace.ingress_nginx.metadata[0].name
values = [
templatefile(
@@ -257,7 +283,7 @@ resource "helm_release" "external_secrets" {
name = "external-secrets"
repository = "https://charts.external-secrets.io"
chart = "external-secrets"
version = "0.9.20"
version = "0.10.0"
namespace = kubernetes_namespace.external_secrets.metadata[0].name
values = [
templatefile(
@@ -284,7 +310,7 @@ resource "helm_release" "keda" {
name = "keda"
repository = "https://kedacore.github.io/charts"
chart = "keda"
version = "2.14.2"
version = "2.15.1"
namespace = kubernetes_namespace.keda.metadata[0].name
values = [
templatefile(
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.