Require CodeBuild image from v4 onward

**Why?** Issue: awslabs#626 In prior versions of ADF, the CodeBuild image default was set to `UBUNTU_14_04_PYTHON_3_7_1`. This container image was no longer supported by the AWS CodeBuild service. Hence, using this version introduces a security risk as it is no longer patched. Moving to the latest CodeBuild image `STANDARD_7_0` was proposed when we switched to CDK v2. This change of the default image to use was one of the main reasons why just upgrading to CDK v2 required a major version release. As updating the default introduces a breaking change that might impact the pipelines of ADF. **What?** In the future, if we would only update the default we would require a new major version upgrade when `STANDARD_7_0` is deprecated too. Instead, this change proposes to require the image for the CodeBuild provider in the default properties of the build and deploy (when using CodeBuild to deploy) stages. For targets, it continues to be marked optional. But in case the target does not have an image set and nor does the default deploy provider, it will raise a `ValueError`.
sbkok · May 21, 2024 · 4059775 · 4059775
1 parent 867551c
commit 4059775
Show file tree

Hide file tree

Showing 9 changed files with 147 additions and 77 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -35,14 +35,21 @@ these customizations to CDK v2 as well.
 
 #### CodeBuild default image
 
-As written in the [CodeBuild provider
+As was written in the [CodeBuild provider
 docs](./docs/providers-guide.md#properties-3), it is a best-practice to define
 the exact CodeBuild container image you would like to use for each pipeline.
 
 However, in case you rely on the default, in prior ADF releases it would
 default to `UBUNTU_14_04_PYTHON_3_7_1`. This container image is no longer
-supported. With ADF v4.0, the new default is `STANDARD_7_0`.
-Also referred to as: `aws/codebuild/standard:7.0`.
+supported. With ADF v4.0, using the CodeBuild provider requires defining the
+specific CodeBuild container image to use. This way, it will not fallback to
+a default that might be secure today but deprecated in the future.
+
+For each pipeline definition in the deployment maps, the CodeBuild image will
+need to be defined. Alternatively, upgrade ADF and check which pipelines failed
+to deploy after. Most likely all pipelines already define the CodeBuild image
+to use, as the previous default image is [not supported by
+AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html#deprecated-images).
 
 #### ADF Parameters in AWS Systems Manager Parameter Store
 

diff --git a/docs/providers-guide.md b/docs/providers-guide.md
@@ -220,11 +220,9 @@ Provider type: `codebuild`.
 
 #### Properties
 
-- *image* *(String|Object)* - default: `STANDARD_7_0`.
-  - It is recommended to specify the container image your pipeline requires.
-    Relying on the default value might impact the pipeline in future updates
-    of ADF if the default were to change.
-  - The Image that the AWS CodeBuild will use. Images can be found
+- *image* *(String|Object)*.
+  - It is required to specify the container image your pipeline requires.
+  - Specify the Image that the AWS CodeBuild will use. Images can be found
     [here](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-codebuild.LinuxBuildImage.html).
   - Image can also take an object that contains a reference to a public docker
     hub image with a prefix of `docker-hub://`, such as

diff --git a/.../initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_codebuild.py b/.../initial_commit/bootstrap_repository/adf-build/shared/cdk/cdk_constructs/adf_codebuild.py
@@ -22,7 +22,6 @@
 
 ADF_DEPLOYMENT_REGION = os.environ["AWS_REGION"]
 ADF_DEPLOYMENT_ACCOUNT_ID = os.environ["ACCOUNT_ID"]
-DEFAULT_CODEBUILD_IMAGE = "STANDARD_7_0"
 DEFAULT_BUILD_SPEC_FILENAME = 'buildspec.yml'
 DEFAULT_DEPLOY_SPEC_FILENAME = 'deployspec.yml'
 ADF_DEFAULT_BUILD_ROLE_NAME = 'adf-codebuild-role'
@@ -339,14 +338,9 @@ def determine_build_spec(codebuild_id, default_props, target=None):
 
     @staticmethod
     def get_image_by_name(specific_image: str):
-        image_name = (
-            (
-                specific_image
-                or DEFAULT_CODEBUILD_IMAGE
-            ).upper()
-        )
-        if hasattr(_codebuild.LinuxBuildImage, image_name):
-            return getattr(_codebuild.LinuxBuildImage, image_name)
+        cdk_image_name = specific_image.upper()
+        if hasattr(_codebuild.LinuxBuildImage, cdk_image_name):
+            return getattr(_codebuild.LinuxBuildImage, cdk_image_name)
         if specific_image.startswith('docker-hub://'):
             specific_image = specific_image.split('docker-hub://')[-1]
             return _codebuild.LinuxBuildImage.from_docker_registry(
@@ -399,6 +393,9 @@ def determine_build_image(codebuild_id, scope, target, map_params):
                 specific_image.get('tag', 'latest'),
             )
 
+        if not specific_image:
+            raise ValueError("Required CodeBuild image property is not configured")
+
         return CodeBuild.get_image_by_name(specific_image)
 
     @staticmethod

diff --git a/...ory/adf-build/shared/cdk/cdk_constructs/tests/test_adf_codebuild_determine_build_image.py b/...ory/adf-build/shared/cdk/cdk_constructs/tests/test_adf_codebuild_determine_build_image.py
@@ -9,23 +9,17 @@
     aws_codebuild as _codebuild,
     Stack,
 )
-from cdk_constructs.adf_codebuild import CodeBuild, DEFAULT_CODEBUILD_IMAGE
+from cdk_constructs.adf_codebuild import CodeBuild
 
-SIMPLE_TARGET = {
-    'properties': {},
-}
 SPECIFIC_CODEBUILD_IMAGE_STR = 'STANDARD_7_0'
 SPECIFIC_CODEBUILD_IMAGE_ALT_STR = 'STANDARD_6_0'
 SPECIFIC_CODEBUILD_IMAGE_ALT2_STR = 'STANDARD_5_0'
 SPECIFIC_CODEBUILD_IMAGE_ECR = {
     'repository_arn': 'arn:aws:ecr:region:111111111111:repository/test',
     'tag': 'specific',
 }
-CODEBUILD_SPECIFIC_MAP_PARAMS_STR = {
-    'provider': 'codebuild',
-    'properties': {
-        'image': SPECIFIC_CODEBUILD_IMAGE_STR,
-    }
+SIMPLE_TARGET = {
+    'properties': {},
 }
 CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR = {
     'provider': 'codebuild',
@@ -48,8 +42,16 @@
 
 CODEBUILD_BASE_MAP_PARAMS = {
     'default_providers': {
-        'build': {},
-        'deploy': {},
+        'build': {
+            'properties': {
+                'image': SPECIFIC_CODEBUILD_IMAGE_STR,
+            },
+        },
+        'deploy': {
+            'properties': {
+                'image': SPECIFIC_CODEBUILD_IMAGE_STR,
+            },
+        },
     },
 }
 
@@ -85,7 +87,7 @@ def test_determine_build_image_build_defaults(ecr_repo, build_image):
 
     assert result == getattr(
         _codebuild.LinuxBuildImage,
-        DEFAULT_CODEBUILD_IMAGE,
+        SPECIFIC_CODEBUILD_IMAGE_STR,
     )
     ecr_repo.from_repository_arn.assert_not_called()
     build_image.from_ecr_repository.assert_not_called()
@@ -109,11 +111,11 @@ def test_determine_build_image_build_str(ecr_repo, build_image):
     target = None
     map_params = deepcopy(CODEBUILD_BASE_MAP_PARAMS)
     map_params['default_providers']['build'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_STR
+        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
     # Set deploy one to alternative, so we can test
     # that it is not using this in build steps
     map_params['default_providers']['deploy'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
+        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT2_STR
 
     result = CodeBuild.determine_build_image(
         codebuild_id='some_id',
@@ -124,7 +126,7 @@ def test_determine_build_image_build_str(ecr_repo, build_image):
 
     assert result == getattr(
         _codebuild.LinuxBuildImage,
-        SPECIFIC_CODEBUILD_IMAGE_STR,
+        SPECIFIC_CODEBUILD_IMAGE_ALT_STR,
     )
     ecr_repo.from_repository_arn.assert_not_called()
     build_image.from_ecr_repository.assert_not_called()
@@ -266,7 +268,7 @@ def test_determine_build_image_deploy_defaults(ecr_repo, build_image):
 
     assert result == getattr(
         _codebuild.LinuxBuildImage,
-        DEFAULT_CODEBUILD_IMAGE,
+        SPECIFIC_CODEBUILD_IMAGE_STR,
     )
     ecr_repo.from_repository_arn.assert_not_called()
     build_image.from_ecr_repository.assert_not_called()
@@ -288,12 +290,12 @@ def test_determine_build_image_deploy_target_str(ecr_repo, build_image):
         not the default deploy specific config.
     """
     scope = Stack()
-    target = CODEBUILD_SPECIFIC_MAP_PARAMS_STR
+    target = CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
     map_params = deepcopy(CODEBUILD_BASE_MAP_PARAMS)
     # Set build one to alternative, so we can test
     # that it is not using this in deploy steps
     map_params['default_providers']['build'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
+        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT2_STR
 
     result = CodeBuild.determine_build_image(
         codebuild_id='some_id',
@@ -304,7 +306,7 @@ def test_determine_build_image_deploy_target_str(ecr_repo, build_image):
 
     assert result == getattr(
         _codebuild.LinuxBuildImage,
-        SPECIFIC_CODEBUILD_IMAGE_STR,
+        SPECIFIC_CODEBUILD_IMAGE_ALT_STR,
     )
     ecr_repo.from_repository_arn.assert_not_called()
     build_image.from_ecr_repository.assert_not_called()
@@ -328,11 +330,11 @@ def test_determine_build_image_deploy_str(ecr_repo, build_image):
     target = SIMPLE_TARGET
     map_params = deepcopy(CODEBUILD_BASE_MAP_PARAMS)
     map_params['default_providers']['deploy'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_STR
+        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
     # Set build one to alternative, so we can test
     # that it is not using this in deploy steps
     map_params['default_providers']['build'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
+        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT2_STR
 
     result = CodeBuild.determine_build_image(
         codebuild_id='some_id',
@@ -343,7 +345,7 @@ def test_determine_build_image_deploy_str(ecr_repo, build_image):
 
     assert result == getattr(
         _codebuild.LinuxBuildImage,
-        SPECIFIC_CODEBUILD_IMAGE_STR,
+        SPECIFIC_CODEBUILD_IMAGE_ALT_STR,
     )
     ecr_repo.from_repository_arn.assert_not_called()
     build_image.from_ecr_repository.assert_not_called()
@@ -366,12 +368,6 @@ def test_determine_build_image_deploy_target_str_too(ecr_repo, build_image):
     scope = Stack()
     target = CODEBUILD_SPECIFIC_MAP_PARAMS_ALT2_STR
     map_params = deepcopy(CODEBUILD_BASE_MAP_PARAMS)
-    map_params['default_providers']['deploy'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_STR
-    # Set build one to alternative, so we can test
-    # that it is not using this in deploy steps
-    map_params['default_providers']['build'] = \
-        CODEBUILD_SPECIFIC_MAP_PARAMS_ALT_STR
 
     result = CodeBuild.determine_build_image(
         codebuild_id='some_id',

diff --git a/.../bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_default_pipeline_type.py b/.../bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_default_pipeline_type.py
@@ -46,7 +46,7 @@ def test_pipeline_creation_outputs_as_expected_when_input_has_1_target_with_2_wa
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -113,7 +113,7 @@ def test_pipeline_creation_outputs_as_expected_when_input_has_2_targets_with_2_w
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {

diff --git a/...mmit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_pipeline_creation.py b/...mmit/bootstrap_repository/adf-build/shared/cdk/cdk_stacks/tests/test_pipeline_creation.py
@@ -60,7 +60,7 @@ def test_pipeline_creation_outputs_as_expected_when_source_is_s3_and_build_is_co
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -117,7 +117,7 @@ def test_pipeline_creation_outputs_as_expected_when_source_is_codecommit_and_bui
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -179,7 +179,7 @@ def test_pipeline_creation_outputs_as_expected_when_source_is_codecommit_with_co
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -252,7 +252,7 @@ def test_pipeline_creation_outputs_with_codeartifact_trigger():
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -313,7 +313,7 @@ def test_pipeline_creation_outputs_with_codeartifact_trigger_with_package_name()
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -381,7 +381,7 @@ def test_pipeline_creation_outputs_with_invalid_trigger_type():
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {
@@ -434,7 +434,7 @@ def test_pipeline_creation_outputs_as_expected_when_notification_endpoint_is_cha
     }
     stack_input["pipeline_input"]["default_providers"]["build"] = {
         "provider": "codebuild",
-        "properties": {"account_id": "123456789012"},
+        "properties": {"image": "STANDARD_7_0"},
     }
 
     stack_input["ssm_params"][region_name] = {