Use FIPS S3 endpoints only when available in region

[harishxr]

Summary

This pull request enhances the S3 client creation logic to support FIPS-complaint S3 endpoints based on the region of the S3 bucket. The changes ensure that when a host is running in FIPS mode, the agent checks if the S3 bucket's region supports FIPS endpoints. If it does, the FIPs-compliant endpoints are used; otherwise, standard endpoints are used.

Implementation details

factory.go:

• Added a helper function isS3FIPSCompliantRegion to check if a region supports FIPS endpoints.
• Modified the createAWSConfig function to accept a useFIPSEndpoint boolean parameter to determine if FIPS-compliant endpoints should be used.
• Updated NewS3ManagerClient and NewS3Client to:
  • Create an initial AWS session to determine the bucket's region.
  • Check if the bucket's region supports FIPS endpoints and configure the AWS session accordingly.

Unit tests factory_test.go:

• Modified TestCreateAWSConfig to cover:
  • Standard regions without FIPS enabled.
  • Standard regions with FIPS enabled.
  • FIPS supported S3 regions with FIPS enabled.

Testing

• Ran make test
• Built test agent using the changes made in the PR and ran agent on FIPS Enabled host.
• Agent was able to identify it running in a FIPS enabled env based on the following log

/var/log/ecs/ecs-agent.log.2024-07-17-19:level=info time=2024-07-17T19:29:28Z msg="FIPS mode detected on the host"

Part 1:

• Created an S3 bucket in ap-southeast-2 which is a region that does not have FIPS S3 endpoints and placed a test1.env file in the S3 bucket with the following content

TEST1=ABC
TEST2=XYZ

• Ran a task with the following task definition

{
    "taskDefinitionArn": "arn:aws:ecs:us-west-2:redacted:task-definition/fips-test:3",
    "containerDefinitions": [
        {
            "name": "sleepy300",
            "image": "busybox",
            "cpu": 100,
            "memory": 10,
            "portMappings": [],
            "essential": true,
            "command": [
                "sleep",
                "10000000"
            ],
            "environment": [],
            "environmentFiles": [
                {
                    "value": "arn:aws:s3:::redacted-syd/test1.env",
                    "type": "s3"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "systemControls": []
        }
    ],
    "family": "fips-test",
    "taskRoleArn": "arn:aws:iam::redacted:role/TaskIAMRoleArn",
    "executionRoleArn": "arn:aws:iam::redacted:role/TaskExecutionRoleARN",
    "networkMode": "host",
    "revision": 2,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role-network-host"
        },
        {
            "name": "ecs.capability.env-files.s3"
        },
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EXTERNAL",
        "EC2"
    ],
    "cpu": "256",
    "memory": "512",
    "registeredAt": "2024-07-16T20:15:43.550Z",
    "registeredBy": "arn:aws:sts::redacted:assumed-role/Admin/test",
    "tags": []
}

• Verified in the ECS console for the running task under Environment Variables and Files that the env file was attached to the container.
• Logged onto the running container using docker exec -it <container_id> sh and executed the following commands to access the values defined in the env files

/ # echo $TEST1
ABC
/ # echo $TEST2
XYZ

• Verified using the logs that FIPS S3 Endpoints are not trying to being used

/var/log/ecs/ecs-agent.log.2024-07-17-19:level=debug time=2024-07-17T19:30:49Z msg="FIPS mode detected, using virtual-host–style URLs for bucket location"

Part 2:

• Created an S3 bucket in us-west-2 which is a region that has FIPS S3 endpoints and placed a test1.env file in the S3 bucket with the following content

TEST1=ABC
TEST2=XYZ

• Ran a task with the following task definition

{
    "taskDefinitionArn": "arn:aws:ecs:us-west-2:redacted:task-definition/fips-test:2",
    "containerDefinitions": [
        {
            "name": "sleepy300",
            "image": "busybox",
            "cpu": 100,
            "memory": 10,
            "portMappings": [],
            "essential": true,
            "command": [
                "sleep",
                "10000000"
            ],
            "environment": [],
            "environmentFiles": [
                {
                    "value": "arn:aws:s3:::redacted-syd/test1.env",
                    "type": "s3"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "systemControls": []
        }
    ],
    "family": "fips-test",
    "taskRoleArn": "arn:aws:iam::redacted:role/TaskIAMRoleArn",
    "executionRoleArn": "arn:aws:iam::redacted:role/TaskExecutionRoleARN",
    "networkMode": "host",
    "revision": 2,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role-network-host"
        },
        {
            "name": "ecs.capability.env-files.s3"
        },
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EXTERNAL",
        "EC2"
    ],
    "cpu": "256",
    "memory": "512",
    "registeredAt": "2024-05-28T22:54:48.812Z",
    "registeredBy": "arn:aws:sts::redacted:assumed-role/Admin/test",
    "tags": []
}

• Verified in the ECS console for the running task under Environment Variables and Files that the env file was attached to the container.
• Logged onto the running container using docker exec -it <container_id> sh and executed the following commands to access the values defined in the env files

/ # echo $TEST1
ABC
/ # echo $TEST2
XYZ

• Verified using the logs that FIPS S3 Endpoints are being used

/var/log/ecs/ecs-agent.log:level=debug time=2024-07-17T20:13:52Z msg="FIPS mode detected, using virtual-host–style URLs for bucket location"
/var/log/ecs/ecs-agent.log:level=debug time=2024-07-17T20:13:52Z msg="FIPS mode detected, using FIPS-compliant S3 endpoint in supported regions"

New tests cover the changes: yes

Description for the changelog

Use FIPS S3 endpoints only when available in region
Does this PR include breaking model changes? If so, Have you added transformation functions?

No

References

• Regions where FIPS S3 endpoints are available - https://aws.amazon.com/compliance/fips/

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.