Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ecs): add validation checks to memory cpu combinations of FARGATE compatible task definitions #30166

Merged
merged 13 commits into from
Jul 30, 2024

Conversation

mrlikl
Copy link
Contributor

@mrlikl mrlikl commented May 12, 2024

Issue # (if applicable)

Closes #22216

Reason for this change

Adds a validation that would point out invalid CPU - Memory limits for a FARGATE compatible task definition. This is based on the document https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html

Description of changes

Added a validateFargateTaskDefinitionMemoryCpu method in the base task definition and run this validation when a task definition has FARGATE in requiresCompatibilities.

This is in line with the ECS requirement that even if a task definition has EC2 compatibility (along with FARGATE), FARGATE validations apply

Description of how you validated changes

Validated the changes by providing incorrect as well as correct values, fixed all the invalid combinations that are present in the unit tests

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added bug This issue is a bug. effort/small Small work item – less than a day of effort p2 labels May 12, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team May 12, 2024 18:48
@github-actions github-actions bot added the beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK label May 12, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@mrlikl mrlikl changed the title feat(aws-ecs): add validation checks to memory cpu combinations of FARGATE compatible task definitions feat(ecs): add validation checks to memory cpu combinations of FARGATE compatible task definitions May 12, 2024
@mrlikl mrlikl marked this pull request as draft May 12, 2024 18:56
@mrlikl mrlikl marked this pull request as ready for review May 12, 2024 19:36
@aws-cdk-automation aws-cdk-automation dismissed their stale review May 12, 2024 19:38

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label May 12, 2024
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍 Left some comments for adjustments

// Validate CPU and memory combinations if fargate compatible
if (isFargateCompatible(props.compatibility)) {
// Check the combination as per doc https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html
this.validateFargateTaskDefinitionMemoryCpu(props.cpu!, props.memoryMiB!);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we take the opportunity to remove the !s? It's a bit annoying to see them.

    if (this.isFargateCompatible) {
      if (!props.cpu || !props.memoryMiB) {
        throw new Error(`Fargate-compatible tasks require both CPU (${props.cpu}) and memory (${props.memoryMiB}) specifications`);
      }

      // validate the cpu and memory size for the Windows operation system family.
      if (props.runtimePlatform?.operatingSystemFamily?.isWindows()) {
        this.checkFargateWindowsBasedTasksSize(props.cpu, props.memoryMiB, props.runtimePlatform);
      }

      // Check the combination as per doc https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html
      this.validateFargateTaskDefinitionMemoryCpu(props.cpu, props.memoryMiB);
    }

Comment on lines 910 to 929
const validCpuMemoryCombinations = [
{ cpu: 256, memory: [512, 1024, 2048] },
{ cpu: 512, memory: this.range(1024, 4096, 1024) },
{ cpu: 1024, memory: this.range(2048, 8192, 1024) },
{ cpu: 2048, memory: this.range(4096, 16384, 1024) },
{ cpu: 4096, memory: this.range(8192, 30720, 1024) },
{ cpu: 8192, memory: this.range(16384, 61440, 4096) },
{ cpu: 16384, memory: this.range(32768, 122880, 8192) },
];

const isValidCombination = validCpuMemoryCombinations.some((combo) => {
return combo.cpu === Number(cpu) && combo.memory.includes(Number(memory));
});

if (!isValidCombination) {
throw new Error('Invalid CPU and memory combinations for FARGATE compatible task definition - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html');
}
}

private range(start: number, end: number, step: number): number[] {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const validCpuMemoryCombinations = [
{ cpu: 256, memory: [512, 1024, 2048] },
{ cpu: 512, memory: this.range(1024, 4096, 1024) },
{ cpu: 1024, memory: this.range(2048, 8192, 1024) },
{ cpu: 2048, memory: this.range(4096, 16384, 1024) },
{ cpu: 4096, memory: this.range(8192, 30720, 1024) },
{ cpu: 8192, memory: this.range(16384, 61440, 4096) },
{ cpu: 16384, memory: this.range(32768, 122880, 8192) },
];
const isValidCombination = validCpuMemoryCombinations.some((combo) => {
return combo.cpu === Number(cpu) && combo.memory.includes(Number(memory));
});
if (!isValidCombination) {
throw new Error('Invalid CPU and memory combinations for FARGATE compatible task definition - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html');
}
}
private range(start: number, end: number, step: number): number[] {
const validCpuMemoryCombinations = [
{ cpu: '256', memory: ['512', '1024', '2048'] },
{ cpu: '512', memory: this.range(1024, 4096, 1024) },
{ cpu: '1024', memory: this.range(2048, 8192, 1024) },
{ cpu: '2048', memory: this.range(4096, 16384, 1024) },
{ cpu: '4096', memory: this.range(8192, 30720, 1024) },
{ cpu: '8192', memory: this.range(16384, 61440, 4096) },
{ cpu: '16384', memory: this.range(32768, 122880, 8192) },
];
const isValidCombination = validCpuMemoryCombinations.some((combo) => {
return combo.cpu === cpu && combo.memory.includes(memory);
});
if (!isValidCombination) {
throw new Error('Invalid CPU and memory combinations for FARGATE compatible task definition - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html');
}
}
private range(start: number, end: number, step: number): string[] {

What about avoiding Number -> String casting?

@@ -21,28 +21,24 @@ describe('fargate task definition', () => {

});

test('support lazy cpu and memory values', () => {
test('does not support lazy cpu and memory values', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be able to support this by adding validation on the node (like this):

this.node.addValidation({ validate: () => this. validateFargateTaskDefinitionMemoryCpu(...);

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jun 15, 2024
}

// Check the combination as per doc https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html
this.validateFargateTaskDefinitionMemoryCpu(props.cpu, props.memoryMiB);
Copy link
Contributor

@lpizzinidev lpizzinidev Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
this.validateFargateTaskDefinitionMemoryCpu(props.cpu, props.memoryMiB);
this.node.addValidation({
validate: () => this.validateFargateTaskDefinitionMemoryCpu(props.cpu, props.memoryMiB)
});

Please note https://github.com/aws/aws-cdk/pull/30166/files#r1641289793

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made all the suggested changes @lpizzinidev

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look once

@mrlikl mrlikl requested a review from lpizzinidev June 18, 2024 18:09
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍

@pahud pahud added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 9, 2024
@Leo10Gama Leo10Gama self-assigned this Jul 25, 2024
Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Left a few comments and nits to be addressed.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 26, 2024
@mergify mergify bot dismissed Leo10Gama’s stale review July 29, 2024 04:37

Pull request has been modified.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 29, 2024
Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes, and fixing a few other typos around the repo! Just one minor concern about some of the new logic-shuffling.

packages/aws-cdk-lib/aws-ecs/lib/base/task-definition.ts Outdated Show resolved Hide resolved
Copy link
Member

@Leo10Gama Leo10Gama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for your contribution!

@paulhcsun paulhcsun added the pr/do-not-merge This PR should not be merged at this time. label Jul 30, 2024
@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jul 30, 2024
@Leo10Gama Leo10Gama removed the pr/do-not-merge This PR should not be merged at this time. label Jul 30, 2024
Copy link
Contributor

mergify bot commented Jul 30, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot temporarily deployed to test-pipeline July 30, 2024 18:28 Inactive
@aws-cdk-automation
Copy link
Collaborator

➡️ PR build request submitted to test-main-pipeline ⬅️

A maintainer must now check the pipeline and add the pr-linter/cli-integ-tested label once the pipeline succeeds.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: b4f36b2
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 8b4685e into aws:main Jul 30, 2024
10 of 11 checks passed
Copy link
Contributor

mergify bot commented Jul 30, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. effort/small Small work item – less than a day of effort p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(ecs): Incorrect ECS Fargate cpu/memory settings are not detected
6 participants