Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws-codebuild: using SMALL compute type with GPU codebuild environment #25857

Closed
pviolette3 opened this issue Jun 5, 2023 · 3 comments · Fixed by #25880
Closed

aws-codebuild: using SMALL compute type with GPU codebuild environment #25857

pviolette3 opened this issue Jun 5, 2023 · 3 comments · Fixed by #25880
Labels
@aws-cdk/aws-codebuild Related to AWS CodeBuild bug This issue is a bug. effort/small Small work item – less than a day of effort good first issue Related to contributions. See CONTRIBUTING.md p1

Comments

@pviolette3
Copy link

Describe the bug

It seems like AWS CodeBuild recently added support for SMALL compute type with GPU compute environments (blog post from Mar 17, 2023). CDK CodeBuild seems to have a now-outdated check to ensure only LARGE compute type can be used with GPU instances https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-codebuild/lib/linux-gpu-build-image.ts#L146

which leads to an error if the small GPU codebuild is used:

RuntimeError: Invalid CodeBuild environment: GPU images only support ComputeType 'BUILD_GENERAL1_LARGE' - 'BUILD_GENERAL1_SMALL' was given

For more context on the small GPU compute type support:

  1. The docs show support for SMALL compute type with GPU environment:

image

  1. CodeBuild UI also allows you to specify the build type.

image

Expected Behavior

CDK users should be allowed to select compute type for BUILD_GENERAL1_SMALL along with a GPU environment now that codebuild does support it.

Current Behavior

An exception is raised:

RuntimeError: Invalid CodeBuild environment: GPU images only support ComputeType 'BUILD_GENERAL1_LARGE' - 'BUILD_GENERAL1_SMALL' was given

It looks like this error comes from CDK code https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-codebuild/lib/linux-gpu-build-image.ts#L146

Reproduction Steps

from aws_cdk import Stack, App, aws_codebuild
from constructs import Construct


class BadBuildEnvironment(Stack):
    def __init__(
        self,
        scope: Construct,
        construct_id: str,
        **kwargs,
    ) -> None:
        super().__init__(scope, construct_id, **kwargs)
        aws_codebuild.BuildEnvironment(
            build_image=aws_codebuild.LinuxGpuBuildImage.aws_deep_learning_containers_image(
                repository_name="pytorch-training",
                tag="1.13.1-gpu-py39-cu117-ubuntu20.04-ec2",
            ),
            compute_type=aws_codebuild.ComputeType.SMALL,
        )


BadBuildEnvironment(app, "BadBuildEnvironment")
app.synth()

Then I run

cdk synth BadBuildEnvironment and see:

RuntimeError: Invalid CodeBuild environment: GPU images only support ComputeType 'BUILD_GENERAL1_LARGE' - 'BUILD_GENERAL1_SMALL' was given

Possible Solution

Update the validation to allow for SMALL type since it's now supported on the codebuild side.

Additional Information/Context

No response

CDK CLI Version

2.77.0 (build 06a0b19)

Framework Version

No response

Node.js Version

v20.0.0

OS

Linux HEUWL07705 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Language

Python

Language Version

No response

Other information

No response

@pviolette3 pviolette3 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jun 5, 2023
@github-actions github-actions bot added the @aws-cdk/aws-codebuild Related to AWS CodeBuild label Jun 5, 2023
@peterwoodworth peterwoodworth added good first issue Related to contributions. See CONTRIBUTING.md p1 effort/small Small work item – less than a day of effort and removed needs-triage This issue or PR still needs to be triaged. labels Jun 6, 2023
@peterwoodworth
Copy link
Contributor

Thanks for submitting this, and for sharing where in the codebase we're checking for this, it should be a pretty easy fix! We accept contributions 🙂

@pviolette3
Copy link
Author

Thanks for the quick triaging. I can likely take a stab at this by the end of this month, and potentially sooner. I will comment when I begin if I have some more time. To any potential contributors, please feel free to start this if I haven't commented anything over the coming weeks.

If anyone else wants to look in the meantime I think the Arm build image has almost copy/pastable code since ARM supports both SMALL and LARGE:

mergify bot added a commit to lpizzinidev/aws-cdk that referenced this issue Jun 12, 2023
@mergify mergify bot closed this as completed in #25880 Jun 12, 2023
mergify bot pushed a commit that referenced this issue Jun 12, 2023
…pute type with Linux GPU build image (#25880)

This fix allows specifying the `BUILD_GENERAL1_SMALL` compute type when using a Linux GPU to build an image as defined in the [docs](https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-compute-types.html).

Closes #25857.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-codebuild Related to AWS CodeBuild bug This issue is a bug. effort/small Small work item – less than a day of effort good first issue Related to contributions. See CONTRIBUTING.md p1
Projects
None yet
2 participants