AWS::CodeBuild::Project - Support Batch Builds (From Trigger) #621

wulfmann · 2020-09-03T13:57:06Z

1. AWS::CodeBuild::Project - Support Batch Builds

When creating a CodeBuild project there is no way to specify that the webhook trigger should run the project as a 'Batch Build'

2. Scope of request

a) new option for an existing attribute is desired

There is already a webhook option in the project triggers section of the provider, but it is a boolean value. It would seem that support for a new 'BuildType' option should be added.

3. Expected behavior

Allow me to specify a trigger type of STANDARD or BATCH

4. Suggest specific test cases

Samples:

Create a CodeBuild Project with a trigger type of STANDARD

Create a CodeBuild Project with a trigger type of BATCH

5. Helpful Links to speed up research and evaluation

Trigger batch build with cli

6. Category (required)

Developer Tools (CodeBuild)

wulfmann · 2020-09-03T13:57:31Z

I can include screenshots from the console if that is helpful.

tjenkinson · 2020-09-14T08:35:20Z

It's not supported at the moment unfortunately. If you're using the CDK it's possible to create a custom resource using AwsCustomResource which worked for us. Using the CodeBuild service and createWebhook action

wulfmann · 2020-09-15T02:58:10Z

Thanks for the tip @tjenkinson ! That seems like it will be adequate until this ticket is worked.

subinataws · 2020-09-17T17:23:54Z

@wulfmann - Thanks for reporting this issue. Team is working on a fix and will address it in the next week or so.

tjenkinson · 2020-09-25T09:36:52Z

just opened #633 which is support for triggering batch builds from codepipeline

tjenkinson · 2020-11-20T11:13:46Z

@subinataws any update on this?

johnhanks1 · 2020-11-23T18:14:34Z

Hello @tjenkinson

We have enabled this for projects. This can be used with CloudFormation with the BuildType option, the valid inputs are BUILD or BUILD_BATCH example below

Resources:
  CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Ref ProjectName
      ...
      Source:
        Type: GITHUB
        Location: !Ref SourceLocation
        InsecureSsl: true
        GitCloneDepth: 1
        ReportBuildStatus: false
      Triggers:
        Webhook: true
        BuildType: BUILD_BATCH

We are working on updating or official documentation. Sorry for the confusion.

tjenkinson · 2020-11-24T11:50:47Z

That's great thanks. Next step is getting it into the CDK :)

tjenkinson · 2020-11-30T20:52:36Z

@johnhanks1 does this setting apply for any trigger or just webhooks?

Question from @skinny85 on the cdk PR (aws/aws-cdk#11743 (comment))

So this setting works only for builds started through the webhook? Not through, for instance, CloudWatch Events?

tjenkinson · 2020-12-05T10:23:18Z

Found it in the docs now but it says

Not currently supported by AWS CloudFormation.

tjenkinson · 2020-12-11T17:00:16Z

@johnhanks1 would be great to get some clarity on this because it's blocking the CDK PR. Thanks!

This adds a `startBatchBuild` option to the code build source, to trigger a batch build. The cloudformation property isn't in the official docs yet but is mentioned [here](aws-cloudformation/cloudformation-coverage-roadmap#621 (comment)). Closes #11663

This adds a `startBatchBuild` option to the code build source, to trigger a batch build. The cloudformation property isn't in the official docs yet but is mentioned [here](aws-cloudformation/cloudformation-coverage-roadmap#621 (comment)). Closes aws#11663

paulsbnet · 2021-01-06T19:28:52Z

Hello @tjenkinson

We have enabled this for projects. This can be used with CloudFormation with the BuildType option, the valid inputs are BUILD or BUILD_BATCH example below
Resources:
  CodeBuildProject:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Ref ProjectName
      ...
      Source:
        Type: GITHUB
        Location: !Ref SourceLocation
        InsecureSsl: true
        GitCloneDepth: 1
        ReportBuildStatus: false
      Triggers:
        Webhook: true
        BuildType: BUILD_BATCH
We are working on updating or official documentation. Sorry for the confusion.

Howdy!

I attempted to update the codebuild project I am working on that requires batch builds in order to build properly and it seems that its still not executing when I run the buildspec.yml that I declare the batch in.
I used the awscli to pull the build project information and saw that the cloudformation update did work and the buildType was set to BUILD_BATCH.
Is this feature fully tested and working at this point or is there more to be done? It seems as though I have all the required parts to get this feature working.

Thanks for any assistance that can be provided.

tjenkinson · 2021-01-06T20:02:23Z

Does the config in the console show it's set to the batch mode?

paulsbnet · 2021-01-06T22:05:31Z

Does the config in the console show it's set to the batch mode?

Managed to get it working.
Needed to add these to the cloudformation in order for it to work:

      BuildBatchConfig:
        CombineArtifacts: true
        ServiceRole: !GetAtt Role.Arn

Thank you for your time though!

In order for a CodeBuild to run in batch mode, a batch service role is needed, as described [here in the docs](https://docs.aws.amazon.com/codebuild/latest/userguide/batch-build.html). >Batch builds introduce a new security role in the batch configuration. This new role is required as CodeBuild must be able to call the StartBuild, StopBuild, and RetryBuild actions on your behalf to run builds as part of a batch. Customers should use a new role, and not the same role they use in their build... At first I thought lets add this by default, but then I realised when `BatchConfiguration` is set to something, in the aws console the default 'start build' button behaviour changes to start a batch build by default instead :/ So now this adds a new `supportBatchBuildType` option, which when `true` adds minimum amount of `BatchConfiguration` needed for batch builds to run. I also updated the doc blocks for the webhook option and CodePipeline action option, because users of those also need to set this option. It would be nice to auto-enable this if a webhook or CodeBuild action is configured, but that sounds pretty complicated. I'm not sure why anyone would need to customise this role, given it appears to only be used internally to do those 3 things, so this PR does not make it configurable. My thinking is that this could be added later if needed, but this PR just gets batch builds working. In the future if people want control of the other `BatchConfiguration` options I was thinking these could be added and would require `supportBatchBuildType` to be `true`. related: aws-cloudformation/cloudformation-coverage-roadmap#621

tscully49 · 2021-02-10T19:14:23Z

Does this work with the local codebuild agent docker image? I can't seem to get it to kick off batch builds thus far.

jigar-lab · 2021-09-20T19:55:59Z

This is now supported from CFN and CDK both.

jigar-lab · 2021-09-20T21:51:57Z

Does this work with the local codebuild agent docker image? I can't seem to get it to kick off batch builds thus far.

Thanks for reporting this. We have added a backlog item to assess the feasibility and add batch build support in local codebuild agent.

wulfmann changed the title ~~AWS::CodeBuild::Project - Support Batch Builds~~ AWS::CodeBuild::Project - Support Batch Builds (From Trigger) Sep 3, 2020

tjenkinson mentioned this issue Nov 24, 2020

[aws-codebuild] support webhook ''BUILD_BATCH" BuildType aws/aws-cdk#11663

Closed

2 tasks

tjenkinson mentioned this issue Nov 27, 2020

feat(codebuild): add startBatchBuild option aws/aws-cdk#11743

Merged

tjenkinson mentioned this issue Jan 15, 2021

feat(aws-codebuild): add enableBatchBuilds() to Project aws/aws-cdk#12531

Merged

WaelA added the Coverage label Aug 3, 2021

cfn-github-issues-bot closed this as completed Feb 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS::CodeBuild::Project - Support Batch Builds (From Trigger) #621

AWS::CodeBuild::Project - Support Batch Builds (From Trigger) #621

wulfmann commented Sep 3, 2020

wulfmann commented Sep 3, 2020

tjenkinson commented Sep 14, 2020

wulfmann commented Sep 15, 2020

subinataws commented Sep 17, 2020

tjenkinson commented Sep 25, 2020

tjenkinson commented Nov 20, 2020

johnhanks1 commented Nov 23, 2020

tjenkinson commented Nov 24, 2020

tjenkinson commented Nov 30, 2020

tjenkinson commented Dec 5, 2020

tjenkinson commented Dec 11, 2020

paulsbnet commented Jan 6, 2021

tjenkinson commented Jan 6, 2021

paulsbnet commented Jan 6, 2021 •

edited

Loading

tscully49 commented Feb 10, 2021

jigar-lab commented Sep 20, 2021

jigar-lab commented Sep 20, 2021 •

edited

Loading

AWS::CodeBuild::Project - Support Batch Builds (From Trigger) #621

AWS::CodeBuild::Project - Support Batch Builds (From Trigger) #621

Comments

wulfmann commented Sep 3, 2020

1. AWS::CodeBuild::Project - Support Batch Builds

2. Scope of request

3. Expected behavior

4. Suggest specific test cases

Create a CodeBuild Project with a trigger type of STANDARD

Create a CodeBuild Project with a trigger type of BATCH

5. Helpful Links to speed up research and evaluation

6. Category (required)

wulfmann commented Sep 3, 2020

tjenkinson commented Sep 14, 2020

wulfmann commented Sep 15, 2020

subinataws commented Sep 17, 2020

tjenkinson commented Sep 25, 2020

tjenkinson commented Nov 20, 2020

johnhanks1 commented Nov 23, 2020

tjenkinson commented Nov 24, 2020

tjenkinson commented Nov 30, 2020

tjenkinson commented Dec 5, 2020

tjenkinson commented Dec 11, 2020

paulsbnet commented Jan 6, 2021

tjenkinson commented Jan 6, 2021

paulsbnet commented Jan 6, 2021 • edited Loading

tscully49 commented Feb 10, 2021

jigar-lab commented Sep 20, 2021

jigar-lab commented Sep 20, 2021 • edited Loading

paulsbnet commented Jan 6, 2021 •

edited

Loading

jigar-lab commented Sep 20, 2021 •

edited

Loading