Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 ETL fails when "Build: enabled: False #318

Closed
psgivens opened this issue Jan 21, 2021 · 1 comment · Fixed by #335
Closed

S3 ETL fails when "Build: enabled: False #318

psgivens opened this issue Jan 21, 2021 · 1 comment · Fixed by #335
Assignees
Milestone

Comments

@psgivens
Copy link

psgivens commented Jan 21, 2021

An example such as the following fails because the source output artifact doesn't match the deploy input artifact. I've provided the solution in this ticket.

- name: sample-s3-pipeline
  default_providers:
    source:
      provider: s3
      properties:
        bucket_name: packer-bucket-test
        object_key: input.zip
        account_id: 44444444444444
    build:
      enabled: False # if you wish to disable the build phase in a pipeline
    deploy:
      provider: s3
  targets:
    - target: 222222222222
      properties:
        bucket_name: test-adf-bucket-eu
        object_key: output.zip

S3 output artifact is "output-source".

    if self.category == 'Source':
        action_props["output_artifacts"] = [
            _codepipeline.CfnPipeline.OutputArtifactProperty(
                name="output-source"
            )
        ]

When Build is set to "enabled: false", CodeBuild is skipped and the S3 deploy's input artifact is "{0}-build".format(self.map_params['name']

    if self.category == 'Deploy':
        action_props["input_artifacts"] = [
            _codepipeline.CfnPipeline.InputArtifactProperty(
                name="{0}-build".format(self.map_params['name'])
            )
        ]

The generated CloudFormation for the pipeline fails with a complaint that there isn't an output format for "{0}-build".

Solution: Add a build the following block when building the Deploy action after the initial set block..

        if not self.map_params.get('default_providers', {}).get('build', {}).get('enabled', True):
            action_props["input_artifacts"] = [
                _codepipeline.CfnPipeline.InputArtifactProperty(
                    name="output-source"
                )
            ]
@sbkok sbkok self-assigned this Feb 26, 2021
@sbkok sbkok added this to the v3.1.1 milestone Feb 26, 2021
@sbkok
Copy link
Collaborator

sbkok commented Feb 26, 2021

I'm working on a fix for this problem, will update once it is ready.

sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Mar 3, 2021
…sabled

**Why?**

When the build stage is disabled, the deploy stages incorrectly tried to
use the build output artifact. These should use the source artifact
instead.

**Linked issues**

* awslabs#236
* awslabs#318
sbkok added a commit that referenced this issue Mar 3, 2021
…sabled (#334)

**Why?**

When the build stage is disabled, the deploy stages incorrectly tried to
use the build output artifact. These should use the source artifact
instead.

**Linked issues**

* #236
* #318
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Mar 3, 2021
**Fixes 🐞**

* Fixes `timeout` and `environment_variables` to be used when defined in
  the default CodeBuild Deployment provider properties awslabs#307, closes awslabs#306.
* Fixes intrinsic functions for account_region param files awslabs#333, closes awslabs#147.
* Fixes use of deployment from source directly when build stage is
  disabled awslabs#334, closes awslabs#236 and closes awslabs#318.
@sbkok sbkok mentioned this issue Mar 3, 2021
@sbkok sbkok closed this as completed in #335 Mar 3, 2021
sbkok added a commit that referenced this issue Mar 3, 2021
**Fixes 🐞**

* Fixes `timeout` and `environment_variables` to be used when defined in
  the default CodeBuild Deployment provider properties #307, closes #306.
* Fixes intrinsic functions for account_region param files #333, closes #147.
* Fixes use of deployment from source directly when build stage is
  disabled #334, closes #236 and closes #318.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants