Codepipeline Bitbucket Integration #11389

vibhuyadav · 2019-12-20T16:13:07Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Codepipeline now supports integration with Bitbucket. Please refer this announcement
https://aws.amazon.com/about-aws/whats-new/2019/12/aws-codepipeline-now-supports-atlassian-bitbucket-cloud/

New or Affected Resource(s)

aws_codepipeline

Potential Terraform Configuration

resource "aws_codepipeline" "this" {
  name     = "${var.codecommit_name}-pipeline"
  role_arn = var.codepipeline_role_arn

  artifact_store {
    [..]
  }

  stage {
    name = "Source"

    action {
      name             = "Source"
      category         = "Source"
      owner            = "ThirdParty"
      provider         = "Bitbucket"
      version          = "1"
      output_artifacts = ["source_output"]

      configuration = {
        Owner  = "my-organization"
        Repo   = "test"
        Branch = "master"
      }
    }
    [..]
  }

  stage {
    name = "Build"

    action {
      name            = "BuildFrontend"
      category        = "Build"
      owner           = "AWS"
      provider        = "CodeBuild"
      input_artifacts = ["source"]

      output_artifacts = [
        [..]
      ]

      version = "1"

      configuration = {
        ProjectName = aws_codebuild_project.this.name
      }
    }

    action {
      name            = "BuildBackend"
      category        = "Build"
      owner           = "AWS"
      provider        = "CodeBuild"
      input_artifacts = ["source"]

      output_artifacts = [
        [..]
      ]

      version = "1"

      configuration = {
        ProjectName = aws_codebuild_project.this.name
      }
    }
  }

  stage {
    name = "DeployDevelopment"

    action {
      [..]
    }

    action {
      [..]
    }
  }

bflad · 2019-12-20T16:51:34Z

I believe this may also relate to the new CodeStar Connections service added in AWS Go SDK v1.26.6. If so, see also #11375 and https://github.com/terraform-providers/terraform-provider-aws/blob/master/.github/CONTRIBUTING.md#new-service

vibhuyadav · 2020-01-14T23:44:43Z

You are probably right. Here is the json response for Codepipeline set with CodeStarSourceConnection. The pipeline was created by hand on the console.

$ aws codepipeline get-pipeline --name my-example --region us-east-1 > issue_11389.json

{ "pipeline": { "roleArn": "arn:aws:iam::111111111111:role/code-pipeline-role", "stages": [ { "name": "Bitbucket", "actions": [ { "inputArtifacts": [], "name": "bitbucket", "region": "us-east-1", "actionTypeId": { "category": "Source", "owner": "AWS", "version": "1", "provider": "CodeStarSourceConnection" }, "outputArtifacts": [ { "name": "bitbucket-artifact" } ], "configuration": { "ConnectionArn": "arn:aws:codestar-connections:us-east-1:111111111111:connection/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx", "FullRepositoryId": "foo-bar/my-example", "BranchName": "master", "OutputArtifactFormat": "CODEBUILD_CLONE_REF" }, "runOrder": 1 } ] }, ... ...

laghao · 2020-01-20T16:24:36Z

Is bitbucket provider supported already in resource "aws_codepipeline"?

romeroadrian · 2020-01-20T17:09:29Z

I've managed to get bitbucket working using something like the following:

stage {
    name = "Source"

    action {
      name             = "Source"
      category         = "Source"
      owner            = "AWS"
      provider         = "CodeStarSourceConnection"
      version          = "1"
      output_artifacts = ["source"]

      configuration = {
        BranchName           = "master"
        ConnectionArn        = "the connection arn"
        FullRepositoryId     = "foo/bar"
        OutputArtifactFormat = "CODE_ZIP"
      }
    }
  }

I had to previously do the oauth thing to connect aws with bitbucket and manually copy the arn of the codestar connection (the aws provider doesn't support code star resources yet afaik)

vibhuyadav · 2020-01-20T23:24:07Z

That is right. I can confirm this works
` stage {
name = "Source"

action {
  name             = "Source"
  category         = "Source"
  owner            = "AWS"
  provider         = "CodeStarSourceConnection"
  version          = "1"
  output_artifacts = ["SourceArtifact"]

  configuration = {
    ConnectionArn        = "arn:aws:codestar-connections:us-east-1:${var.aws_account_id}:connection/89cb0638-6cf7-4e37-9f11-228803e7aec7"
    FullRepositoryId     = "extron-electronics/${var.project_name}"
    BranchName           = "develop"
    OutputArtifactFormat = "CODEBUILD_CLONE_REF"
  }
}

}`

mavencode01 · 2020-02-17T18:05:01Z

@vibhuyadav I tried configuring the Bitbucket source like you defined but am getting error message The provided role does not have sufficient permissions

Could you please share the role / policy you used to create the Bitbucket connection ?

Thank you.

vibhuyadav · 2020-02-19T21:38:15Z

@mavencode01 Make sure you have allow on codestar-connections:UseConnection on the policy

marcoslopesbritto · 2020-04-16T02:03:11Z

Dear vibhuyadav, what policy do you refer to?

marcoslopesbritto · 2020-04-16T02:05:23Z

I saw all IAM's roles (CodeDeploy, CloudFormation, ToolChain and PermissionBoundary), but I couldn't find the correct to put a codestar-connections:UseConnection statement.

vibhuyadav · 2020-04-16T20:18:56Z

@marcoslopesbritto I just created a new customer-managed policy. Here is what all I have in the policy and I attached it to the role behind the pipeline.

resource "aws_iam_role_policy" "codepipeline_policy" {
  name = "codepipeline_policy"
  role = aws_iam_role.codepipeline_service_role.id

  policy = <<EOF
{
    "Statement": [
        {
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "*",
            "Effect": "Allow",
            "Condition": {
                "StringEqualsIfExists": {
                    "iam:PassedToService": [
                        "cloudformation.amazonaws.com",
                        "elasticbeanstalk.amazonaws.com",
                        "ec2.amazonaws.com",
                        "ecs-tasks.amazonaws.com"
                    ]
                }
            }
        },
        {
            "Action": [
                "codecommit:CancelUploadArchive",
                "codecommit:GetBranch",
                "codecommit:GetCommit",
                "codecommit:GetUploadArchiveStatus",
                "codecommit:UploadArchive",
                "codestar-connections:*"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "codedeploy:CreateDeployment",
                "codedeploy:GetApplication",
                "codedeploy:GetApplicationRevision",
                "codedeploy:GetDeployment",
                "codedeploy:GetDeploymentConfig",
                "codedeploy:RegisterApplicationRevision"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "elasticbeanstalk:*",
                "ec2:*",
                "elasticloadbalancing:*",
                "autoscaling:*",
                "cloudwatch:*",
                "s3:*",
                "sns:*",
                "cloudformation:*",
                "rds:*",
                "sqs:*",
                "ecs:*"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "lambda:InvokeFunction",
                "lambda:ListFunctions"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "opsworks:CreateDeployment",
                "opsworks:DescribeApps",
                "opsworks:DescribeCommands",
                "opsworks:DescribeDeployments",
                "opsworks:DescribeInstances",
                "opsworks:DescribeStacks",
                "opsworks:UpdateApp",
                "opsworks:UpdateStack"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:DeleteStack",
                "cloudformation:DescribeStacks",
                "cloudformation:UpdateStack",
                "cloudformation:CreateChangeSet",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:SetStackPolicy",
                "cloudformation:ValidateTemplate"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "codebuild:BatchGetBuilds",
                "codebuild:StartBuild"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Effect": "Allow",
            "Action": [
                "devicefarm:ListProjects",
                "devicefarm:ListDevicePools",
                "devicefarm:GetRun",
                "devicefarm:GetUpload",
                "devicefarm:CreateUpload",
                "devicefarm:ScheduleRun"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "servicecatalog:ListProvisioningArtifacts",
                "servicecatalog:CreateProvisioningArtifact",
                "servicecatalog:DescribeProvisioningArtifact",
                "servicecatalog:DeleteProvisioningArtifact",
                "servicecatalog:UpdateProduct"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:ValidateTemplate"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ecr:DescribeImages"
            ],
            "Resource": "*"
        }
    ],
    "Version": "2012-10-17"
}
EOF

}

ghost · 2020-06-19T03:30:37Z

@marcoslopesbritto I just created a new customer-managed policy. Here is what all I have in the policy and I attached it to the role behind the pipeline.

Yep that's the default policy AWS put.
Some caveats:

You probably don't need most of the services that it opens
Won't work if you use KMS to store your artifacts in S3

To add KMS support, simply add the following block to the previous policy:

 {
     "Sid": "kms",
      "Effect": "Allow",
      "Action": [
          "kms:Decrypt",
          "kms:DescribeKey",
          "kms:Encrypt",
          "kms:GenerateDataKey*",
          "kms:ReEncrypt*"
      ],
      "Resource": "[your KMS key arn]"
  }

Here is the one I have been using, which gives you usage of :

CodeBuild
CodeDeploy
CodeStar connections
KMS
Lambda calls
ServiceCatalog
S3
SNS

policy = <<-EOF
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "s3",
        "Effect": "Allow",
        "Action": [
          "s3:*"
        ],
        "Resource": [
          "${aws_s3_bucket.codepipeline_bucket.arn}",
          "${aws_s3_bucket.codepipeline_bucket.arn}/*"
        ]
      },
      {
        "Sid": "codebuild",
        "Effect": "Allow",
        "Action": [
          "codebuild:BatchGetBuilds",
          "codebuild:StartBuild"
        ],
        "Resource": "*"
      },
      {
        "Sid": "codedeploy",
        "Effect": "Allow",
        "Action": [
          "codedeploy:CreateDeployment",
          "codedeploy:GetApplication",
          "codedeploy:GetApplicationRevision",
          "codedeploy:GetDeployment",
          "codedeploy:GetDeploymentConfig",
          "codedeploy:RegisterApplicationRevision"
        ],
        "Resource": "*"
      },
      {
        "Sid": "codestar",
        "Effect": "Allow",
        "Action": [
          "codestar-connections:UseConnection"
        ],
        "Resource": "*"
      },
      {
        "Sid": "ecr",
        "Effect": "Allow",
        "Action": [
          "ecr:DescribeImages"
        ],
        "Resource": "*"
      },
      {
          "Sid": "kms",
          "Effect": "Allow",
          "Action": [
              "kms:Decrypt",
              "kms:DescribeKey",
              "kms:Encrypt",
              "kms:GenerateDataKey*",
              "kms:ReEncrypt*"
          ],
          "Resource": "[your KMS key arn]"
      },
      {
        "Sid": "lambda",
        "Effect": "Allow",
        "Action": [
          "lambda:InvokeFunction",
          "lambda:ListFunctions"
        ],
        "Resource": "*"
      },
      {
        "Sid": "servicecatalog",
        "Effect": "Allow",
        "Action": [
          "servicecatalog:CreateProvisioningArtifact",
          "servicecatalog:DeleteProvisioningArtifact",
          "servicecatalog:DescribeProvisioningArtifact",
          "servicecatalog:ListProvisioningArtifacts",
          "servicecatalog:UpdateProduct"
        ],
        "Resource": "*"
      },
      {
        "Sid": "sns",
        "Effect": "Allow",
        "Action": [
          "sns:CheckIfPhoneNumberIsOptedOut",
          "sns:CreateTopic",
          "sns:GetEndpointAttributes",
          "sns:GetPlatformApplicationAttributes",
          "sns:GetSMSAttributes",
          "sns:GetSubscriptionAttributes",
          "sns:GetTopicAttributes",
          "sns:ListEndpointsByPlatformApplication",
          "sns:ListPhoneNumbersOptedOut",
          "sns:ListPlatformApplications",
          "sns:ListSubscriptions",
          "sns:ListSubscriptionsByTopic",
          "sns:ListTagsForResource",
          "sns:ListTopics",
          "sns:Publish",
          "sns:SetTopicAttributes",
          "sns:TagResource",
          "sns:UntagResource"
        ],
        "Resource": "*"
      }
    ]
  }
EOF

Could be simplified / secured even further, especially with targeting the exact resources that CodePipeline need access to.

ghost · 2020-12-18T22:26:42Z

This has been released in version 3.22.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2021-01-15T17:10:01Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

vibhuyadav added the enhancement Requests to existing resources that expand the functionality or scope. label Dec 20, 2019

ghost added the service/codepipeline Issues and PRs that pertain to the codepipeline service. label Dec 20, 2019

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Dec 20, 2019

bflad removed the needs-triage Waiting for first response or review from a maintainer. label Dec 20, 2019

ksenechal mentioned this issue Apr 4, 2020

#11389 Add codestar connection resource #12637

Closed

bflad mentioned this issue Oct 2, 2020

Add support for CodeStar Connections #15453

Closed

5 tasks

This was referenced Nov 1, 2020

Add CodeStar Connections service client #15960

Merged

Add aws_codestarconnections_connection resource #15990

Merged

gdavison closed this as completed in #15990 Dec 15, 2020

gdavison added this to the v3.22.0 milestone Dec 15, 2020

ghost locked as resolved and limited conversation to collaborators Jan 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Codepipeline Bitbucket Integration #11389

Codepipeline Bitbucket Integration #11389

vibhuyadav commented Dec 20, 2019

bflad commented Dec 20, 2019

vibhuyadav commented Jan 14, 2020

laghao commented Jan 20, 2020

romeroadrian commented Jan 20, 2020 •

edited

Loading

vibhuyadav commented Jan 20, 2020

mavencode01 commented Feb 17, 2020

vibhuyadav commented Feb 19, 2020

marcoslopesbritto commented Apr 16, 2020

marcoslopesbritto commented Apr 16, 2020 •

edited

Loading

vibhuyadav commented Apr 16, 2020 •

edited

Loading

ghost commented Jun 19, 2020

ghost commented Dec 18, 2020

ghost commented Jan 15, 2021

Codepipeline Bitbucket Integration #11389

Codepipeline Bitbucket Integration #11389

Comments

vibhuyadav commented Dec 20, 2019

Community Note

Description

New or Affected Resource(s)

Potential Terraform Configuration

bflad commented Dec 20, 2019

vibhuyadav commented Jan 14, 2020

laghao commented Jan 20, 2020

romeroadrian commented Jan 20, 2020 • edited Loading

vibhuyadav commented Jan 20, 2020

mavencode01 commented Feb 17, 2020

vibhuyadav commented Feb 19, 2020

marcoslopesbritto commented Apr 16, 2020

marcoslopesbritto commented Apr 16, 2020 • edited Loading

vibhuyadav commented Apr 16, 2020 • edited Loading

ghost commented Jun 19, 2020

ghost commented Dec 18, 2020

ghost commented Jan 15, 2021

romeroadrian commented Jan 20, 2020 •

edited

Loading

marcoslopesbritto commented Apr 16, 2020 •

edited

Loading

vibhuyadav commented Apr 16, 2020 •

edited

Loading