[bug]: github_branch does not refresh state if branch have been deleted

[nikophil]

Hello,

I'm trying to update some files in the repositories with terraform.

What I want to achieve is to create a PR on my repositories with some modified files. The problem occurs when the PR gets merged and the branch is deleted by github itself. When applying another time I get a 404 error when creating the file.

Terraform Version

$ tf -v
Terraform v1.0.11
on linux_amd64
+ provider registry.terraform.io/hashicorp/github v4.18.0
+ provider registry.terraform.io/integrations/github v4.18.0

Affected Resource(s)

Please list the resources as a list, for example:

• github_branch
• github_repository_file

Terraform Configuration Files

resource "github_branch" "dependabot" {
  repository = "repo"
  branch     = "terraform-dependabot"
}

resource "github_repository_file" "dependabot" {
  repository = "repo"
  branch              = "terraform-dependabot"
  file                = ".github/dependabot.yml"
  content             = file("github-conf-files/dependabot.yml")
  commit_message      = "chore: Dependabot file updated by terraform"
  commit_author       = data.github_user.self.login
  commit_email        = "[email protected]"
  overwrite_on_create = true
}

resource "github_repository_pull_request" "dependabot" {
  repository = "repo"
  base_ref   = "develop"
  head_ref   = "terraform-dependabot"
  title      = "chore: Dependabot file updated by terraform"
}

Expected Behavior

The resource "github_branch" directive should refresh its state if the branch have been deleted. And then the branch should be re-created and the file should be created afterwards.

Actual Behavior

│ Error: unexpected status code: 404 Not Found
│ 
│   with github_repository_file.dependabot,
│   on repository-config-files.tf line 13, in resource "github_repository_file" "dependabot":
│   13: resource "github_repository_file" "dependabot" {

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply # will create the branch / the file / the PR
2. merge the PR with delete_branch_on_merge option activated
3. terraform apply # error 404 when trying to create the file: the branch does not exist

Thanks for your help :)

[toelke]

FWIW, the behaviour is the same since at least 4.2.0 (the earliest version currently available in the terraform registry), but the error message was much clearer until including version 4.13.0:

╷
│ Error: Branch [redacted] not found in repository [redacted] or repository is not readable
│ 
│   with module.[redacted],
│   on modules/[redacted].tf line 5, in resource "github_repository_file" "[redacted]":
│    5: resource "github_repository_file" "[redacted]" {
│ 
╵

Does anyone have a working solution for how to handle deleted branches? E.g. when they are deleted after a PR?

[mcintyre321]

@toelke Did you find a solution for this?

[toelke]

"Yes". We have this in our README:

When terraform gives you cryptic 404 errors, then the branch terraform was deleted in some repositoy.
Either re-create the branch in the repository manually or remove the affected github_repository_file with terraform state rm "module.[...].github_repository_file.[...]".

We are also toying with the idea of using this abomination which we call "very hack, such curl":

resource "null_resource" "local-exec" {
  for_each = var.files
  provisioner "local-exec" {
    interpreter = ["bash", "-exuc"]
    command     = <<EOC
      # first, check if the file exists in the default-branch and has the correct content
      if [ $(sha1sum <(curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/${var.repository}/contents/${each.key} | jq -r '.content' | base64 -d) | awk '{ printf $1;}') == ${sha1(each.value)} ]; then
        exit 0
      fi
      # then, check if the file exists in the branch and has the correct content
      if [ $(sha1sum <(curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/${var.repository}/contents/${each.key}?ref=${var.branch} | jq -r '.content' | base64 -d) | awk '{ printf $1;}') == ${sha1(each.value)} ]; then
        exit 0
      fi
      branch_is_new=false
      # Check if the target branch exists and create it if not
      if [ "$(curl -s -w "%%{http_code}" -X HEAD -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/${var.repository}/branches/${var.branch})" == "404" ]; then
        default_branch_sha=$(curl -s -X GET -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/${var.repository}/branches/${var.default_branch} | jq -r .commit.sha)
        curl -X POST -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/${var.repository}/git/refs -d '{"ref": "refs/heads/${var.branch}", "sha": "'$default_branch_sha'"}'
        branch_is_new=true
      fi
      # Commit the file
      curl -H "Authorization: token $GITHUB_TOKEN" -X PUT https://api.github.com/repos/${var.repository}/contents/${each.key} -d '{"message": "commit from terraform", "content": "${base64encode(each.value)}", "branch": "${var.branch}" }'
      # If necessary, create a PR
      if [ $branch_is_new == "true" ]; then
        curl -H "Authorization: token $GITHUB_TOKEN" -X POST https://api.github.com/repos/${var.repository}/pulls -d '{"title": "Updated file via terraform", "head": "${var.branch}", "base": "${var.default_branch}" }'
      fi
    EOC
  }

  triggers = {
    always_run = "${timestamp()}"
  }
}

[mcintyre321]

😎 very nifty

I'm also trying to model the "i want this file on master, if it doesn't exist, raise a PR and wait for it to be merged". Which despite being a valid expression of desired state, is a bit of a square peg round hole situation given the resource types we have available.

Maybe need a custom provider here...

[toelke]

Well, my curl-snippets does exactly that: Check if the file has the correct content in the default-branch and if not, create a branch, create the file and create a PR...

[mcintyre321]

Yes, I think that's pretty great what you've got there tbh, but am I right in thinking that it won't work for managing lifetime, i.e. if I delete a source file or do `terraform destroy` it won't clean up? Maybe I can do a when: destroy provisioner based off your code 🤔

…

On Wed, 23 Feb 2022, 12:25 Philipp Riederer, ***@***.***> wrote: Well, my curl-snippets does exactly that: Check if the file has the correct content in the default-branch and if not, create a branch, create the file and create a PR... — Reply to this email directly, view it on GitHub <#981 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACDJ6WNNUDFR5AEBID25MLU4TGZ3ANCNFSM5IMTTIQQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[toelke]

Yes, you are totally right about that.

[github-actions]

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the Status: Pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!

[merkata]

FWIW here's something that works for me:

protect the branch via a branch protection rule like

resource "github_branch_protection" "templated-protect-files" {
  for_each      = toset(var.repo_names)
  repository_id = each.value
  
  pattern = "feature/template-files"
  allows_deletions = false
}

I'm doing this over multiple repos hence the for_each. When you raise a PR from this branch (I use a github actions workflow) and it gets merged, the branch just stays there. Not the most elegant thing, though it works until the issue is properly resolved.