v0.22.0+: approve_policies: results in error "Pull request must be mergeable before running apply."

[pseudomorph]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Running atlantis approve_policies results in failed apply status messages.

Reproduction Steps

Run no-op plan on stack, then approve policies.

Logs

{
  "level": "error",
  "ts": "2023-01-12T18:12:38.776Z",
  "caller": "events/instrumented_project_command_runner.go:78",
  "msg": "Failure running approve policies operation: Pull request must be mergeable before running apply.",
  "json": {
    "repo": "****",
    "pull": "53"
  },
  "stacktrace": "see below"
}

error from stacktrace

github.com/runatlantis/atlantis/server/events.RunAndEmitStats
	github.com/runatlantis/atlantis/server/events/instrumented_project_command_runner.go:78
github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).ApprovePolicies
	github.com/runatlantis/atlantis/server/events/instrumented_project_command_runner.go:49
github.com/runatlantis/atlantis/server/events.(*ApprovePoliciesCommandRunner).buildApprovePolicyCommandResults
	github.com/runatlantis/atlantis/server/events/approve_policies_command_runner.go:102
github.com/runatlantis/atlantis/server/events.(*ApprovePoliciesCommandRunner).Run
	github.com/runatlantis/atlantis/server/events/approve_policies_command_runner.go:73
github.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand
	github.com/runatlantis/atlantis/server/events/command_runner.go:296"}

Environment details

• Atlantis version: v0.22.2
• Atlantis flags: atlantis server --config /app/files/config.yaml --repo-config /app/files/repositories.yaml --enable-policy-checks

Atlantis server-side config file:

atlantis-url: ****
allow-draft-prs: true
allow-fork-prs: true
checkout-strategy: merge
automerge: true
gh-user: ****
log-level: debug
port: 3000
repo-whitelist: ****
hide-prev-plan-comments: true

Additional Context

The policy checks work as expected, but something seems to be triggering an apply in code. None of our default or custom workflows are specifying anything for apply workflow. I cannot find any connection which would implicate this behavior between workflow steps or command running in code. I'm a bit stumped.

[nitrocode]

@pseudomorph

I wonder if this is a regression. Did you see this issue in previous versions? Is this issue present in v0.22.1 ?

[pseudomorph]

Looks like this is the case. I lowered version down to v0.20.0 and the problem went away.

[nitrocode]

Could you try 0.22.1 and keep going lower until it's resolved? I'm curious where this regression came from.

[nitrocode]

Here are some of the approve_policies PRs closed recently https://github.com/runatlantis/atlantis/pulls?q=is%3Apr+approve_policies+is%3Aclosed

[pseudomorph]

Worked my way up from v0.21.0. It appears to show up when I run with v0.22.0

[nitrocode]

Hmm and then if you have more time, to further see if we can identify the correct pr, could you check the following pre releases?

this are the pre-releaes between v0.21.0 (nov 28) and v0.22.0 (jan 11)

• this one was incorrectly named v0.21.0-pre.20221207
• v0.21.1-pre.20221213
• v0.22.0-pre.20221219
• v0.22.0-pre.20221226

Thank you

[nitrocode]

@krrrr38 is it possible that this issue resulted after merging #2939 or #2877. I'm having trouble tracking down the exact change.

atlantis/server/events/instrumented_project_command_runner.go

Lines 76 to 80 in c94e63c

	if result.Failure != "" {
	executionFailure.Inc(1)
	logger.Err("Failure running %s operation: %s", commandName, result.Failure)
	return result
	}

[nitrocode]

@pseudomorph is your PR mergeable? Were you able to run approve policies without needing your pr to be mergesble before?

[krrrr38]

@nitrocode this.line is invalid
https://github.com/runatlantis/atlantis/blob/main/server/events/instrumented_project_command_runner.go#L49-L49

[nitrocode]

Ah thank you @krrrr38 so then this regression came from pr #2767 cc @Fabianoshz

[pseudomorph]

Wow. Awesome work and collaboration here y'all!

[Fabianoshz]

Hi guys, that seems to be a quick fix, give me a minute and I will open a PR.

[nitrocode]

it looks like @Fabianoshz fixed this issue very quickly. Nice job!

@pseudomorph once the new image is uploaded, could you try the atlantis image dev tag? Thats the tag that's updated whenever a new PR is merged.

[pseudomorph]

I tested against the built image and confirmed the issue is gone.