Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add option allowing usage of custom policy check tools #3765

Merged
merged 4 commits into from
Oct 6, 2023
Merged

feat: Add option allowing usage of custom policy check tools #3765

merged 4 commits into from
Oct 6, 2023

Conversation

bgalkows
Copy link
Contributor

@bgalkows bgalkows commented Sep 13, 2023 •
edited
Loading

what

  • Adds custom_policy_check setting in repo config and atlantis.yaml
  • Allows overrides for custom_policy_check in atlantis.yaml
  • Renames ConftestOutput field to PolicyOutput in PolicySetResult struct for clarity
  • When set, project_command_runner no longer tries to parse the policy output as a JSON object directly into a struct assuming Conftest output format
    • Instead of parsing from JSON, assigns custom tool output into the struct directly
    • Custom policy output is parsed for "fail" substrings to determine passing

why

  • This change enables users to leverage other policy tools besides Conftest in their custom workflows
    • Other tools will not have the same level of Atlantis integration, but they will at least run without errors in a policy check and interact with the approve_policies command

tests

  • I have tested this in a private repo, both with the server-side setting and atlantis.yaml override setting. Approve_policies behavior is unaffected.
  • Open to any suggestions for adding a test case where relevant. I didn't see many tests for policy check steps to use for reference.
    • This PR's only behavior change is within the Policy Check step, when the flag is set

references

@bgalkows bgalkows requested a review from a team as a code owner September 13, 2023 16:39
@github-actions github-actions bot added docs Documentation go Pull requests that update Go code labels Sep 13, 2023
@bgalkows bgalkows marked this pull request as draft September 13, 2023 16:47
@bgalkows bgalkows changed the title Add option allowing usage of custom policy check tools feat: Add option allowing usage of custom policy check tools Sep 13, 2023
jamengual
jamengual reviewed Sep 13, 2023
View reviewed changes
Copy link
Contributor

@jamengual jamengual left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@X-Guardian I know you worked on this a bit and I was wondering if you have any input
@GenPage

runatlantis.io/docs/repo-level-atlantis-yaml.md
| repo_locking | bool | `true` | no | Get a repository lock in this project when plan. |
| repo_locking | bool | `true` | no | Get a repository lock in this project when plan.

| custom_policy_check | bool | `false` | no | Enable using policy check tools other than Conftest |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add an example on how to use this with another tool?

Copy link
Contributor Author

@bgalkows bgalkows Sep 14, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamengual Added custom-policy-checks.md with examples for both ways to set the flag and linked to it from server-side-repo-config.md

@jamengual jamengual added waiting-on-review Waiting for a review from a maintainer feature New functionality/enhancement conftest-policy labels Sep 13, 2023
@bgalkows bgalkows marked this pull request as ready for review September 14, 2023 23:46
@bgalkows bgalkows requested a review from jamengual September 17, 2023 03:21
GenPage
GenPage reviewed Sep 25, 2023
View reviewed changes
Copy link
Member

@GenPage GenPage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't a big fan of trying to jam custom policy check support into the existing model, however I understand the need to have support for other tooling. I'm going to leave this open for a couple weeks to gather other feedback from the community before deciding on whether to merge this as-is or not.

@GenPage GenPage added needs discussion Large change that needs review from community/maintainers work-in-progress and removed waiting-on-review Waiting for a review from a maintainer labels Sep 25, 2023
@bgalkows
Copy link
Contributor Author

bgalkows commented Oct 6, 2023

Updated the PolicySetResult struct's fields to be more general for clarity when used with custom tooling

@bgalkows bgalkows force-pushed the bgalkows/custom-policy-check branch from 298d708 to d29ed31 Compare October 6, 2023 06:06
@GenPage GenPage merged commit 22060fe into runatlantis:main Oct 6, 2023
14 checks passed
ijames-gc pushed a commit to gocardless/atlantis that referenced this pull request Feb 13, 2024
@bgalkows @ijames-gc
feat: Add option allowing usage of custom policy check tools (runatla…
7fbcfde 
…ntis#3765)

* Adding new flag everywhere relevant, implementing policy result workaround

* Fixing unit test str matching, adding custom policy conditional to step_runner

* Adding documentation steps for custom policy tools

* Refactoring ConftestOutput attribute to PolicyOutput
ijames-gc pushed a commit to gocardless/atlantis that referenced this pull request Feb 13, 2024
@bgalkows @ijames-gc
feat: Add option allowing usage of custom policy check tools (runatla…
e357783 
…ntis#3765)

* Adding new flag everywhere relevant, implementing policy result workaround

* Fixing unit test str matching, adding custom policy conditional to step_runner

* Adding documentation steps for custom policy tools

* Refactoring ConftestOutput attribute to PolicyOutput
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GenPage GenPage GenPage approved these changes

@jamengual jamengual Awaiting requested review from jamengual

Assignees
No one assigned
Labels
conftest-policy docs Documentation feature New functionality/enhancement go Pull requests that update Go code needs discussion Large change that needs review from community/maintainers work-in-progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Custom policy checks with non-JSON output produce errors
3 participants
@bgalkows @jamengual @GenPage