Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

platform(general): Get resources from platform and filter taggable resources for policies #3621

Merged
merged 25 commits into from
Oct 30, 2022

Conversation

mikeurbanski1
Copy link
Contributor

@mikeurbanski1 mikeurbanski1 commented Oct 6, 2022

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Description

  • Gets the list of all resource types and their attributes from the platform
  • Also gets a list of attributes that we want to filter by resource type (this is just tags / labels for the various providers right now)
  • Gets the provider from the updated platform policy metadata response
  • Checkov will apply the logic to replace any policies using these attributes with a resource type of "all" with the resource types for the given attribute and provider (if available).

So, for example, if you have a platform policy with provider aws and a condition that checks tags with resource types all, then the check will only apply to taggable AWS Terraform resources. If you use the attribute Tags.*.Key, then it will only apply to CloudFormation.

If you have a Checkov policy that uses all with the attribute tags, then it will apply it to AWS, Azure, and Ali (because checkov policies do not have a provider in the spec).

This also removes the local list of resource types from checkov, which was pretty much totally dead code in reality, because platform custom policies didn't even have the provider in the API response.

Requires some platform changes (some done, some in progress) before this can be 100% safely merged (but it should still work)

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my feature, policy, or fix is effective and works
  • New and existing tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@mikeurbanski1 mikeurbanski1 changed the title platform(general) Get resources from platform and filter taggable resources for policies platform(general): Get resources from platform and filter taggable resources for policies Oct 6, 2022
@mikeurbanski1
Copy link
Contributor Author

I am in typing hell

@mikeurbanski1 mikeurbanski1 reopened this Oct 7, 2022
@mikeurbanski1 mikeurbanski1 marked this pull request as ready for review October 25, 2022 17:16
Copy link
Contributor

@tronxd tronxd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, had some comments

@mikeurbanski1
Copy link
Contributor Author

@gruebel @nimrodkor @rotemavni added you to sanity check whether this can be safely merged. I ran some tests against prod (where the platform changes do not exist) and it seems to work fine in checkov. And I don't think anything here will break a platform scan in the checkov wrapper.

Copy link
Contributor

@gruebel gruebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, great work. The only question I had, you already answered in the description 🥇

@nimrodkor nimrodkor merged commit 1cd0ed3 into master Oct 30, 2022
@nimrodkor nimrodkor deleted the get-taggable-resources-from-platform branch October 30, 2022 06:41
nimrodkor pushed a commit that referenced this pull request Oct 30, 2022
…gable resources for policies (#3621)"

This reverts commit 1cd0ed3.
nimrodkor added a commit that referenced this pull request Oct 30, 2022
…gable resources for policies" (#3762)

Revert "platform(general): Get resources from platform and filter taggable resources for policies (#3621)"

This reverts commit 1cd0ed3.
PelegLi pushed a commit that referenced this pull request Oct 30, 2022
…sources for policies (#3621)

* get attributes from the platform that should be applied to specific resources (not all)

* use a set for the taggable resource types

* add test of full policy scan

* fix tests

* remove local resource types

* cleanup comments and code

* remove unused imports

* fix mypy?

* fix mypy!

* make resource types list or set

* make resource types list or set

* change resource types to set

* resource types as list

* handle custom policies injected by platform

* save provider to resource mapping

* add abstract method shells

* mypy

* add unique orders for integrations

* resolve review comments

* fix test
PelegLi pushed a commit that referenced this pull request Oct 30, 2022
…sources for policies (#3621)

* get attributes from the platform that should be applied to specific resources (not all)

* use a set for the taggable resource types

* add test of full policy scan

* fix tests

* remove local resource types

* cleanup comments and code

* remove unused imports

* fix mypy?

* fix mypy!

* make resource types list or set

* make resource types list or set

* change resource types to set

* resource types as list

* handle custom policies injected by platform

* save provider to resource mapping

* add abstract method shells

* mypy

* add unique orders for integrations

* resolve review comments

* fix test
PelegLi pushed a commit that referenced this pull request Oct 30, 2022
…sources for policies (#3621)

* get attributes from the platform that should be applied to specific resources (not all)

* use a set for the taggable resource types

* add test of full policy scan

* fix tests

* remove local resource types

* cleanup comments and code

* remove unused imports

* fix mypy?

* fix mypy!

* make resource types list or set

* make resource types list or set

* change resource types to set

* resource types as list

* handle custom policies injected by platform

* save provider to resource mapping

* add abstract method shells

* mypy

* add unique orders for integrations

* resolve review comments

* fix test
PelegLi pushed a commit that referenced this pull request Oct 30, 2022
…sources for policies (#3621)

* get attributes from the platform that should be applied to specific resources (not all)

* use a set for the taggable resource types

* add test of full policy scan

* fix tests

* remove local resource types

* cleanup comments and code

* remove unused imports

* fix mypy?

* fix mypy!

* make resource types list or set

* make resource types list or set

* change resource types to set

* resource types as list

* handle custom policies injected by platform

* save provider to resource mapping

* add abstract method shells

* mypy

* add unique orders for integrations

* resolve review comments

* fix test
PelegLi pushed a commit that referenced this pull request Oct 30, 2022
…gable resources for policies" (#3762)

Revert "platform(general): Get resources from platform and filter taggable resources for policies (#3621)"

This reverts commit 1cd0ed3.
PelegLi pushed a commit that referenced this pull request Oct 30, 2022
…gable resources for policies" (#3762)

Revert "platform(general): Get resources from platform and filter taggable resources for policies (#3621)"

This reverts commit 1cd0ed3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants