Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⭐️ GCP resources as assets P0 #819

Merged
merged 13 commits into from
Jan 30, 2023
Merged

⭐️ GCP resources as assets P0 #819

merged 13 commits into from
Jan 30, 2023

Conversation

imilchev
Copy link
Member

@imilchev imilchev commented Jan 23, 2023

The following GCP assets are added as potential discovery options when scanning GCP:

  • Compute images ./cnquery shell gcp --discover compute-images
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-compute-image"
  runtime: "gcp"
  title: "GCP Compute Image"
}
  • GKE clusters ./cnquery shell gcp --discover gke-clusters
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-gke-cluster"
  runtime: "gcp"
  title: "GCP GKE Cluster"
}
  • VPC firewalls ./cnquery shell gcp --discover compute-firewalls
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-compute-firewall"
  runtime: "gcp"
  title: "GCP Compute Firewall"
}
  • GCS buckets ./cnquery shell gcp --discover storage-buckets
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-storage-bucket"
  runtime: "gcp"
  title: "GCP Storage Bucket"
}
  • VPC networks ./cnquery shell gcp --discover compute-networks
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-compute-network"
  runtime: "gcp"
  title: "GCP Compute Network"
}
  • VPC subnetworks./cnquery shell gcp --discover compute-subnetworks
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-compute-subnetwork"
  runtime: "gcp"
  title: "GCP Compute Subnetwork"
}
  • BigQuery datasets ./cnquery shell gcp --discover bigquery-datasets
cnquery> platform{name title runtime kind}
platform: {
  kind: "gcp-object"
  name: "gcp-bigquery-dataset"
  runtime: "gcp"
  title: "GCP BigQuery Dataset"
}

@imilchev imilchev force-pushed the ivan/gcp-assets-1 branch 3 times, most recently from e7ecbab to 0c3d33e Compare January 24, 2023 15:11
@imilchev imilchev marked this pull request as ready for review January 25, 2023 15:51
@imilchev imilchev requested a review from a team as a code owner January 25, 2023 15:51
motor/discovery/gcp/mql_asset_objects.go Show resolved Hide resolved
motor/discovery/gcp/mql_assets.go Outdated Show resolved Hide resolved
motor/discovery/gcp/discovery.go Show resolved Hide resolved
resources/packs/gcp/compute.go Outdated Show resolved Hide resolved
motor/discovery/gcp/mql_assets.go Show resolved Hide resolved
}

func NewMQLAssetsDiscovery(provider *gcpprovider.Provider) (*MqlDiscovery, error) {
m, err := motor.New(provider)
Copy link
Contributor

@preslavgerchev preslavgerchev Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not yet sure if this will work if you pass credentials to authenticate, I had the same trouble with Azure.

Basically, the secret in the provider is unresolved and you need to resolve this by using the credentials query that the resolver gets passed.

This is the same reason why we use motor.NewMotorConnection in the resolver directly instead of motor.New.

Tldr: Should probably be using motor.NewMotorConnection here

Edit: It actually depends on how you're passing the provider down to these objects. If you're using the one that you get initially from calling motor.NewMotorConnection then it is probably fine 🤷

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm not sure i understand -- can you give an example of what you mean when you say "pass credentials to authenticate", like the command you're using/thinking of?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can describe that. I have this inventory file for GCP:

apiVersion: v1
kind: Inventory
metadata:
  name: inventory
spec:
  assets:
    - name: cool-stuff
      connections:
        - backend: 13
          credentials:
            - secret_id: storage/random-bucket2/foo
              type: 6
              secret_encoding: 3
          options:
          discover:
            targets:
              - compute-images
              - projects
  vault:
    name: gcp-berglas
    type: gcp-berglas
    options:
      project_id: mondoo-dev-262313

That breaks with discovering assets because the secret isn't resolved by default

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thanks for the example!

@imilchev imilchev force-pushed the ivan/gcp-assets-1 branch 2 times, most recently from 6b608f9 to c4ed964 Compare January 26, 2023 14:10
Copy link
Contributor

@czunker czunker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for all the new assets 🎉

@vjeffrey
Copy link
Contributor

vjeffrey commented Jan 27, 2023

lookin good! do you have a policy ready that we can test this with?

@vjeffrey
Copy link
Contributor

i'm gonna give this another local test after planning this morning so we can get to merging!

@imilchev imilchev merged commit e7858f0 into main Jan 30, 2023
@imilchev imilchev deleted the ivan/gcp-assets-1 branch January 30, 2023 21:54
@github-actions github-actions bot locked and limited conversation to collaborators Jan 30, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants