Add link to BSidesSF 2018 talk in Readme. #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ecdavis
approved these changes
Mar 2, 2019
achantavy
added a commit
that referenced
this pull request
Sep 18, 2024
### Summary > Describe your changes. Fixes job name not being correctly passed to the graph statement from the data model. ### Checklist Provide proof that this works (this makes reviews move faster). Please perform one or more of the following: - [x] Include console log trace showing what happened before and after your changes. Before ``` INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.job:Finished job EMRCluster ``` ``` INFO:cartography.intel.aws.inspector:Running AWS Inspector cleanup INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.job:Finished job AWSInspectorFinding INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.job:Finished job AWSInspectorPackage ``` After ``` INFO:cartography.graph.statement:Completed EMRCluster statement #1 INFO:cartography.graph.statement:Completed EMRCluster statement #2 INFO:cartography.graph.job:Finished job EMRCluster ``` ``` INFO:cartography.intel.aws.inspector:Running AWS Inspector cleanup INFO:cartography.graph.statement:Completed AWSInspectorFinding statement #1 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement #2 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement #3 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement #4 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement #5 INFO:cartography.graph.job:Finished job AWSInspectorFinding INFO:cartography.graph.statement:Completed AWSInspectorPackage statement #1 INFO:cartography.graph.statement:Completed AWSInspectorPackage statement #2 INFO:cartography.graph.statement:Completed AWSInspectorPackage statement #3 INFO:cartography.graph.job:Finished job AWSInspectorPackage ```
achantavy
pushed a commit
that referenced
this pull request
Nov 19, 2024
**Summary** Mapped in [AWS Identity Center](https://aws.amazon.com/iam/identity-center/) and the access it provides to AWS accounts. New Nodes: (AWSIdentityCenter), (AWSPermissionSet), (AWSSSOUser) New Relationships: (AWSAccount)-[RESOURCE]->(AWSIdentityCenter) (AWSIdentityCenter)-[HAS_PERMISSION_SET]->(AWSPermissionSet) (AWSSSOUser)<-[ALLOWED_BY]-(AWSRole) (OktaUser)<-[CAN_ASSUME_IDENTITY]-(AWSSSOUser) (AWSPermissionSet)-[ASSIGNED_TO_ROLE]->(AWSRole)   **Console Trace** INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-east-1 INFO:cartography.intel.aws.identitycenter:Loading 1 Identity Center instances for region us-east-1 INFO:cartography.intel.aws.identitycenter:Loading 32 permission sets for instance arn:aws:sso:::instance/ssoins-72237a0dcb8c6df7 in region us-east-1 INFO:cartography.intel.aws.identitycenter:Loading 777 permission set role assignments INFO:cartography.intel.aws.identitycenter:Loading 803 SSO users for identity store d-906747a0b9 in region us-east-1 INFO:cartography.intel.aws.identitycenter:Getting role assignments for 803 users INFO:cartography.intel.aws.identitycenter:Loading 24292 role assignments INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-east-2 INFO:cartography.intel.aws.identitycenter:Loading 0 Identity Center instances for region us-east-2 INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-west-1 INFO:cartography.intel.aws.identitycenter:Loading 0 Identity Center instances for region us-west-1 INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-west-2 INFO:cartography.intel.aws.identitycenter:Loading 0 Identity Center instances for region us-west-2 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #1 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #2 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #3 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #4 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #5 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #6 **Related issues or links** Fixes - #990 Checklist Provide proof that this works (this makes reviews move faster). Please perform one or more of the following: [ x ] Update/add unit or integration tests. [ X ] Include a screenshot showing what the graph looked like before and after your changes. [ X ] Include console log trace showing what happened before and after your changes. If you are changing a node or relationship: [ x ] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md). If you are implementing a new intel module: [ X ] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node). ---------
achantavy
pushed a commit
that referenced
this pull request
Nov 20, 2024
### Summary This PR adds support to ingest dependencies from Semgrep for the NPM ecosystem, as well as introducing a CLI flag allowing users to specify which ecosystems to ingest. ### Related issues or links #1368 added support for ingesting dependencies from Semgrep (only for the `gomod` ecosystem) ### Demo Before these changes, a project with both Go and NPM dependencies will only have GoLibrary nodes in the dependency graph: <img width="1036" alt="image" src="https://github.com/user-attachments/assets/31d97626-be70-4c80-9a5b-71c26056a53b"> After these changes, for the same project the graph contains both GoLibrary and NpmLibrary nodes: <img width="1039" alt="image" src="https://github.com/user-attachments/assets/d09cc265-ccd6-463e-bd01-2b3e7c6d1778"> <details> <summary>Logs from semgrep module before these changes</summary> ``` INFO:cartography.sync:Starting sync stage 'semgrep' INFO:cartography.intel.semgrep.deployment:Loading Semgrep deployment info {'id': ...} into the graph... INFO:cartography.intel.semgrep.dependencies:Running Semgrep dependencies sync job. INFO:cartography.intel.semgrep.dependencies:Retrieving Semgrep dependencies for deployment 'X'. INFO:cartography.intel.semgrep.dependencies:Processed page 0 of Semgrep dependencies. ... INFO:cartography.intel.semgrep.dependencies:Processed page X of Semgrep dependencies. INFO:cartography.intel.semgrep.dependencies:Retrieved X Semgrep dependencies in X pages. INFO:cartography.intel.semgrep.dependencies:Loading X GoLibrary objects into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep Go Library cleanup job. INFO:cartography.graph.statement:Completed GoLibrary statement #1 ... INFO:cartography.graph.statement:Completed GoLibrary statement #X INFO:cartography.graph.job:Finished job GoLibrary INFO:cartography.intel.semgrep.findings:Running Semgrep SCA findings sync job. ... INFO:cartography.sync:Finishing sync stage 'semgrep' INFO:cartography.sync:Finishing sync with update tag '1730497895' ``` </details> <details> <summary>Logs from semgrep module after these changes</summary> ``` INFO:cartography.intel.semgrep.deployment:Loading SemgrepDeployment {'id': ...} into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep dependencies sync job. INFO:cartography.intel.semgrep.dependencies:Retrieving Semgrep gomod dependencies for deployment 'X'. INFO:cartography.intel.semgrep.dependencies:Processed page 0 of Semgrep gomod dependencies. INFO:cartography.intel.semgrep.dependencies:Processed page X of Semgrep gomod dependencies. INFO:cartography.intel.semgrep.dependencies:Retrieved X Semgrep gomod dependencies in X pages. INFO:cartography.intel.semgrep.dependencies:Loading X GoLibrary objects into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep Dependencies cleanup job for GoLibrary. INFO:cartography.graph.statement:Completed GoLibrary statement #1 INFO:cartography.graph.statement:Completed GoLibrary statement #2 INFO:cartography.graph.statement:Completed GoLibrary statement #3 INFO:cartography.graph.job:Finished job GoLibrary INFO:cartography.intel.semgrep.dependencies:Retrieving Semgrep npm dependencies for deployment 'X'. INFO:cartography.intel.semgrep.dependencies:Processed page 0 of Semgrep npm dependencies. ... INFO:cartography.intel.semgrep.dependencies:Processed page X of Semgrep npm dependencies. INFO:cartography.intel.semgrep.dependencies:Retrieved X Semgrep npm dependencies in X pages. INFO:cartography.intel.semgrep.dependencies:Loading X NpmLibrary objects into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep Dependencies cleanup job for NpmLibrary. INFO:cartography.graph.statement:Completed NpmLibrary statement #1 INFO:cartography.graph.statement:Completed NpmLibrary statement #2 INFO:cartography.graph.statement:Completed NpmLibrary statement #3 INFO:cartography.graph.job:Finished job NpmLibrary INFO:cartography.intel.semgrep.findings:Running Semgrep SCA findings sync job. ... INFO:cartography.sync:Finishing sync stage 'semgrep' INFO:cartography.sync:Finishing sync with update tag '1731969699' ``` </details> ### Checklist Provide proof that this works (this makes reviews move faster). Please perform one or more of the following: - [x] Update/add unit or integration tests. - [x] Include a screenshot showing what the graph looked like before and after your changes. - [x] Include console log trace showing what happened before and after your changes. If you are changing a node or relationship: - [x] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md). If you are implementing a new intel module: - [x] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node). --------- Signed-off-by: Hans Wernetti <[email protected]>
chandanchowdhury
pushed a commit
to chandanchowdhury/cartography
that referenced
this pull request
Nov 27, 2024
…artography-cncf#1355) ### Summary > Describe your changes. Fixes job name not being correctly passed to the graph statement from the data model. ### Checklist Provide proof that this works (this makes reviews move faster). Please perform one or more of the following: - [x] Include console log trace showing what happened before and after your changes. Before ``` INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.job:Finished job EMRCluster ``` ``` INFO:cartography.intel.aws.inspector:Running AWS Inspector cleanup INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.job:Finished job AWSInspectorFinding INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.statement:Completed None statement #None INFO:cartography.graph.job:Finished job AWSInspectorPackage ``` After ``` INFO:cartography.graph.statement:Completed EMRCluster statement #1 INFO:cartography.graph.statement:Completed EMRCluster statement cartography-cncf#2 INFO:cartography.graph.job:Finished job EMRCluster ``` ``` INFO:cartography.intel.aws.inspector:Running AWS Inspector cleanup INFO:cartography.graph.statement:Completed AWSInspectorFinding statement #1 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement cartography-cncf#2 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement cartography-cncf#3 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement cartography-cncf#4 INFO:cartography.graph.statement:Completed AWSInspectorFinding statement cartography-cncf#5 INFO:cartography.graph.job:Finished job AWSInspectorFinding INFO:cartography.graph.statement:Completed AWSInspectorPackage statement #1 INFO:cartography.graph.statement:Completed AWSInspectorPackage statement cartography-cncf#2 INFO:cartography.graph.statement:Completed AWSInspectorPackage statement cartography-cncf#3 INFO:cartography.graph.job:Finished job AWSInspectorPackage ``` Signed-off-by: chandanchowdhury <[email protected]>
chandanchowdhury
pushed a commit
to chandanchowdhury/cartography
that referenced
this pull request
Nov 27, 2024
…cf#1380) **Summary** Mapped in [AWS Identity Center](https://aws.amazon.com/iam/identity-center/) and the access it provides to AWS accounts. New Nodes: (AWSIdentityCenter), (AWSPermissionSet), (AWSSSOUser) New Relationships: (AWSAccount)-[RESOURCE]->(AWSIdentityCenter) (AWSIdentityCenter)-[HAS_PERMISSION_SET]->(AWSPermissionSet) (AWSSSOUser)<-[ALLOWED_BY]-(AWSRole) (OktaUser)<-[CAN_ASSUME_IDENTITY]-(AWSSSOUser) (AWSPermissionSet)-[ASSIGNED_TO_ROLE]->(AWSRole)   **Console Trace** INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-east-1 INFO:cartography.intel.aws.identitycenter:Loading 1 Identity Center instances for region us-east-1 INFO:cartography.intel.aws.identitycenter:Loading 32 permission sets for instance arn:aws:sso:::instance/ssoins-72237a0dcb8c6df7 in region us-east-1 INFO:cartography.intel.aws.identitycenter:Loading 777 permission set role assignments INFO:cartography.intel.aws.identitycenter:Loading 803 SSO users for identity store d-906747a0b9 in region us-east-1 INFO:cartography.intel.aws.identitycenter:Getting role assignments for 803 users INFO:cartography.intel.aws.identitycenter:Loading 24292 role assignments INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-east-2 INFO:cartography.intel.aws.identitycenter:Loading 0 Identity Center instances for region us-east-2 INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-west-1 INFO:cartography.intel.aws.identitycenter:Loading 0 Identity Center instances for region us-west-1 INFO:cartography.intel.aws.identitycenter:Syncing Identity Center instances for region us-west-2 INFO:cartography.intel.aws.identitycenter:Loading 0 Identity Center instances for region us-west-2 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement #1 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement cartography-cncf#2 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement cartography-cncf#3 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement cartography-cncf#4 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement cartography-cncf#5 INFO:cartography.graph.statement:Completed aws_import_identity_center_cleanup statement cartography-cncf#6 **Related issues or links** Fixes - cartography-cncf#990 Checklist Provide proof that this works (this makes reviews move faster). Please perform one or more of the following: [ x ] Update/add unit or integration tests. [ X ] Include a screenshot showing what the graph looked like before and after your changes. [ X ] Include console log trace showing what happened before and after your changes. If you are changing a node or relationship: [ x ] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md). If you are implementing a new intel module: [ X ] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node). --------- Signed-off-by: chandanchowdhury <[email protected]>
chandanchowdhury
pushed a commit
to chandanchowdhury/cartography
that referenced
this pull request
Nov 27, 2024
…ncf#1385) ### Summary This PR adds support to ingest dependencies from Semgrep for the NPM ecosystem, as well as introducing a CLI flag allowing users to specify which ecosystems to ingest. ### Related issues or links cartography-cncf#1368 added support for ingesting dependencies from Semgrep (only for the `gomod` ecosystem) ### Demo Before these changes, a project with both Go and NPM dependencies will only have GoLibrary nodes in the dependency graph: <img width="1036" alt="image" src="https://github.com/user-attachments/assets/31d97626-be70-4c80-9a5b-71c26056a53b"> After these changes, for the same project the graph contains both GoLibrary and NpmLibrary nodes: <img width="1039" alt="image" src="https://github.com/user-attachments/assets/d09cc265-ccd6-463e-bd01-2b3e7c6d1778"> <details> <summary>Logs from semgrep module before these changes</summary> ``` INFO:cartography.sync:Starting sync stage 'semgrep' INFO:cartography.intel.semgrep.deployment:Loading Semgrep deployment info {'id': ...} into the graph... INFO:cartography.intel.semgrep.dependencies:Running Semgrep dependencies sync job. INFO:cartography.intel.semgrep.dependencies:Retrieving Semgrep dependencies for deployment 'X'. INFO:cartography.intel.semgrep.dependencies:Processed page 0 of Semgrep dependencies. ... INFO:cartography.intel.semgrep.dependencies:Processed page X of Semgrep dependencies. INFO:cartography.intel.semgrep.dependencies:Retrieved X Semgrep dependencies in X pages. INFO:cartography.intel.semgrep.dependencies:Loading X GoLibrary objects into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep Go Library cleanup job. INFO:cartography.graph.statement:Completed GoLibrary statement #1 ... INFO:cartography.graph.statement:Completed GoLibrary statement #X INFO:cartography.graph.job:Finished job GoLibrary INFO:cartography.intel.semgrep.findings:Running Semgrep SCA findings sync job. ... INFO:cartography.sync:Finishing sync stage 'semgrep' INFO:cartography.sync:Finishing sync with update tag '1730497895' ``` </details> <details> <summary>Logs from semgrep module after these changes</summary> ``` INFO:cartography.intel.semgrep.deployment:Loading SemgrepDeployment {'id': ...} into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep dependencies sync job. INFO:cartography.intel.semgrep.dependencies:Retrieving Semgrep gomod dependencies for deployment 'X'. INFO:cartography.intel.semgrep.dependencies:Processed page 0 of Semgrep gomod dependencies. INFO:cartography.intel.semgrep.dependencies:Processed page X of Semgrep gomod dependencies. INFO:cartography.intel.semgrep.dependencies:Retrieved X Semgrep gomod dependencies in X pages. INFO:cartography.intel.semgrep.dependencies:Loading X GoLibrary objects into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep Dependencies cleanup job for GoLibrary. INFO:cartography.graph.statement:Completed GoLibrary statement #1 INFO:cartography.graph.statement:Completed GoLibrary statement cartography-cncf#2 INFO:cartography.graph.statement:Completed GoLibrary statement cartography-cncf#3 INFO:cartography.graph.job:Finished job GoLibrary INFO:cartography.intel.semgrep.dependencies:Retrieving Semgrep npm dependencies for deployment 'X'. INFO:cartography.intel.semgrep.dependencies:Processed page 0 of Semgrep npm dependencies. ... INFO:cartography.intel.semgrep.dependencies:Processed page X of Semgrep npm dependencies. INFO:cartography.intel.semgrep.dependencies:Retrieved X Semgrep npm dependencies in X pages. INFO:cartography.intel.semgrep.dependencies:Loading X NpmLibrary objects into the graph. INFO:cartography.intel.semgrep.dependencies:Running Semgrep Dependencies cleanup job for NpmLibrary. INFO:cartography.graph.statement:Completed NpmLibrary statement #1 INFO:cartography.graph.statement:Completed NpmLibrary statement cartography-cncf#2 INFO:cartography.graph.statement:Completed NpmLibrary statement cartography-cncf#3 INFO:cartography.graph.job:Finished job NpmLibrary INFO:cartography.intel.semgrep.findings:Running Semgrep SCA findings sync job. ... INFO:cartography.sync:Finishing sync stage 'semgrep' INFO:cartography.sync:Finishing sync with update tag '1731969699' ``` </details> ### Checklist Provide proof that this works (this makes reviews move faster). Please perform one or more of the following: - [x] Update/add unit or integration tests. - [x] Include a screenshot showing what the graph looked like before and after your changes. - [x] Include console log trace showing what happened before and after your changes. If you are changing a node or relationship: - [x] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md). If you are implementing a new intel module: - [x] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node). --------- Signed-off-by: Hans Wernetti <[email protected]> Signed-off-by: chandanchowdhury <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.