diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 45ccb9ec7d3..ec640106228 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -186,6 +186,7 @@ /packages/ping_one @elastic/security-external-integrations /packages/platform_observability @elastic/infra-monitoring-ui /packages/postgresql @elastic/obs-infraobs-integrations +/packages/prisma_cloud @elastic/security-external-integrations /packages/problemchild @elastic/ml-ui @elastic/sec-applied-ml /packages/prometheus @elastic/obs-cloudnative-monitoring /packages/prometheus_input @elastic/obs-infraobs-integrations diff --git a/packages/prisma_cloud/_dev/build/build.yml b/packages/prisma_cloud/_dev/build/build.yml new file mode 100644 index 00000000000..0757a6edf35 --- /dev/null +++ b/packages/prisma_cloud/_dev/build/build.yml @@ -0,0 +1,4 @@ +dependencies: + ecs: + reference: git@v8.10.0 + import_mappings: true diff --git a/packages/prisma_cloud/_dev/build/docs/README.md b/packages/prisma_cloud/_dev/build/docs/README.md new file mode 100644 index 00000000000..1348dc9e324 --- /dev/null +++ b/packages/prisma_cloud/_dev/build/docs/README.md @@ -0,0 +1,164 @@ +# Prisma Cloud + +This [Prisma Cloud](https://docs.paloaltonetworks.com/prisma/prisma-cloud/prisma-cloud-admin-compute/welcome) is a cloud infrastructure security solution and a Security Operations Center (SOC) enablement tool that enables you to address risks and secure your workloads in a heterogeneous environment (hybrid and multi cloud) from a single console. It provides complete visibility and control over risks within your public cloud infrastructure—Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Oracle Cloud Infrastructure (OCI), Alibaba Cloud— and enables you to manage vulnerabilities, detect anomalies, ensure compliance, and provide runtime defense in heterogeneous environments, such as Windows, Linux, Kubernetes, Red Hat OpenShift, AWS Lambda, Azure Functions, and GCP Cloud Functions. + +## Prisma Cloud Security Posture Management (CSPM) + +Single pane of glass for both CSPM (Cloud Security Posture Management) & CWPP (Cloud Workload Protection Platform). Compute (formerly Twistlock, a CWPP solution) is delivered as part of the larger Prisma Cloud system. Palo Alto Networks runs, manages, and updates Compute Console for you. You deploy and manage Defenders in your environment. You access the Compute Console from a tab within the Prisma Cloud user interface. + +CSPM uses REST API mode to collect data. Elastic Agent fetches data via API endpoints. + +## Prisma Cloud Workload Protection (CWP) + +Self-hosted, stand-alone, self-operated version of Compute (formerly Twistlock). Download the entire software suite, and run it in any environment. You deploy and manage both Console and Defenders. + +CWP can be used in two different modes to collect data: +- REST API mode. +- Syslog mode: This includes TCP and UDP. + +## Compatibility + +This module has been tested against the latest CSPM version **v2** and CWP version **v30.03**. + +## Data streams + +The Prisma Cloud integration collects data for the following five events: + +| Event Type | +|-------------------------------| +| Alert | +| Audit | +| Host | +| Host Profile | +| Incident Audit | + +**NOTE**: + +1. Alert and Audit data-streams are part of [CSPM](https://pan.dev/prisma-cloud/api/cspm/) module, whereas Host, Host Profile and Incident Audit are part of [CWP](https://pan.dev/prisma-cloud/api/cwpp/) module. +2. Currently, we are unable to collect logs of Incident Audit datastream via defined API. Hence, we have not added the configuration of Incident Audit data stream via REST API. + +## Requirements + +- Elastic Agent must be installed. +- You can install only one Elastic Agent per host. +- Elastic Agent is required to stream data through the REST API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines. + +### Installing and managing an Elastic Agent: + +You have a few options for installing and managing an Elastic Agent: + +### Install a Fleet-managed Elastic Agent (recommended): + +With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier. + +### Install Elastic Agent in standalone mode (advanced users): + +With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only. + +### Install Elastic Agent in a containerized environment: + +You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes. + +There are some minimum requirements for running Elastic Agent and for more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html). + +The minimum **kibana.version** required is **8.10.1**. + +## Setup + +### To collect data through REST API, follow the below steps: + +### CSPM + +1. Considering you already have a Prisma Cloud account, to obtain an access key ID and secret access key from the Prisma Cloud system administrator, refer this [link](https://docs.paloaltonetworks.com/prisma/prisma-cloud/prisma-cloud-admin/manage-prisma-cloud-administrators/create-access-keys). +2. The base URL of your CSPM API request depends on the region of your Prisma Cloud tenant and is similar to your Prisma Cloud administrative console URL. Obtain your URL from this [link](https://pan.dev/prisma-cloud/api/cspm/api-urls/). + +### CWP + +1. Assuming you've already generated your access key ID and secret access key from the Prisma Cloud Console; if not, see the section above. +2. The base URL of your CWP API request depends on the console path and the API version of your Prisma Cloud Compute console. +3. To find your API version, log in to your Prisma Cloud Compute console, click the bell icon in the top right of the page, your API version is displayed. +4. To get your console path, navigate to Compute > Manage > System > Downloads. you can find your console path listed under Path to Console. +5. Now you can create your base URL in this format: `https:///api/v`. + +**NOTE**: You can specify a date and time for the access key validity. If you do not select key expiry, the key is set to never expire; if you select it, but do not specify a date, the key expires in a month. + +### Enabling the integration in Elastic: + +1. In Kibana go to Management > Integrations +2. In "Search for integrations" search bar, type Palo Alto Prisma Cloud. +3. Click on the "Palo Alto Prisma Cloud" integration from the search results. +4. Click on the Add Palo Alto Prisma Cloud Integration button to add the integration. +5. While adding the integration, if you want to collect Alert and Audit data via REST API, then you have to put the following details: + - username + - password + - url + - interval + - time amount + - time unit + - batch size + + or if you want to collect Host, Host Profile and Incident Audit data via REST API, then you have to put the following details: + - username + - password + - url + - interval + - offset + - batch size + + or if you want to collect Host, Host Profile and Incident Audit data via TCP/UDP, then you have to put the following details: + - listen address + - listen port + +**NOTE**: Your Access key ID is your username and Secret Access key is your password. + +## Logs Reference + +### Alert + +This is the `Alert` dataset. + +#### Example + +{{event "alert"}} + +{{fields "alert"}} + +### Audit + +This is the `Audit` dataset. + +#### Example + +{{event "audit"}} + +{{fields "audit"}} + +### Host + +This is the `Host` dataset. + +#### Example + +{{event "host"}} + +{{fields "host"}} + +### Host Profile + +This is the `Host Profile` dataset. + +#### Example + +{{event "host_profile"}} + +{{fields "host_profile"}} + +### Incident Audit + +This is the `Incident Audit` dataset. + +#### Example + +{{event "incident_audit"}} + +{{fields "incident_audit"}} \ No newline at end of file diff --git a/packages/prisma_cloud/_dev/deploy/docker/docker-compose.yml b/packages/prisma_cloud/_dev/deploy/docker/docker-compose.yml new file mode 100644 index 00000000000..df0b431ee17 --- /dev/null +++ b/packages/prisma_cloud/_dev/deploy/docker/docker-compose.yml @@ -0,0 +1,45 @@ +version: '2.3' +services: + prisma_cloud-host-tcp: + image: docker.elastic.co/observability/stream:v0.10.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9508 -p=tcp /sample_logs/host.log + prisma_cloud-host-udp: + image: docker.elastic.co/observability/stream:v0.10.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9509 -p=udp /sample_logs/host.log + prisma_cloud-host_profile-tcp: + image: docker.elastic.co/observability/stream:v0.10.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9510 -p=tcp /sample_logs/host_profile.log + prisma_cloud-host_profile-udp: + image: docker.elastic.co/observability/stream:v0.10.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9511 -p=udp /sample_logs/host_profile.log + prisma_cloud-incident_audit-tcp: + image: docker.elastic.co/observability/stream:v0.10.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9512 -p=tcp /sample_logs/incident_audit.log + prisma_cloud-incident_audit-udp: + image: docker.elastic.co/observability/stream:v0.10.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9513 -p=udp /sample_logs/incident_audit.log + prisma_cloud: + image: docker.elastic.co/observability/stream:v0.10.0 + hostname: prisma_cloud + ports: + - 8090 + volumes: + - ./files:/files:ro + environment: + PORT: '8090' + command: + - http-server + - --addr=:8090 + - --config=/files/config.yml diff --git a/packages/prisma_cloud/_dev/deploy/docker/files/config.yml b/packages/prisma_cloud/_dev/deploy/docker/files/config.yml new file mode 100644 index 00000000000..7ef632af910 --- /dev/null +++ b/packages/prisma_cloud/_dev/deploy/docker/files/config.yml @@ -0,0 +1,85 @@ +rules: + - path: /login + methods: ['POST'] + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: | + {"message":"login_successful","token":"xxxx","customerNames":[{"customerName":"Company (Tech Partner Only) - 84706136261xxxxxx32","prismaId":"1121575xxxx8690944","tosAccepted":true}]} + - path: /v2/alert + methods: ['GET'] + request_headers: + x-redlock-auth: + - 'xxxx' + responses: + - status_code: 200 + body: | + {"totalRows":1,"items":[{"id":"N-3910","alertAdditionalInfo":{"scannerVersion":"CS_2.0"},"alertAttribution":{"attributionEventList":[{"event":"first_event","event_ts":1694003441966,"username":"alex123"}],"resourceCreatedBy":"string","resourceCreatedOn":0},"status":"open","reason":"NEW_ALERT","firstSeen":1694003441966,"history":[{"modifiedOn":"1694003441966","modifiedBy":"alex123","reason":"Reason1","status":"OPEN"}],"lastSeen":1694003441966,"alertTime":1694003441966,"lastUpdated":1694003441966,"policyId":"ad23603d-754e-4499-8988-b801xxx85898","metadata":null,"policy":{"policyId":"ad23603d-754e-4499-8988-b8017xxxx98","name":"AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0\/0)","policyType":"network","systemDefault":true,"complianceMetadata":[{"complianceId":"qwer345bv","customAssigned":true,"policyId":"werf435tr","requirementDescription":"Description of policy compliance.","requirementId":"req-123-xyz","requirementName":"rigidity","sectionDescription":"Description of section.","sectionId":"sect-453-abc","sectionLabel":"label-1","standardDescription":"Description of standard.","standardId":"stand-543-pqr","standardName":"Class 1"}],"description":"This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0\/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.","severity":"high","recommendation":"The following steps are recommended to restrict unrestricted access from the Internet:\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\n2. Identify the network component on which restrictive rules can be implemented.\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\n a) The overly permissive Security Group rules can be made more restrictive.\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.","labels":["Prisma_Cloud","Attack Path Rule"],"lastModifiedOn":1687474999057,"lastModifiedBy":"template@redlock.io","deleted":false,"findingTypes":[],"remediable":false,"remediation":{"actions":[{"operation":"buy","payload":"erefwsdf"}],"cliScriptTemplate":"temp1","description":"Description of CLI Script Template."}},"alertRules":[],"resource":{"rrn":"rrn:aws:instance:us-east-1:710000059376:e7ddce5a1ffcb47bxxxxxerf2635a3b4d9da3:i-04578e0008100947","id":"i-04578exxxx8100947","name":"IS-37133","account":"AWS Cloud Account","accountId":"710002259376","cloudAccountGroups":["Default Account Group"],"region":"AWS Virginia","regionId":"us-east-1","resourceType":"INSTANCE","resourceApiName":"aws-ec2-describe-instances","cloudServiceName":"Amazon EC2","url":"https:\/\/console.aws.amazon.com\/ec2\/v2\/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947","data":null,"additionalInfo":null,"cloudType":"aws","resourceTs":1694003441915,"unifiedAssetId":"66c543b6261c4d9edxxxxxb42e15f4","resourceConfigJsonAvailable":false,"resourceDetailsAvailable":true},"investigateOptions":{"alertId":"N-3910"}}]} + - path: /audit/redlock + methods: ['GET'] + request_headers: + x-redlock-auth: + - 'xxxx' + responses: + - status_code: 200 + body: | + [{"timestamp":1694594439068,"user":"john.user@google.com","ipAddress":"81.2.69.192","actionType":"LOGIN","resourceName":"john.user@google.com","action":"'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key.","resourceType":"Login","result":"Successful"}] + - path: /authenticate + methods: ['POST'] + responses: + - status_code: 200 + headers: + Content-Type: + - 'application/json' + body: | + {"token":"xxxx"} + - path: /hosts + methods: ['GET'] + request_headers: + Authorization: + - 'Bearer xxxx' + query_params: + offset: 0 + limit: 50 + responses: + - status_code: 200 + body: | + [{"_id":"DESKTOP-6PQXXMS","binaries":[{"altered":true,"cveCount":0,"deps":["string"],"fileMode":0,"functionLayer":"string","md5":"string","missingPkg":true,"name":"string","path":"string","pkgRootDir":"string","services":["string"],"version":"string"}],"cloudMetadata":{"accountID":"Non-onboarded cloud accounts","awsExecutionEnv":"string","image":"string","labels":[{"key":"string","sourceName":"string","sourceType":["namespace"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"name":"string","provider":["aws"],"region":"string","resourceID":"string","resourceURL":"string","type":"string","vmID":"string","vmImageID":"string"},"type":"host","hostname":"DESKTOP-6PQXXMS","scanTime":"2023-08-23T11:48:41.803Z","Secrets":[],"osDistro":"windows","osDistroVersion":"string","osDistroRelease":"Windows","distro":"Microsoft Windows [Version 10.0.19045.2006]","packageManager":true,"packages":[{"pkgs":[{"binaryIdx":[0],"binaryPkgs":["string"],"cveCount":0,"defaultGem":true,"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"functionLayer":"string","goPkg":true,"jarIdentifier":"string","layerTime":0,"license":"string","name":"string","osPackage":true,"path":"string","version":"string"}],"pkgsType":"nodejs"}],"isARM64":false,"packageCorrelationDone":true,"redHatNonRPMImage":false,"image":{"created":"0001-01-01T00:00:00Z","entrypoint":["string"],"env":["string"],"healthcheck":true,"id":"string","labels":{},"layers":["string"],"os":"string","repoDigest":["string"],"repoTags":["string"],"user":"string","workingDir":"string"},"allCompliance":{"compliance":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.949Z","exploit":["exploit-db"],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI"]],"text":"string","title":"string","twistlock":true,"type":["container"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}],"enabled":"true"},"clusters":["string"],"repoTag":null,"tags":[{"digest":"string","id":"string","registry":"string","repo":"string","tag":"string"}],"trustResult":{"hostsStatuses":[{"host":"string","status":"trusted"}]},"repoDigests":[],"creationTime":"0001-01-01T00:00:00Z","pushTime":"0001-01-01T00:00:00Z","vulnerabilitiesCount":0,"complianceIssuesCount":4,"vulnerabilityDistribution":{"critical":0,"high":0,"medium":0,"low":0,"total":0},"complianceDistribution":{"critical":4,"high":0,"medium":0,"low":0,"total":4},"vulnerabilityRiskScore":0,"complianceRiskScore":4000000,"riskFactors":{},"firstScanTime":"2023-08-11T06:53:57.456Z","history":[{"baseLayer":true,"created":0,"emptyLayer":true,"id":"string","instruction":"string","sizeBytes":0,"tags":["string"],"vulnerabilities":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.950Z","exploit":["exploit-db"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI"]],"text":"string","title":"string","twistlock":true,"type":["container"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}]}],"hostDevices":[{"ip":"0.0.0.0","name":"string"}],"hosts":{},"id":"string","err":"","collections":["All"],"instances":[{"host":"string","image":"string","modified":"2023-09-08T04:01:49.951Z","registry":"string","repo":"string","tag":"string"}],"scanID":0,"trustStatus":"","externalLabels":[{"key":"string","sourceName":"string","sourceType":["namespace"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"firewallProtection":{"enabled":false,"supported":false,"outOfBandMode":"Observation","ports":[0],"tlsPorts":[0],"unprotectedProcesses":[{"port":0,"process":"string","tls":true}]},"applications":[{"installedFromPackage":true,"knownVulnerabilities":0,"layerTime":0,"name":"string","path":"string","service":true,"version":"string"}],"appEmbedded":false,"wildFireUsage":null,"agentless":false,"malwareAnalyzedTime":"0001-01-01T00:00:00Z"}] + - path: /hosts + methods: ['GET'] + request_headers: + Authorization: + - 'Bearer xxxx' + query_params: + offset: 1 + limit: 50 + responses: + - status_code: 200 + body: | + null + - path: /profiles/host + methods: ['GET'] + request_headers: + Authorization: + - 'Bearer xxxx' + query_params: + offset: 0 + limit: 50 + responses: + - status_code: 200 + body: | + [{"_id":"DESKTOP-6PQXXMS","hash":1,"created":"2023-08-11T06:53:48.855Z","time":"0001-01-01T00:00:00Z","collections":["All"]}] + - path: /profiles/host + methods: ['GET'] + request_headers: + Authorization: + - 'Bearer xxxx' + query_params: + offset: 1 + limit: 50 + responses: + - status_code: 200 + body: | + [] diff --git a/packages/prisma_cloud/_dev/deploy/docker/sample_logs/host.log b/packages/prisma_cloud/_dev/deploy/docker/sample_logs/host.log new file mode 100644 index 00000000000..002ae3e9698 --- /dev/null +++ b/packages/prisma_cloud/_dev/deploy/docker/sample_logs/host.log @@ -0,0 +1 @@ +{"_id":"DESKTOP-6PQXXMS","binaries":[{"altered":true,"cveCount":0,"deps":["string"],"fileMode":0,"functionLayer":"string","md5":"string","missingPkg":true,"name":"string","path":"string","pkgRootDir":"string","services":["string"],"version":"string"}],"cloudMetadata":{"accountID":"Non-onboarded cloud accounts","awsExecutionEnv":"string","image":"string","labels":[{"key":"string","sourceName":"string","sourceType":["namespace"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"name":"string","provider":["aws"],"region":"string","resourceID":"string","resourceURL":"string","type":"string","vmID":"string","vmImageID":"string"},"type":"host","hostname":"DESKTOP-6PQXXMS","scanTime":"2023-08-23T11:48:41.803Z","Secrets":[],"osDistro":"windows","osDistroVersion":"string","osDistroRelease":"Windows","distro":"Microsoft Windows [Version 10.0.19045.2006]","packageManager":true,"packages":[{"pkgs":[{"binaryIdx":[0],"binaryPkgs":["string"],"cveCount":0,"defaultGem":true,"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"functionLayer":"string","goPkg":true,"jarIdentifier":"string","layerTime":0,"license":"string","name":"string","osPackage":true,"path":"string","version":"string"}],"pkgsType":"nodejs"}],"isARM64":false,"packageCorrelationDone":true,"redHatNonRPMImage":false,"image":{"created":"0001-01-01T00:00:00Z","entrypoint":["string"],"env":["string"],"healthcheck":true,"id":"string","labels":{},"layers":["string"],"os":"string","repoDigest":["string"],"repoTags":["string"],"user":"string","workingDir":"string"},"allCompliance":{"compliance":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.949Z","exploit":["exploit-db"],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI"]],"text":"string","title":"string","twistlock":true,"type":["container"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}],"enabled":"true"},"clusters":["string"],"repoTag":null,"tags":[{"digest":"string","id":"string","registry":"string","repo":"string","tag":"string"}],"trustResult":{"hostsStatuses":[{"host":"string","status":"trusted"}]},"repoDigests":[],"creationTime":"0001-01-01T00:00:00Z","pushTime":"0001-01-01T00:00:00Z","vulnerabilitiesCount":0,"complianceIssuesCount":4,"vulnerabilityDistribution":{"critical":0,"high":0,"medium":0,"low":0,"total":0},"complianceDistribution":{"critical":4,"high":0,"medium":0,"low":0,"total":4},"vulnerabilityRiskScore":0,"complianceRiskScore":4000000,"riskFactors":{},"firstScanTime":"2023-08-11T06:53:57.456Z","history":[{"baseLayer":true,"created":0,"emptyLayer":true,"id":"string","instruction":"string","sizeBytes":0,"tags":["string"],"vulnerabilities":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.950Z","exploit":["exploit-db"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI"]],"text":"string","title":"string","twistlock":true,"type":["container"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}]}],"hostDevices":[{"ip":"0.0.0.0","name":"string"}],"hosts":{},"id":"string","err":"","collections":["All"],"instances":[{"host":"string","image":"string","modified":"2023-09-08T04:01:49.951Z","registry":"string","repo":"string","tag":"string"}],"scanID":0,"trustStatus":"","externalLabels":[{"key":"string","sourceName":"string","sourceType":["namespace"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"firewallProtection":{"enabled":false,"supported":false,"outOfBandMode":"Observation","ports":[0],"tlsPorts":[0],"unprotectedProcesses":[{"port":0,"process":"string","tls":true}]},"applications":[{"installedFromPackage":true,"knownVulnerabilities":0,"layerTime":0,"name":"string","path":"string","service":true,"version":"string"}],"appEmbedded":false,"wildFireUsage":null,"agentless":false,"malwareAnalyzedTime":"0001-01-01T00:00:00Z"} diff --git a/packages/prisma_cloud/_dev/deploy/docker/sample_logs/host_profile.log b/packages/prisma_cloud/_dev/deploy/docker/sample_logs/host_profile.log new file mode 100644 index 00000000000..43c981f1894 --- /dev/null +++ b/packages/prisma_cloud/_dev/deploy/docker/sample_logs/host_profile.log @@ -0,0 +1 @@ +{"_id":"DESKTOP-6PXXAMS","hash":1,"created":"2023-08-11T06:53:48.855Z","time":"0001-01-01T00:00:00Z","collections":["All"]} \ No newline at end of file diff --git a/packages/prisma_cloud/_dev/deploy/docker/sample_logs/incident_audit.log b/packages/prisma_cloud/_dev/deploy/docker/sample_logs/incident_audit.log new file mode 100644 index 00000000000..3a7a5b5caea --- /dev/null +++ b/packages/prisma_cloud/_dev/deploy/docker/sample_logs/incident_audit.log @@ -0,0 +1 @@ +{"_id":"thgry1736","accountID":"accounttest","acknowledged":true,"app":"test","appID":"test123","audits":[{"_id":"id1234","accountID":"accounttest","app":"test","appID":"test123","attackTechniques":["exploitationForPrivilegeEscalation"],"attackType":["cloudMetadataProbing"],"cluster":"clustertest","collections":["collectiontest"],"command":"commandtest","container":true,"containerId":"testcontainerid","containerName":"testcontainername","count":0,"country":"in","domain":"testdomain","effect":"block","err":"testerr","filepath":"testfilepath","fqdn":"testfqdn","function":"testfun","functionID":"testfunid","hostname":"testhostname","imageId":"testimgid","imageName":"testimgname","interactive":true,"ip":"81.2.69.142","label":"testlabel","labels":{"sjhia": "ifo"},"md5":"testmd5","msg":"testmsg","namespace":"testnamespace","os":"testos","pid":0,"port":0,"processPath":"testprocesspath","profileId":"testprofileid","provider":"aws","rawEvent":"testrawevent","region":"testregion","requestID":"testrequestid","resourceID":"testresourceid","ruleName":"testrulename","runtime":"python","severity":"low","time":"2023-08-30T08:42:17.834Z","type":"processes","user":"testuser","version":"testversion","vmID":"testvmid","wildFireReportURL":"testwildfirereporturl"}],"category":"portScanning","cluster":"testcluster","collections":["testcollections"],"containerID":"testcontainer","containerName":"testcontainername","customRuleName":"testcustomrulename","fqdn":"testfqdn","function":"testfunction","functionID":"testfunctionid","hostname":"testhostname","imageID":"testimgid","imageName":"testimgname","labels":{"aaa":"bbb"},"namespace":"testnamespace","profileID":"testprofileid","provider":"aws","region":"testregion","resourceID":"testresourceid","runtime":"testruntime","serialNum":0,"shouldCollect":true,"time":"2023-08-30T08:42:17.834Z","type":"host","vmID":"testvmid","windows":true} \ No newline at end of file diff --git a/packages/prisma_cloud/changelog.yml b/packages/prisma_cloud/changelog.yml new file mode 100644 index 00000000000..fd15de59934 --- /dev/null +++ b/packages/prisma_cloud/changelog.yml @@ -0,0 +1,6 @@ +# newer versions go on top +- version: "0.1.0" + changes: + - description: Initial release. + type: enhancement + link: https://github.com/elastic/integrations/pull/8135 diff --git a/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-alert.log b/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-alert.log new file mode 100644 index 00000000000..10a571b11d2 --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-alert.log @@ -0,0 +1,4 @@ +{"id":"N-3910","alertAdditionalInfo":{"scannerVersion":"CS_2.0"},"alertAttribution":{"attributionEventList":[{"event":"first_event","event_ts":1694003441966,"username":"alex123"}],"resourceCreatedBy":"alex123","resourceCreatedOn":1694003441966},"status":"open","reason":"NEW_ALERT","firstSeen":1694003441966,"history":[{"modifiedOn":"1694003441966","modifiedBy":"alex123","reason":"Reason1","status":"OPEN"}],"lastSeen":1694003441966,"alertTime":1694003441966,"lastUpdated":1694003441966,"policyId":"ad23603d-754e-4499-8988-b80xxxxx898","metadata":{"saveSearchId":"8ec3f878-0f5e-4782-b4cd-98xxxxx17be5"},"policy":{"policyId":"ad23603d-754e-4499-8988-b8xxx785898","name":"AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0\/0)","policyType":"network","systemDefault":true,"complianceMetadata":[{"complianceId":"qwer345bv","customAssigned":true,"policyId":"werf435tr","requirementDescription":"Description of policy compliance.","requirementId":"req-123-xyz","requirementName":"rigidity","sectionDescription":"Description of section.","sectionId":"sect-453-abc","sectionLabel":"label-1","standardDescription":"Description of standard.","standardId":"stand-543-pqr","standardName":"Class 1"}],"description":"This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0\/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.","severity":"high","recommendation":"The following steps are recommended to restrict unrestricted access from the Internet:\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\n2. Identify the network component on which restrictive rules can be implemented.\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\n a) The overly permissive Security Group rules can be made more restrictive.\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.","labels":["Prisma_Cloud","Attack Path Rule"],"lastModifiedOn":1687474999057,"lastModifiedBy":"template@redlock.io","deleted":false,"findingTypes":[],"remediable":false,"remediation":{"actions":[{"operation":"buy","payload":"erefwsdf"}],"cliScriptTemplate":"temp1","description":"Description of CLI Script Template."}},"alertRules":[],"resource":{"additionalInfo":{"accessKeyAge":"1691053278000","abc":"hello","how":"are you","inactiveSinceTs":"N\/A"},"rrn":"rrn:aws:instance:us-xxxx-1:711162259376:e7ddce5a1ffcb47b59bfd898c622635a3b4d9da3:i-04578e97178100947","id":"i-04578e97170000047","name":"IS-37133","account":"AWS Cloud Account","accountId":"7111xx259376","cloudAccountGroups":["Default Account Group"],"region":"AWS Virginia","regionId":"us-xxxx-1","resourceType":"INSTANCE","resourceApiName":"aws-ec2-xxxx-instances","cloudServiceName":"Amazon EC2","url":"https:\/\/console.aws.amazon.com\/ec2\/v2\/home?region=us-xxxx-1#Instances:instanceId=i-04578e9xxxxx00947","data":null,"cloudType":"aws","resourceTs":1694044541915,"unifiedAssetId":"66c543b6261xxxxx23c8eab42e15f4","resourceConfigJsonAvailable":false,"resourceDetailsAvailable":true},"investigateOptions":{"alertId":"N-3910"}} +{"riskDetail":{"policyScores":[{"cloudType":"ALL","complianceMetadata":[{"complianceId":"string","customAssigned":true,"policyId":"string","requirementDescription":"string","requirementId":"string","requirementName":"string","sectionDescription":"string","sectionId":"string","sectionLabel":"string","standardDescription":"string","standardId":"string","standardName":"string"}],"createdBy":"string","createdOn":0,"deleted":true,"description":"string","enabled":true,"findingTypes":["string"],"labels":["string"],"lastModifiedBy":"string","lastModifiedOn":0,"name":"string","overridden":true,"points":"string","policyId":"string","policySubTypes":["run"],"policyType":"config","policyUpi":"string","recommendation":"string","remediable":true,"remediation":{"actions":[{"operation":"string","payload":"string"}],"cliScriptTemplate":"string","description":"string"},"restrictAlertDismissal":true,"riskScore":{"maxScore":0,"score":0},"rule":{"apiName":"string","cloudAccount":"string","cloudType":"string","criteria":"string","dataCriteria":{"classificationResult":"string","exposure":"private","extension":["string"]},"name":"string","parameters":{},"resourceIdPath":"string","resourceType":"string","type":"Config"},"ruleLastModifiedOn":0,"severity":"high","systemDefault":true}],"rating":"string","riskScore":{"maxScore":0,"score":0},"score":"string"}} +{"resource":{"rrn":"rrn::storageBucket:eu-north-1:711162259376:c7f9a5b64e82b00567aa854845xxxxxba575fce:is-34375","id":"is-3xx75","name":"is-3x375","account":"AWS Cloud Account","accountId":"7111xxx59376","cloudAccountGroups":["Default Account Group"],"region":"AWS Stockholm","regionId":"eu-north-1","resourceType":"STORAGE_BUCKET","resourceApiName":"aws-s3api-get-bucket-acl","cloudServiceName":"Amazon S3","url":"https:\/\/console.aws.amazon.com\/s3\/buckets\/is-3xx75\/?region=eu-xxxx-1#","data":{"acl":{"owner":{"id":"1ec213dd8ce39db5ea8daxxx552449effacee6337758d64dff989f85018307b1"},"grants":[{"grantee":{"identifier":"1ec213dd8ce39db5ea8dxxxx2449effacee6337758d64dff989f85018307b1","typeIdentifier":"id"},"permission":"FullControl"}],"grantsAsList":[{"grantee":{"identifier":"1ec213dd8ce39db5ea8da3d6552449effacee6337758d64dff989f85xxx307b1","typeIdentifier":"id"},"permission":"FullControl"}],"requesterCharged":false},"owner":{"id":"1ec213dd8ce39db5exxx3d6552449effacee6337758d64dff989f85018307b1","displayName":"crest.it"},"policy":{},"tagSets":{"aws:cloudformation:stack-id":"arn:aws:cloudformation:ap-xxxxx-1:711162259376:stack\/fens-cloud-account-onboarding\/58400380-27b8-11ee-9c3e-064665xxx276","aws:cloudformation:logical-id":"FensCloudTrailS3Bucket","aws:cloudformation:stack-name":"fens-cloud-account-onboarding"},"accountId":"71116xxx9376","bucketName":"is-3x375","creationDate":"2023-06-08T12:05:18.000Z","sseAlgorithm":"AX256","policyAvailable":true,"ownershipControls":{"rules":[{"ownership":"BucketOwnerEnforced"}]},"loggingConfiguration":{"targetGrants":[{"grantee":{"identifier":"1ec213dd8ce39db5ea8dxxxx552449effacee6337758d64dff989f85018307b1","typeIdentifier":"id"},"permission":"FullControl"}]},"versioningConfiguration":{"status":"Off"},"denyUnencryptedUploadsPolicies":[],"publicAccessBlockConfiguration":{"blockPublicAcls":true,"ignorePublicAcls":true,"blockPublicPolicy":true,"restrictPublicBuckets":true},"accountLevelPublicAccessBlockConfiguration":{"blockPublicAcls":false,"ignorePublicAcls":false,"blockPublicPolicy":false,"restrictPublicBuckets":false}},"additionalInfo":{},"cloudType":"aws","resourceTs":1690452507282,"unifiedAssetId":"fd032029a5a32c6bf8xxx1f98c87a37f","resourceConfigJsonAvailable":true,"resourceDetailsAvailable":true}} +{"resource":{"rrn":"rrn::securityGroup:us-east-1:711162259376:437dc48e29e24dbca78xxxx1118264783cefdadd:sg-01e9ec2a09ac00f4f","id":"sg-01e9ec2a09axxxf4f","name":"default","account":"AWS Cloud Account","accountId":"711xxx259376","cloudAccountGroups":["Default Account Group"],"region":"AWS Virginia","regionId":"us-xxxx-1","resourceType":"SECURITY_GROUP","resourceApiName":"aws-ec2-describe-security-groups","cloudServiceName":"Amazon VPC","url":"https:\/\/console.aws.amazon.com\/vpc\/home?region=us-xxxx-1#securityGroups:filter=sg-01e9exxx09ac00f4f","data":{"description":"default VPC security group","groupId":"sg-01e9ec2axxxc00f4f","groupName":"default","ipPermissions":[{"ipRanges":[],"prefixListIds":[],"userIdGroupPairs":[{"groupId":"sg-01e9ecxxx9ac00f4f","userId":"71116xxx9376"}],"ipProtocol":"-1","ipv4Ranges":[],"ipv6Ranges":[]}],"ipPermissionsEgress":[{"ipRanges":["0.0.0.0\/0"],"prefixListIds":[],"userIdGroupPairs":[],"ipProtocol":"-1","ipv4Ranges":[{"cidrIp":"0.0.0.0\/0"}],"ipv6Ranges":[]}],"isShared":false,"ownerId":"711xxx259376","region":"us-east-1","tags":[],"vpcId":"vpc-00f0ef31xxx957e98"},"additionalInfo":{},"cloudType":"aws","resourceTs":1695443838603,"unifiedAssetId":"54535f5c777e41xxxx97291cab6097e7","resourceConfigJsonAvailable":true,"resourceDetailsAvailable":true}} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json b/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json new file mode 100644 index 00000000000..9bb1f7dbc4b --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-alert.log-expected.json @@ -0,0 +1,546 @@ +{ + "expected": [ + { + "@timestamp": "2023-09-06T12:30:41.966Z", + "cloud": { + "account": { + "id": "7111xx259376" + }, + "provider": "aws", + "service": { + "name": "Amazon EC2" + } + }, + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "threat" + ], + "end": "2023-09-06T12:30:41.966Z", + "id": "N-3910", + "kind": "alert", + "original": "{\"id\":\"N-3910\",\"alertAdditionalInfo\":{\"scannerVersion\":\"CS_2.0\"},\"alertAttribution\":{\"attributionEventList\":[{\"event\":\"first_event\",\"event_ts\":1694003441966,\"username\":\"alex123\"}],\"resourceCreatedBy\":\"alex123\",\"resourceCreatedOn\":1694003441966},\"status\":\"open\",\"reason\":\"NEW_ALERT\",\"firstSeen\":1694003441966,\"history\":[{\"modifiedOn\":\"1694003441966\",\"modifiedBy\":\"alex123\",\"reason\":\"Reason1\",\"status\":\"OPEN\"}],\"lastSeen\":1694003441966,\"alertTime\":1694003441966,\"lastUpdated\":1694003441966,\"policyId\":\"ad23603d-754e-4499-8988-b80xxxxx898\",\"metadata\":{\"saveSearchId\":\"8ec3f878-0f5e-4782-b4cd-98xxxxx17be5\"},\"policy\":{\"policyId\":\"ad23603d-754e-4499-8988-b8xxx785898\",\"name\":\"AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0\\/0)\",\"policyType\":\"network\",\"systemDefault\":true,\"complianceMetadata\":[{\"complianceId\":\"qwer345bv\",\"customAssigned\":true,\"policyId\":\"werf435tr\",\"requirementDescription\":\"Description of policy compliance.\",\"requirementId\":\"req-123-xyz\",\"requirementName\":\"rigidity\",\"sectionDescription\":\"Description of section.\",\"sectionId\":\"sect-453-abc\",\"sectionLabel\":\"label-1\",\"standardDescription\":\"Description of standard.\",\"standardId\":\"stand-543-pqr\",\"standardName\":\"Class 1\"}],\"description\":\"This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0\\/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.\",\"severity\":\"high\",\"recommendation\":\"The following steps are recommended to restrict unrestricted access from the Internet:\\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\\n2. Identify the network component on which restrictive rules can be implemented.\\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\\n a) The overly permissive Security Group rules can be made more restrictive.\\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.\",\"labels\":[\"Prisma_Cloud\",\"Attack Path Rule\"],\"lastModifiedOn\":1687474999057,\"lastModifiedBy\":\"template@redlock.io\",\"deleted\":false,\"findingTypes\":[],\"remediable\":false,\"remediation\":{\"actions\":[{\"operation\":\"buy\",\"payload\":\"erefwsdf\"}],\"cliScriptTemplate\":\"temp1\",\"description\":\"Description of CLI Script Template.\"}},\"alertRules\":[],\"resource\":{\"additionalInfo\":{\"accessKeyAge\":\"1691053278000\",\"abc\":\"hello\",\"how\":\"are you\",\"inactiveSinceTs\":\"N\\/A\"},\"rrn\":\"rrn:aws:instance:us-xxxx-1:711162259376:e7ddce5a1ffcb47b59bfd898c622635a3b4d9da3:i-04578e97178100947\",\"id\":\"i-04578e97170000047\",\"name\":\"IS-37133\",\"account\":\"AWS Cloud Account\",\"accountId\":\"7111xx259376\",\"cloudAccountGroups\":[\"Default Account Group\"],\"region\":\"AWS Virginia\",\"regionId\":\"us-xxxx-1\",\"resourceType\":\"INSTANCE\",\"resourceApiName\":\"aws-ec2-xxxx-instances\",\"cloudServiceName\":\"Amazon EC2\",\"url\":\"https:\\/\\/console.aws.amazon.com\\/ec2\\/v2\\/home?region=us-xxxx-1#Instances:instanceId=i-04578e9xxxxx00947\",\"data\":null,\"cloudType\":\"aws\",\"resourceTs\":1694044541915,\"unifiedAssetId\":\"66c543b6261xxxxx23c8eab42e15f4\",\"resourceConfigJsonAvailable\":false,\"resourceDetailsAvailable\":true},\"investigateOptions\":{\"alertId\":\"N-3910\"}}", + "start": "2023-09-06T12:30:41.966Z", + "type": [ + "indicator" + ] + }, + "prisma_cloud": { + "alert": { + "additional_info": { + "scanner_version": "CS_2.0" + }, + "attribution": { + "event_list": [ + { + "ts": "2023-09-06T12:30:41.966Z", + "username": "alex123", + "value": "first_event" + } + ], + "resource": { + "created_by": "alex123", + "created_on": "2023-09-06T12:30:41.966Z" + } + }, + "first_seen": "2023-09-06T12:30:41.966Z", + "history": [ + { + "modified_by": "alex123", + "modified_on": "2023-09-06T12:30:41.966Z", + "reason": "Reason1", + "status": "OPEN" + } + ], + "id": "N-3910", + "last": { + "seen": "2023-09-06T12:30:41.966Z", + "updated": "2023-09-06T12:30:41.966Z" + }, + "metadata": { + "save_search_id": "8ec3f878-0f5e-4782-b4cd-98xxxxx17be5" + }, + "policy": { + "compliance_metadata": [ + { + "compliance_id": "qwer345bv", + "custom_assigned": true, + "policy_id": "werf435tr", + "requirement": { + "description": "Description of policy compliance.", + "id": "req-123-xyz", + "name": "rigidity" + }, + "section": { + "description": "Description of section.", + "id": "sect-453-abc", + "label": "label-1" + }, + "standard": { + "description": "Description of standard.", + "id": "stand-543-pqr", + "name": "Class 1" + } + } + ], + "deleted": false, + "description": "This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.", + "id": "ad23603d-754e-4499-8988-b8xxx785898", + "labels": [ + "Prisma_Cloud", + "Attack Path Rule" + ], + "last_modified_by": "template@redlock.io", + "last_modified_on": "2023-06-22T23:03:19.057Z", + "name": "AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0/0)", + "recommendation": "The following steps are recommended to restrict unrestricted access from the Internet:\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\n2. Identify the network component on which restrictive rules can be implemented.\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\n a) The overly permissive Security Group rules can be made more restrictive.\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.", + "remediable": false, + "remediation": { + "actions": [ + { + "operation": "buy", + "payload": "erefwsdf" + } + ], + "cli_script_template": "temp1", + "description": "Description of CLI Script Template." + }, + "severity": "high", + "system_default": true, + "type": "network" + }, + "policy_id": "ad23603d-754e-4499-8988-b80xxxxx898", + "reason": "NEW_ALERT", + "resource": { + "account": { + "id": "7111xx259376", + "value": "AWS Cloud Account" + }, + "additional_info": { + "abc": "hello", + "accessKeyAge": "1691053278000", + "how": "are you", + "inactiveSinceTs": "N/A" + }, + "api_name": "aws-ec2-xxxx-instances", + "cloud": { + "account": { + "groups": [ + "Default Account Group" + ] + }, + "service_name": "Amazon EC2", + "type": "aws" + }, + "config_json_available": false, + "details_available": true, + "id": "i-04578e97170000047", + "name": "IS-37133", + "region": { + "id": "us-xxxx-1", + "value": "AWS Virginia" + }, + "rrn": "rrn:aws:instance:us-xxxx-1:711162259376:e7ddce5a1ffcb47b59bfd898c622635a3b4d9da3:i-04578e97178100947", + "ts": "2023-09-06T23:55:41.915Z", + "type": "INSTANCE", + "unified_asset_id": "66c543b6261xxxxx23c8eab42e15f4", + "url": "https://console.aws.amazon.com/ec2/v2/home?region=us-xxxx-1#Instances:instanceId=i-04578e9xxxxx00947" + }, + "status": "open", + "time": "2023-09-06T12:30:41.966Z" + } + }, + "related": { + "user": [ + "alex123" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "url": { + "domain": "console.aws.amazon.com", + "fragment": "Instances:instanceId=i-04578e9xxxxx00947", + "original": "https://console.aws.amazon.com/ec2/v2/home?region=us-xxxx-1#Instances:instanceId=i-04578e9xxxxx00947", + "path": "/ec2/v2/home", + "query": "region=us-xxxx-1", + "scheme": "https" + } + }, + { + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "threat" + ], + "kind": "alert", + "original": "{\"riskDetail\":{\"policyScores\":[{\"cloudType\":\"ALL\",\"complianceMetadata\":[{\"complianceId\":\"string\",\"customAssigned\":true,\"policyId\":\"string\",\"requirementDescription\":\"string\",\"requirementId\":\"string\",\"requirementName\":\"string\",\"sectionDescription\":\"string\",\"sectionId\":\"string\",\"sectionLabel\":\"string\",\"standardDescription\":\"string\",\"standardId\":\"string\",\"standardName\":\"string\"}],\"createdBy\":\"string\",\"createdOn\":0,\"deleted\":true,\"description\":\"string\",\"enabled\":true,\"findingTypes\":[\"string\"],\"labels\":[\"string\"],\"lastModifiedBy\":\"string\",\"lastModifiedOn\":0,\"name\":\"string\",\"overridden\":true,\"points\":\"string\",\"policyId\":\"string\",\"policySubTypes\":[\"run\"],\"policyType\":\"config\",\"policyUpi\":\"string\",\"recommendation\":\"string\",\"remediable\":true,\"remediation\":{\"actions\":[{\"operation\":\"string\",\"payload\":\"string\"}],\"cliScriptTemplate\":\"string\",\"description\":\"string\"},\"restrictAlertDismissal\":true,\"riskScore\":{\"maxScore\":0,\"score\":0},\"rule\":{\"apiName\":\"string\",\"cloudAccount\":\"string\",\"cloudType\":\"string\",\"criteria\":\"string\",\"dataCriteria\":{\"classificationResult\":\"string\",\"exposure\":\"private\",\"extension\":[\"string\"]},\"name\":\"string\",\"parameters\":{},\"resourceIdPath\":\"string\",\"resourceType\":\"string\",\"type\":\"Config\"},\"ruleLastModifiedOn\":0,\"severity\":\"high\",\"systemDefault\":true}],\"rating\":\"string\",\"riskScore\":{\"maxScore\":0,\"score\":0},\"score\":\"string\"}}", + "type": [ + "indicator" + ] + }, + "prisma_cloud": { + "alert": { + "risk_detail": { + "policy_scores": [ + { + "cloud_type": "ALL", + "compliance_metadata": [ + { + "compliance_id": "string", + "custom_assigned": true, + "policy": { + "id": "string" + }, + "requirement": { + "description": "string", + "id": "string", + "name": "string" + }, + "section": { + "description": "string", + "id": "string", + "label": "string" + }, + "standard": { + "description": "string", + "id": "string", + "name": "string" + } + } + ], + "created": { + "by": "string", + "on": "1970-01-01T00:00:00.000Z" + }, + "deleted": true, + "description": "string", + "enabled": true, + "finding_types": [ + "string" + ], + "labels": [ + "string" + ], + "last_modified": { + "by": "string", + "on": "1970-01-01T00:00:00.000Z" + }, + "name": "string", + "overridden": true, + "points": "string", + "policy": { + "id": "string", + "subtypes": [ + "run" + ], + "type": "config", + "upi": "string" + }, + "recommendation": "string", + "remediable": true, + "remediation": { + "actions": [ + { + "operation": "string", + "payload": "string" + } + ], + "cli_script_template": "string", + "description": "string" + }, + "restrict_alert_dismissal": true, + "risk_score": { + "max": 0, + "value": 0 + }, + "rule": { + "api_name": "string", + "cloud": { + "account": "string", + "type": "string" + }, + "criteria": "string", + "data_criteria": { + "classification_result": "string", + "exposure": "private", + "extension": [ + "string" + ] + }, + "last_modified_on": "1970-01-01T00:00:00.000Z", + "name": "string", + "resource": { + "id_path": "string", + "type": "string" + }, + "type": "Config" + }, + "severity": "high", + "system_default": true + } + ], + "rating": "string", + "risk_score": { + "max": 0, + "value": 0 + }, + "score": "string" + } + } + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + }, + { + "cloud": { + "account": { + "id": "7111xxx59376" + }, + "provider": "aws", + "service": { + "name": "Amazon S3" + } + }, + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "threat" + ], + "kind": "alert", + "original": "{\"resource\":{\"rrn\":\"rrn::storageBucket:eu-north-1:711162259376:c7f9a5b64e82b00567aa854845xxxxxba575fce:is-34375\",\"id\":\"is-3xx75\",\"name\":\"is-3x375\",\"account\":\"AWS Cloud Account\",\"accountId\":\"7111xxx59376\",\"cloudAccountGroups\":[\"Default Account Group\"],\"region\":\"AWS Stockholm\",\"regionId\":\"eu-north-1\",\"resourceType\":\"STORAGE_BUCKET\",\"resourceApiName\":\"aws-s3api-get-bucket-acl\",\"cloudServiceName\":\"Amazon S3\",\"url\":\"https:\\/\\/console.aws.amazon.com\\/s3\\/buckets\\/is-3xx75\\/?region=eu-xxxx-1#\",\"data\":{\"acl\":{\"owner\":{\"id\":\"1ec213dd8ce39db5ea8daxxx552449effacee6337758d64dff989f85018307b1\"},\"grants\":[{\"grantee\":{\"identifier\":\"1ec213dd8ce39db5ea8dxxxx2449effacee6337758d64dff989f85018307b1\",\"typeIdentifier\":\"id\"},\"permission\":\"FullControl\"}],\"grantsAsList\":[{\"grantee\":{\"identifier\":\"1ec213dd8ce39db5ea8da3d6552449effacee6337758d64dff989f85xxx307b1\",\"typeIdentifier\":\"id\"},\"permission\":\"FullControl\"}],\"requesterCharged\":false},\"owner\":{\"id\":\"1ec213dd8ce39db5exxx3d6552449effacee6337758d64dff989f85018307b1\",\"displayName\":\"crest.it\"},\"policy\":{},\"tagSets\":{\"aws:cloudformation:stack-id\":\"arn:aws:cloudformation:ap-xxxxx-1:711162259376:stack\\/fens-cloud-account-onboarding\\/58400380-27b8-11ee-9c3e-064665xxx276\",\"aws:cloudformation:logical-id\":\"FensCloudTrailS3Bucket\",\"aws:cloudformation:stack-name\":\"fens-cloud-account-onboarding\"},\"accountId\":\"71116xxx9376\",\"bucketName\":\"is-3x375\",\"creationDate\":\"2023-06-08T12:05:18.000Z\",\"sseAlgorithm\":\"AX256\",\"policyAvailable\":true,\"ownershipControls\":{\"rules\":[{\"ownership\":\"BucketOwnerEnforced\"}]},\"loggingConfiguration\":{\"targetGrants\":[{\"grantee\":{\"identifier\":\"1ec213dd8ce39db5ea8dxxxx552449effacee6337758d64dff989f85018307b1\",\"typeIdentifier\":\"id\"},\"permission\":\"FullControl\"}]},\"versioningConfiguration\":{\"status\":\"Off\"},\"denyUnencryptedUploadsPolicies\":[],\"publicAccessBlockConfiguration\":{\"blockPublicAcls\":true,\"ignorePublicAcls\":true,\"blockPublicPolicy\":true,\"restrictPublicBuckets\":true},\"accountLevelPublicAccessBlockConfiguration\":{\"blockPublicAcls\":false,\"ignorePublicAcls\":false,\"blockPublicPolicy\":false,\"restrictPublicBuckets\":false}},\"additionalInfo\":{},\"cloudType\":\"aws\",\"resourceTs\":1690452507282,\"unifiedAssetId\":\"fd032029a5a32c6bf8xxx1f98c87a37f\",\"resourceConfigJsonAvailable\":true,\"resourceDetailsAvailable\":true}}", + "type": [ + "indicator" + ] + }, + "prisma_cloud": { + "alert": { + "resource": { + "account": { + "id": "7111xxx59376", + "value": "AWS Cloud Account" + }, + "api_name": "aws-s3api-get-bucket-acl", + "cloud": { + "account": { + "groups": [ + "Default Account Group" + ] + }, + "service_name": "Amazon S3", + "type": "aws" + }, + "config_json_available": true, + "data": { + "accountId": "71116xxx9376", + "accountLevelPublicAccessBlockConfiguration": { + "blockPublicAcls": false, + "blockPublicPolicy": false, + "ignorePublicAcls": false, + "restrictPublicBuckets": false + }, + "acl": { + "grants": [ + { + "grantee": { + "identifier": "1ec213dd8ce39db5ea8dxxxx2449effacee6337758d64dff989f85018307b1", + "typeIdentifier": "id" + }, + "permission": "FullControl" + } + ], + "grantsAsList": [ + { + "grantee": { + "identifier": "1ec213dd8ce39db5ea8da3d6552449effacee6337758d64dff989f85xxx307b1", + "typeIdentifier": "id" + }, + "permission": "FullControl" + } + ], + "owner": { + "id": "1ec213dd8ce39db5ea8daxxx552449effacee6337758d64dff989f85018307b1" + }, + "requesterCharged": false + }, + "bucketName": "is-3x375", + "creationDate": "2023-06-08T12:05:18.000Z", + "loggingConfiguration": { + "targetGrants": [ + { + "grantee": { + "identifier": "1ec213dd8ce39db5ea8dxxxx552449effacee6337758d64dff989f85018307b1", + "typeIdentifier": "id" + }, + "permission": "FullControl" + } + ] + }, + "owner": { + "displayName": "crest.it", + "id": "1ec213dd8ce39db5exxx3d6552449effacee6337758d64dff989f85018307b1" + }, + "ownershipControls": { + "rules": [ + { + "ownership": "BucketOwnerEnforced" + } + ] + }, + "policyAvailable": true, + "publicAccessBlockConfiguration": { + "blockPublicAcls": true, + "blockPublicPolicy": true, + "ignorePublicAcls": true, + "restrictPublicBuckets": true + }, + "sseAlgorithm": "AX256", + "tagSets": { + "aws:cloudformation:logical-id": "FensCloudTrailS3Bucket", + "aws:cloudformation:stack-id": "arn:aws:cloudformation:ap-xxxxx-1:711162259376:stack/fens-cloud-account-onboarding/58400380-27b8-11ee-9c3e-064665xxx276", + "aws:cloudformation:stack-name": "fens-cloud-account-onboarding" + }, + "versioningConfiguration": { + "status": "Off" + } + }, + "details_available": true, + "id": "is-3xx75", + "name": "is-3x375", + "region": { + "id": "eu-north-1", + "value": "AWS Stockholm" + }, + "rrn": "rrn::storageBucket:eu-north-1:711162259376:c7f9a5b64e82b00567aa854845xxxxxba575fce:is-34375", + "ts": "2023-07-27T10:08:27.282Z", + "type": "STORAGE_BUCKET", + "unified_asset_id": "fd032029a5a32c6bf8xxx1f98c87a37f", + "url": "https://console.aws.amazon.com/s3/buckets/is-3xx75/?region=eu-xxxx-1#" + } + } + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "url": { + "domain": "console.aws.amazon.com", + "original": "https://console.aws.amazon.com/s3/buckets/is-3xx75/?region=eu-xxxx-1#", + "path": "/s3/buckets/is-3xx75/", + "query": "region=eu-xxxx-1", + "scheme": "https" + } + }, + { + "cloud": { + "account": { + "id": "711xxx259376" + }, + "provider": "aws", + "service": { + "name": "Amazon VPC" + } + }, + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "threat" + ], + "kind": "alert", + "original": "{\"resource\":{\"rrn\":\"rrn::securityGroup:us-east-1:711162259376:437dc48e29e24dbca78xxxx1118264783cefdadd:sg-01e9ec2a09ac00f4f\",\"id\":\"sg-01e9ec2a09axxxf4f\",\"name\":\"default\",\"account\":\"AWS Cloud Account\",\"accountId\":\"711xxx259376\",\"cloudAccountGroups\":[\"Default Account Group\"],\"region\":\"AWS Virginia\",\"regionId\":\"us-xxxx-1\",\"resourceType\":\"SECURITY_GROUP\",\"resourceApiName\":\"aws-ec2-describe-security-groups\",\"cloudServiceName\":\"Amazon VPC\",\"url\":\"https:\\/\\/console.aws.amazon.com\\/vpc\\/home?region=us-xxxx-1#securityGroups:filter=sg-01e9exxx09ac00f4f\",\"data\":{\"description\":\"default VPC security group\",\"groupId\":\"sg-01e9ec2axxxc00f4f\",\"groupName\":\"default\",\"ipPermissions\":[{\"ipRanges\":[],\"prefixListIds\":[],\"userIdGroupPairs\":[{\"groupId\":\"sg-01e9ecxxx9ac00f4f\",\"userId\":\"71116xxx9376\"}],\"ipProtocol\":\"-1\",\"ipv4Ranges\":[],\"ipv6Ranges\":[]}],\"ipPermissionsEgress\":[{\"ipRanges\":[\"0.0.0.0\\/0\"],\"prefixListIds\":[],\"userIdGroupPairs\":[],\"ipProtocol\":\"-1\",\"ipv4Ranges\":[{\"cidrIp\":\"0.0.0.0\\/0\"}],\"ipv6Ranges\":[]}],\"isShared\":false,\"ownerId\":\"711xxx259376\",\"region\":\"us-east-1\",\"tags\":[],\"vpcId\":\"vpc-00f0ef31xxx957e98\"},\"additionalInfo\":{},\"cloudType\":\"aws\",\"resourceTs\":1695443838603,\"unifiedAssetId\":\"54535f5c777e41xxxx97291cab6097e7\",\"resourceConfigJsonAvailable\":true,\"resourceDetailsAvailable\":true}}", + "type": [ + "indicator" + ] + }, + "prisma_cloud": { + "alert": { + "resource": { + "account": { + "id": "711xxx259376", + "value": "AWS Cloud Account" + }, + "api_name": "aws-ec2-describe-security-groups", + "cloud": { + "account": { + "groups": [ + "Default Account Group" + ] + }, + "service_name": "Amazon VPC", + "type": "aws" + }, + "config_json_available": true, + "data": { + "description": "default VPC security group", + "groupId": "sg-01e9ec2axxxc00f4f", + "groupName": "default", + "ipPermissions": [ + { + "ipProtocol": "-1", + "userIdGroupPairs": [ + { + "groupId": "sg-01e9ecxxx9ac00f4f", + "userId": "71116xxx9376" + } + ] + } + ], + "ipPermissionsEgress": [ + { + "ipProtocol": "-1", + "ipRanges": [ + "0.0.0.0/0" + ], + "ipv4Ranges": [ + { + "cidrIp": "0.0.0.0/0" + } + ] + } + ], + "isShared": false, + "ownerId": "711xxx259376", + "region": "us-east-1", + "vpcId": "vpc-00f0ef31xxx957e98" + }, + "details_available": true, + "id": "sg-01e9ec2a09axxxf4f", + "name": "default", + "region": { + "id": "us-xxxx-1", + "value": "AWS Virginia" + }, + "rrn": "rrn::securityGroup:us-east-1:711162259376:437dc48e29e24dbca78xxxx1118264783cefdadd:sg-01e9ec2a09ac00f4f", + "ts": "2023-09-23T04:37:18.603Z", + "type": "SECURITY_GROUP", + "unified_asset_id": "54535f5c777e41xxxx97291cab6097e7", + "url": "https://console.aws.amazon.com/vpc/home?region=us-xxxx-1#securityGroups:filter=sg-01e9exxx09ac00f4f" + } + } + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "url": { + "domain": "console.aws.amazon.com", + "fragment": "securityGroups:filter=sg-01e9exxx09ac00f4f", + "original": "https://console.aws.amazon.com/vpc/home?region=us-xxxx-1#securityGroups:filter=sg-01e9exxx09ac00f4f", + "path": "/vpc/home", + "query": "region=us-xxxx-1", + "scheme": "https" + } + } + ] +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-common-config.yml b/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/prisma_cloud/data_stream/alert/_dev/test/system/test-input-config.yml b/packages/prisma_cloud/data_stream/alert/_dev/test/system/test-input-config.yml new file mode 100644 index 00000000000..a2af63f2fcb --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/_dev/test/system/test-input-config.yml @@ -0,0 +1,12 @@ +input: cel +service: prisma_cloud +vars: + username: xxxx + password: xxxx +data_stream: + vars: + url: http://{{Hostname}}:{{Port}} + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs b/packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs new file mode 100644 index 00000000000..d0d3006f132 --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/agent/stream/input.yml.hbs @@ -0,0 +1,130 @@ +config_version: 2 +interval: {{interval}} +{{#if enable_request_tracer}} +resource.tracer.filename: "../../logs/httpjson/http-request-trace-*.ndjson" +{{/if}} +{{#if proxy_url}} +resource.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +resource.url: {{url}} +state: + user: {{username}} + password: {{password}} + batch_size: {{batch_size}} + total_rows: 0 + time_amount: {{time_amount}} + time_unit: {{time_unit}} + want_more: false +redact: + fields: + - password +program: | + ( + state.with(has(state.want_more) && !(state.want_more) + ? + post_request( + state.url + "/login", + "application/json", + {"username":state.user,"password":state.password}.encode_json() + ).do_request().as(resp, bytes(resp.Body).decode_json().as(body, { + "access_token": body.token + })) + : + {} + ).as(state, state.with( + request("GET", + (has(state.want_more) && !(state.want_more) + ? + (state.url + "/v2/alert?" + "timeType=relative&timeAmount=" + + (has(state.cursor) && has(state.cursor.time_amount_in_hours) && state.cursor.time_amount_in_hours != null + ? + string(state.cursor.time_amount_in_hours) + : + string(state.time_amount)) + + "&timeUnit=" + + (has(state.cursor) && has(state.cursor.time_amount_in_hours) && state.cursor.time_amount_in_hours != null + ? + "hour" + : + state.time_unit) + + "&detailed=true" + + "&limit=" + string(state.batch_size)) + : + (state.url + "/v2/alert?" + "timeType=relative&timeAmount=" + + (has(state.cursor) && has(state.cursor.time_amount_in_hours) && state.cursor.time_amount_in_hours != null + ? + string(state.cursor.time_amount_in_hours) + : + string(state.time_amount)) + "&timeUnit=" + + (has(state.cursor) && has(state.cursor.time_amount_in_hours) && state.cursor.time_amount_in_hours != null + ? + "hour" + : + state.time_unit) + "&detailed=true" + + "&limit=" + string(state.batch_size) + + ( + has(state.page_token) + ? + "&pageToken=" + state.page_token + : + "") + ) + )).with({ + "Header":{ + "x-redlock-auth": [state.access_token], + } + }).do_request().as(resp, bytes(resp.Body).decode_json().as(inner_body, { + "events": inner_body.items.map(e, { + "message": e.encode_json(), + }), + "url": state.url, + "page_token": has(inner_body.nextPageToken) ? inner_body.nextPageToken : "", + "cursor": { + "time_amount_in_hours": ( + has(inner_body.items) && inner_body.items.size() > 0 + ? + (now() - timestamp(int(timestamp(0)+duration(string(int(inner_body.items.map(e, e.lastUpdated).max()))+"ms")))).getHours() + : + ( + has(state.cursor) && has(state.cursor.time_amount_in_hours) + ? + state.cursor.time_amount_in_hours + : + null + ) + ) + }, + "total_rows": (int(state.total_rows) + size(inner_body.items)), + "want_more": ((int(state.total_rows)) < inner_body.totalRows), + "user": state.user, + "password": state.password, + "batch_size": string(state.batch_size), + "time_amount": string(state.time_amount), + "time_unit": state.time_unit + })) + ) + ) + ) +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/alert/elasticsearch/ingest_pipeline/default.yml b/packages/prisma_cloud/data_stream/alert/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..6c8e8bd5cda --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,1510 @@ +--- +description: Pipeline for processing alert logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 8.10.0 + - set: + field: event.kind + tag: set_event_kind + value: alert + - append: + field: event.category + tag: append_event_category + value: threat + - append: + field: event.type + tag: append_event_type + value: indicator + - rename: + field: message + tag: rename_message + target_field: event.original + ignore_missing: true + - drop: + if: ctx.event?.original != null && ctx.event.original.isEmpty() + - json: + field: event.original + tag: json_event_original + target_field: json + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - fingerprint: + fields: + - json._id + - json.lastUpdated + target_field: _id + ignore_missing: true + - rename: + field: json.alertAdditionalInfo + tag: rename_alertAdditionalInfo + target_field: prisma_cloud.alert.additional_info + ignore_missing: true + - rename: + field: prisma_cloud.alert.additional_info.scannerVersion + tag: rename_alertAdditionalInfo_scannerVersion + target_field: prisma_cloud.alert.additional_info.scanner_version + ignore_missing: true + - rename: + field: json.alertAttribution + tag: rename_alertAttribution + target_field: prisma_cloud.alert.attribution + ignore_missing: true + - rename: + field: prisma_cloud.alert.attribution.attributionEventList + tag: rename_attributionEventList + target_field: prisma_cloud.alert.attribution.event_list + ignore_missing: true + - foreach: + field: prisma_cloud.alert.attribution.event_list + if: ctx.prisma_cloud?.alert?.attribution?.event_list instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.event + tag: rename_prisma_cloud_alert_attribution_event_list_event + target_field: _ingest._value.value + ignore_missing: true + - foreach: + field: prisma_cloud.alert.attribution.event_list + if: ctx.prisma_cloud?.alert?.attribution?.event_list instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.event_ts + target_field: _ingest._value.ts + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.event_ts + ignore_missing: true + - foreach: + field: prisma_cloud.alert.attribution.event_list + if: ctx.prisma_cloud?.alert?.attribution?.event_list instanceof List + ignore_failure: true + processor: + append: + field: related.user + tag: append_prisma_cloud_alert_attribution_event_list_username_to_related_user + value: '{{{_ingest._value.username}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.alert.attribution.event_list + if: ctx.prisma_cloud?.alert?.attribution?.event_list instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.event_ts + ignore_missing: true + tag: remove_prisma_cloud_alert_attribution_event_list_event_ts + - rename: + field: prisma_cloud.alert.attribution.resourceCreatedBy + tag: rename_resourceCreatedBy + target_field: prisma_cloud.alert.attribution.resource.created_by + ignore_missing: true + - date: + field: prisma_cloud.alert.attribution.resourceCreatedOn + target_field: prisma_cloud.alert.attribution.resource.created_on + tag: date_resourceCreatedOn + formats: + - UNIX_MS + if: ctx.prisma_cloud?.alert?.attribution?.resourceCreatedOn != null && ctx.prisma_cloud.alert.attribution.resourceCreatedOn != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.alertCount + tag: convert_alertCount + target_field: prisma_cloud.alert.count + type: long + ignore_missing: true + if: ctx.json?.alertCount != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.alertTime + tag: date_alertTime + target_field: prisma_cloud.alert.time + formats: + - UNIX_MS + if: ctx.json?.alertTime != null && ctx.json.alertTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.dismissalDuration + tag: rename_dismissalDuration + target_field: prisma_cloud.alert.dismissal.duration + ignore_missing: true + - rename: + field: json.dismissalNote + tag: rename_dismissalNote + target_field: prisma_cloud.alert.dismissal.note + ignore_missing: true + - date: + field: json.dismissalUntilTs + tag: date_dismissalUnitlTs + target_field: prisma_cloud.alert.dismissal.until_ts + formats: + - UNIX_MS + if: ctx.json?.dismissalUntilTs != null && ctx.json.dismissalUntilTs != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.dismissedBy + tag: rename_dismissedBy + target_field: prisma_cloud.alert.dismissed_by + ignore_missing: true + - date: + field: json.eventOccurred + tag: date_eventOccurred + target_field: prisma_cloud.alert.event_occurred + formats: + - UNIX_MS + if: ctx.json?.eventOccurred != null && ctx.json.eventOccurred != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.firstSeen + tag: date_firstSeen + target_field: prisma_cloud.alert.first_seen + formats: + - UNIX_MS + if: ctx.json?.firstSeen != null && ctx.json.firstSeen != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.start + tag: set_first_seen_to_event_start + copy_from: prisma_cloud.alert.first_seen + ignore_empty_value: true + - rename: + field: json.history + tag: rename_history + target_field: prisma_cloud.alert.history + ignore_missing: true + - foreach: + field: prisma_cloud.alert.history + if: ctx.prisma_cloud?.alert?.history instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.modifiedBy + tag: rename_prisma_cloud_alert_history_modifiedBy + target_field: _ingest._value.modified_by + ignore_missing: true + - foreach: + field: prisma_cloud.alert.history + if: ctx.prisma_cloud?.alert?.history instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.modifiedOn + target_field: _ingest._value.modified_on + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.modifiedOn + ignore_missing: true + - foreach: + field: prisma_cloud.alert.history + if: ctx.prisma_cloud?.alert?.history instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.modifiedOn + ignore_missing: true + tag: remove_prisma_cloud_alert_history_modifiedOn + - rename: + field: json.id + tag: rename_id + target_field: prisma_cloud.alert.id + ignore_missing: true + - set: + field: event.id + tag: set_id_to_event_id + copy_from: prisma_cloud.alert.id + ignore_empty_value: true + - date: + field: json.lastSeen + tag: date_lastSeen + target_field: prisma_cloud.alert.last.seen + formats: + - UNIX_MS + if: ctx.json?.lastSeen != null && ctx.json.lastSeen != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.end + tag: set_last_seen_to_event_end + copy_from: prisma_cloud.alert.last.seen + ignore_empty_value: true + - date: + field: json.lastUpdated + tag: date_lastUpdated + formats: + - UNIX_MS + if: ctx.json?.lastUpdated != null && ctx.json.lastUpdated != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.lastUpdated + tag: date_lastUpdated + target_field: prisma_cloud.alert.last.updated + formats: + - UNIX_MS + if: ctx.json?.lastUpdated != null && ctx.json.lastUpdated != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.metadata.saveSearchId + tag: rename_saveSearchId + target_field: prisma_cloud.alert.metadata.save_search_id + ignore_missing: true + - rename: + field: json.policy.cloudType + tag: rename_cloudType + target_field: prisma_cloud.alert.policy.cloud_type + ignore_missing: true + - rename: + field: json.policy.complianceMetadata + tag: rename_complianceMetadata + target_field: prisma_cloud.alert.policy.compliance_metadata + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.complianceId + tag: rename_prisma_cloud_alert_policy_compliance_metadata_complianceId + target_field: _ingest._value.compliance_id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.customAssigned + tag: convert_prisma_cloud_alert_policy_compliance_metadata_customAssigned + target_field: _ingest._value.custom_assigned + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.customAssigned + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.policyId + tag: rename_prisma_cloud_alert_policy_compliance_metadata_policyId + target_field: _ingest._value.policy_id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.requirementDescription + tag: rename_prisma_cloud_alert_policy_compliance_metadata_requirementDescription + target_field: _ingest._value.requirement.description + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.requirementId + tag: rename_prisma_cloud_alert_policy_compliance_metadata_requirementId + target_field: _ingest._value.requirement.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.requirementName + tag: rename_prisma_cloud_alert_policy_compliance_metadata_requirementName + target_field: _ingest._value.requirement.name + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sectionDescription + tag: rename_prisma_cloud_alert_policy_compliance_metadata_sectionDescription + target_field: _ingest._value.section.description + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sectionId + tag: rename_prisma_cloud_alert_policy_compliance_metadata_sectionId + target_field: _ingest._value.section.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sectionLabel + tag: rename_prisma_cloud_alert_policy_compliance_metadata_sectionLabel + target_field: _ingest._value.section.label + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.sectionViewOrder + tag: convert_prisma_cloud_alert_policy_compliance_metadata_sectionViewOrder + target_field: _ingest._value.section.view_order + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.sectionViewOrder + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.requirementViewOrder + tag: convert_prisma_cloud_alert_policy_compliance_metadata_requirementViewOrder + target_field: _ingest._value.requirement.view_order + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.requirementViewOrder + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.systemDefault + tag: convert_prisma_cloud_alert_policy_compliance_metadata_systemDefault + target_field: _ingest._value.system_default + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.systemDefault + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.standardDescription + tag: rename_prisma_cloud_alert_policy_compliance_metadata_standardDescription + target_field: _ingest._value.standard.description + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.standardId + tag: rename_prisma_cloud_alert_policy_compliance_metadata_standardId + target_field: _ingest._value.standard.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.standardName + tag: rename_prisma_cloud_alert_policy_compliance_metadata_standardName + target_field: _ingest._value.standard.name + ignore_missing: true + - foreach: + field: prisma_cloud.alert.policy.compliance_metadata + if: ctx.prisma_cloud?.alert?.policy?.compliance_metadata instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.customAssigned + - _ingest._value.sectionViewOrder + - _ingest._value.requirementViewOrder + - _ingest._value.systemDefault + ignore_missing: true + tag: remove_prisma_cloud_alert_policy_compliance_metadata_customAssigned + - rename: + field: json.policy.createdBy + tag: rename_createdBy + target_field: prisma_cloud.alert.policy.created_by + ignore_missing: true + - date: + field: json.policy.createdOn + tag: date_createdOn + target_field: prisma_cloud.alert.policy.created_on + formats: + - UNIX_MS + if: ctx.json?.policy?.createdOn != null && ctx.json.policy.createdOn != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.policy.deleted + tag: convert_deleted + target_field: prisma_cloud.alert.policy.deleted + type: boolean + ignore_missing: true + if: ctx.json?.policy?.deleted != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policy.description + tag: rename_description + target_field: prisma_cloud.alert.policy.description + ignore_missing: true + - convert: + field: json.policy.enabled + tag: convert_enabled + target_field: prisma_cloud.alert.policy.enabled + type: boolean + ignore_missing: true + if: ctx.json?.policy?.enabled != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policy.findingTypes + tag: rename_findingTypes + target_field: prisma_cloud.alert.policy.finding_types + ignore_missing: true + - rename: + field: json.policy.labels + tag: rename_labels + target_field: prisma_cloud.alert.policy.labels + ignore_missing: true + - rename: + field: json.policy.lastModifiedBy + tag: rename_lastModifiedBy + target_field: prisma_cloud.alert.policy.last_modified_by + ignore_missing: true + - date: + field: json.policy.lastModifiedOn + tag: date_lastModifiedOn + target_field: prisma_cloud.alert.policy.last_modified_on + formats: + - UNIX_MS + if: ctx.json?.policy?.lastModifiedOn != null && ctx.json.policy.lastModifiedOn != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policy.name + tag: rename_name + target_field: prisma_cloud.alert.policy.name + ignore_missing: true + - rename: + field: json.policy.policyId + tag: rename_policyId + target_field: prisma_cloud.alert.policy.id + ignore_missing: true + - rename: + field: json.policy.policyType + tag: rename_policyType + target_field: prisma_cloud.alert.policy.type + ignore_missing: true + - rename: + field: json.policy.policyUpi + tag: rename_policyUpi + target_field: prisma_cloud.alert.policy.upi + ignore_missing: true + - rename: + field: json.policy.recommendation + tag: rename_recommendation + target_field: prisma_cloud.alert.policy.recommendation + ignore_missing: true + - convert: + field: json.policy.remediable + tag: convert_remediable + target_field: prisma_cloud.alert.policy.remediable + type: boolean + ignore_missing: true + if: ctx.json?.policy?.remediable != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policy.remediation + tag: rename_remediation + target_field: prisma_cloud.alert.policy.remediation + ignore_missing: true + - rename: + field: prisma_cloud.alert.policy.remediation.cliScriptTemplate + tag: rename_cliScriptTemplate + target_field: prisma_cloud.alert.policy.remediation.cli_script_template + ignore_missing: true + - convert: + field: json.policy.restrictAlertDismissal + tag: rename_restrictAlertDismissal + target_field: prisma_cloud.alert.policy.restrict_alert_dismissal + type: boolean + ignore_missing: true + if: ctx.json?.policy?.restrictAlertDismissal != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policy.rule.apiName + tag: rename_apiName + target_field: prisma_cloud.alert.policy.rule.api_name + ignore_missing: true + - rename: + field: json.policy.rule.cloudAccount + tag: rename_cloudAccount + target_field: prisma_cloud.alert.policy.rule.cloud.account + ignore_missing: true + - rename: + field: json.policy.rule.cloudType + tag: rename_cloudType + target_field: prisma_cloud.alert.policy.rule.cloud.type + ignore_missing: true + - rename: + field: json.policy.rule.criteria + tag: rename_criteria + target_field: prisma_cloud.alert.policy.rule.criteria + ignore_missing: true + - rename: + field: json.policy.rule.dataCriteria.classificationResult + tag: rename_classifictionResult + target_field: prisma_cloud.alert.policy.rule.data_criteria.classification_result + ignore_missing: true + - rename: + field: json.policy.rule.dataCriteria.exposure + tag: rename_exposure + target_field: prisma_cloud.alert.policy.rule.data_criteria.exposure + ignore_missing: true + - rename: + field: json.policy.rule.dataCriteria.extension + tag: rename_extension + target_field: prisma_cloud.alert.policy.rule.data_criteria.extension + ignore_missing: true + - rename: + field: json.policy.rule.name + tag: rename_rule_name + target_field: prisma_cloud.alert.policy.rule.name + ignore_missing: true + - rename: + field: json.policy.rule.parameters + tag: rename_rule_parameters + target_field: prisma_cloud.alert.policy.rule.parameters + ignore_missing: true + - rename: + field: json.policy.rule.resourceIdPath + tag: rename_resourcePath + target_field: prisma_cloud.alert.policy.rule.resource.id_path + ignore_missing: true + - rename: + field: json.policy.rule.resourceType + tag: rename_resourceType + target_field: prisma_cloud.alert.policy.rule.resource.type + ignore_missing: true + - rename: + field: json.policy.rule.type + tag: rename_rule_type + target_field: prisma_cloud.alert.policy.rule.type + ignore_missing: true + - date: + field: json.policy.ruleLastModifiedOn + tag: rename_ruleLastModifiedOn + target_field: prisma_cloud.alert.policy.rule.last_modified_on + formats: + - UNIX_MS + if: ctx.json?.policy?.ruleLastModifiedOn != null && ctx.json.policy.ruleLastModifiedOn != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policy.severity + tag: rename_severity + target_field: prisma_cloud.alert.policy.severity + ignore_missing: true + - convert: + field: json.policy.systemDefault + tag: rename_systemdDefault + target_field: prisma_cloud.alert.policy.system_default + type: boolean + ignore_missing: true + if: ctx.json?.policy?.systemDefault != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.policyId + tag: rename_policyId + target_field: prisma_cloud.alert.policy_id + ignore_missing: true + - rename: + field: json.reason + tag: rename_reason + target_field: prisma_cloud.alert.reason + ignore_missing: true + - rename: + field: json.resource + tag: rename_resource + target_field: prisma_cloud.alert.resource + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.account + tag: rename_resource_account + target_field: prisma_cloud.alert.resource.account.value + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.accountId + tag: rename_accountId + target_field: prisma_cloud.alert.resource.account.id + ignore_missing: true + - set: + field: cloud.account.id + tag: set_account_id_to_cloud_account_id + copy_from: prisma_cloud.alert.resource.account.id + ignore_empty_value: true + - rename: + field: prisma_cloud.alert.resource.additionalInfo + tag: rename_additionalInfo + target_field: prisma_cloud.alert.resource.additional_info + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.cloudAccountAncestors + tag: rename_cloudAccountAncestors + target_field: prisma_cloud.alert.resource.cloud.account.ancestors + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.cloudAccountGroups + tag: rename_cloudAccountGroups + target_field: prisma_cloud.alert.resource.cloud.account.groups + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.cloudAccountOwners + tag: rename_cloudAccountOwners + target_field: prisma_cloud.alert.resource.cloud.account.owners + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.cloudServiceName + tag: rename_cloudServiceName + target_field: prisma_cloud.alert.resource.cloud.service_name + ignore_missing: true + - set: + field: cloud.service.name + tag: set_service_name_to_cloud_service_name + copy_from: prisma_cloud.alert.resource.cloud.service_name + ignore_empty_value: true + - rename: + field: prisma_cloud.alert.resource.cloudType + tag: rename_cloudType + target_field: prisma_cloud.alert.resource.cloud.type + ignore_missing: true + - set: + field: cloud.provider + tag: rename_type_to_cloud_provider + copy_from: prisma_cloud.alert.resource.cloud.type + ignore_empty_value: true + - rename: + field: prisma_cloud.alert.resource.region + tag: rename_resource_region + target_field: prisma_cloud.alert.resource.region.value + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.regionId + tag: rename_resourceId + target_field: prisma_cloud.alert.resource.region.id + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.resourceApiName + tag: rename_resourceApiName + target_field: prisma_cloud.alert.resource.api_name + ignore_missing: true + - convert: + field: prisma_cloud.alert.resource.resourceConfigJsonAvailable + tag: convert_resourceConfigJsonAvailable + target_field: prisma_cloud.alert.resource.config_json_available + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.alert?.resource?.resourceConfigJsonAvailable != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: prisma_cloud.alert.resource.resourceDetailsAvailable + tag: rename_resourceDetailsAvailable + target_field: prisma_cloud.alert.resource.details_available + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.alert?.resource?.resourceDetailsAvailable != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: prisma_cloud.alert.resource.resourceTags + tag: rename_resourceTags + target_field: prisma_cloud.alert.resource.tags + ignore_missing: true + - date: + field: prisma_cloud.alert.resource.resourceTs + tag: rename_resourceTs + target_field: prisma_cloud.alert.resource.ts + formats: + - UNIX_MS + if: ctx.prisma_cloud?.alert?.resource?.resourceTs != null && ctx.prisma_cloud.alert.resource.resourceTs != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: prisma_cloud.alert.resource.resourceType + tag: rename_resourceType + target_field: prisma_cloud.alert.resource.type + ignore_missing: true + - rename: + field: prisma_cloud.alert.resource.unifiedAssetId + tag: rename_unifiedAssetId + target_field: prisma_cloud.alert.resource.unified_asset_id + ignore_missing: true + - uri_parts: + field: prisma_cloud.alert.resource.url + tag: 'uri_parts_to_split_resource_url' + if: ctx.prisma_cloud?.alert?.resource?.url != null + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.riskDetail + tag: rename_riskDetail + target_field: prisma_cloud.alert.risk_detail + ignore_missing: true + - rename: + field: prisma_cloud.alert.risk_detail.policyScores + tag: rename_policyScores + target_field: prisma_cloud.alert.risk_detail.policy_scores + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.cloudType + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_cloudType + target_field: _ingest._value.cloud_type + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.complianceMetadata + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_complianceMetadata + target_field: _ingest._value.compliance_metadata + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.complianceId + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_complianceId + target_field: _ingest._value.compliance_id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + convert: + field: _ingest._value.customAssigned + target_field: _ingest._value.custom_assigned + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_customAssigned + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.customAssigned + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.policyId + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_policyId + target_field: _ingest._value.policy.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.requirementDescription + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_requirementDescription + target_field: _ingest._value.requirement.description + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.requirementId + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_requirementId + target_field: _ingest._value.requirement.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.requirementName + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_requirementName + target_field: _ingest._value.requirement.name + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.sectionDescription + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_sectionDescription + target_field: _ingest._value.section.description + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.sectionId + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_sectionId + target_field: _ingest._value.section.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.sectionLabel + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_sectionLabel + target_field: _ingest._value.section.label + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.standardDescription + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_standardDescription + target_field: _ingest._value.standard.description + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.standardId + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_standardId + target_field: _ingest._value.standard.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + rename: + field: _ingest._value.standardName + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_standardName + target_field: _ingest._value.standard.name + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.compliance_metadata + ignore_failure: true + processor: + remove: + field: + - _ingest._value.customAssigned + ignore_missing: true + tag: remove_prisma_cloud_alert_risk_detail_policy_scores_compliance_metadata_customAssigned + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.createdBy + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_createdBy + target_field: _ingest._value.created.by + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.createdOn + target_field: _ingest._value.created.on + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.createdOn + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.deleted + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_deleted + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.deleted + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.enabled + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_enabled + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.enabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.findingTypes + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_findingTypes + target_field: _ingest._value.finding_types + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.lastModifiedBy + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_lastModifiedBy + target_field: _ingest._value.last_modified.by + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.lastModifiedOn + target_field: _ingest._value.last_modified.on + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.lastModifiedOn + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.overridden + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_overridden + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.overridden + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.policyId + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_policyId + target_field: _ingest._value.policy.id + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.policySubTypes + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_policySubTypes + target_field: _ingest._value.policy.subtypes + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.policyType + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_policyType + target_field: _ingest._value.policy.type + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.policyUpi + tag: foreach_rename_prisma_cloud_alert_risk_detail_policy_scores_policyUpi + target_field: _ingest._value.policy.upi + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.remediable + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_remediable + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.remediable + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.remediation.cliScriptTemplate + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_cliScriptTemplate + target_field: _ingest._value.remediation.cli_script_template + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.restrictAlertDismissal + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_restrictAlertDismissal + target_field: _ingest._value.restrict_alert_dismissal + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.restrictAlertDismissal + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.riskScore.maxScore + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_maxScore + target_field: _ingest._value.risk_score.max + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.riskScore.maxScore + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.riskScore.score + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_score + target_field: _ingest._value.risk_score.value + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.riskScore.score + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.apiName + tag: foreach_rename_prisma_cloud_alert_risk_detail_policy_scores_apiName + target_field: _ingest._value.rule.api_name + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.cloudAccount + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_cloudAccount + target_field: _ingest._value.rule.cloud.account + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.cloudType + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_cloudType + target_field: _ingest._value.rule.cloud.type + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.dataCriteria + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_dataCriteria + target_field: _ingest._value.rule.data_criteria + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.data_criteria.classificationResult + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_classificationResult + target_field: _ingest._value.rule.data_criteria.classification_result + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.resourceIdPath + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_resourceIdPath + target_field: _ingest._value.rule.resource.id_path + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rule.resourceType + tag: rename_prisma_cloud_alert_risk_detail_policy_scores_resourceType + target_field: _ingest._value.rule.resource.type + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.ruleLastModifiedOn + target_field: _ingest._value.rule.last_modified_on + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.ruleLastModifiedOn + ignore_missing: true + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.systemDefault + tag: convert_prisma_cloud_alert_risk_detail_policy_scores_systemDefault + target_field: _ingest._value.system_default + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.systemDefault + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.alert.risk_detail.policy_scores + if: ctx.prisma_cloud?.alert?.risk_detail?.policy_scores instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.systemDefault + - _ingest._value.ruleLastModifiedOn + - _ingest._value.restrictAlertDismissal + - _ingest._value.lastModifiedOn + - _ingest._value.createdOn + - _ingest._value.riskScore + ignore_missing: true + tag: foreach_prisma_cloud_alert_risk_detail_policy_scores_remove + - convert: + field: prisma_cloud.alert.risk_detail.riskScore.maxScore + tag: foreach_convert_maxScore + target_field: prisma_cloud.alert.risk_detail.risk_score.max + type: long + ignore_missing: true + if: ctx.prisma_cloud?.alert?.risk_detail?.riskScore?.maxScore != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: prisma_cloud.alert.risk_detail.riskScore.score + tag: foreach_convert_score + target_field: prisma_cloud.alert.risk_detail.risk_score.value + type: long + ignore_missing: true + if: ctx.prisma_cloud?.alert?.risk_detail?.riskScore?.score != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.saveSearchId + tag: rename_saveSearchId + target_field: prisma_cloud.alert.save_search_id + ignore_missing: true + - rename: + field: json.status + tag: rename_status + target_field: prisma_cloud.alert.status + ignore_missing: true + - rename: + field: json.triggeredBy + tag: rename_triggeredBy + target_field: prisma_cloud.alert.triggered_by + ignore_missing: true + - remove: + field: + - json + - prisma_cloud.alert.attribution.resourceCreatedOn + - prisma_cloud.alert.risk_detail.riskScore + - prisma_cloud.alert.resource.resourceConfigJsonAvailable + - prisma_cloud.alert.resource.resourceDetailsAvailable + - prisma_cloud.alert.resource.resourceTs + ignore_missing: true + tag: remove_fields + - remove: + field: + - prisma_cloud.alert.first_seen + - prisma_cloud.alert.id + - prisma_cloud.alert.last.seen + - prisma_cloud.alert.last.updated + - prisma_cloud.alert.resource.account.id + - prisma_cloud.alert.resource.cloud.service_name + - prisma_cloud.alert.resource.cloud.type + ignore_missing: true + tag: remove_preserve_duplicate_custom_fields + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + - remove: + field: event.original + tag: remove_event_original + ignore_missing: true + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + - script: + lang: painless + tag: script_painless + source: |- + boolean drop(Object object) { + if (object == null || object == '') { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(v -> drop(v)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(v -> drop(v)); + return (((List) object).length == 0); + } + return false; + } + drop(ctx); + description: Drops null/empty values recursively. + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + tag: set_event_kind_pipeline_error + value: pipeline_error + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + value: pipeline_error diff --git a/packages/prisma_cloud/data_stream/alert/fields/base-fields.yml b/packages/prisma_cloud/data_stream/alert/fields/base-fields.yml new file mode 100644 index 00000000000..ad329df3a9e --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: prisma_cloud +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: prisma_cloud.alert +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/prisma_cloud/data_stream/alert/fields/beats.yml b/packages/prisma_cloud/data_stream/alert/fields/beats.yml new file mode 100644 index 00000000000..b3701b581cf --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/fields/beats.yml @@ -0,0 +1,9 @@ +- name: input.type + type: keyword + description: Type of filebeat input. +- name: log.offset + type: long + description: Log offset. +- name: tags + type: keyword + description: User defined tags. diff --git a/packages/prisma_cloud/data_stream/alert/fields/fields.yml b/packages/prisma_cloud/data_stream/alert/fields/fields.yml new file mode 100644 index 00000000000..73ef5344d0c --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/fields/fields.yml @@ -0,0 +1,464 @@ +- name: prisma_cloud + type: group + fields: + - name: alert + type: group + fields: + - name: additional_info + type: group + fields: + - name: scanner_version + type: keyword + - name: attribution + type: group + fields: + - name: event_list + type: group + fields: + - name: ts + type: date + - name: username + type: keyword + - name: value + type: keyword + - name: resource + type: group + fields: + - name: created_by + type: keyword + - name: created_on + type: date + - name: count + type: long + - name: dismissal + type: group + fields: + - name: duration + type: keyword + - name: note + type: keyword + - name: until_ts + type: date + - name: dismissed_by + type: keyword + - name: event_occurred + type: date + description: Timestamp when the event occurred. Set only for Audit Event policies. + - name: first_seen + type: date + description: Timestamp of the first policy violation for the alert resource (i.e. the alert creation timestamp). + - name: history + type: group + fields: + - name: modified_by + type: keyword + - name: modified_on + type: date + - name: reason + type: keyword + - name: status + type: keyword + - name: id + type: keyword + description: Alert ID. + - name: last + type: group + fields: + - name: seen + type: date + description: Timestamp when alert status was last updated. + - name: updated + type: date + description: Timestamp when alert was last updated. Updates include but are not limited to resource updates, policy updates, alert rule updates, and alert status changes. + - name: metadata + type: group + fields: + - name: save_search_id + type: keyword + - name: policy + type: group + fields: + - name: cloud_type + type: keyword + description: "Possible values: [ALL, AWS, AZURE, GCP, ALIBABA_CLOUD, OCI, IBM] Cloud type (Required for config policies). Not case-sensitive. Default is ALL." + - name: compliance_metadata + type: group + fields: + - name: compliance_id + type: keyword + description: Compliance Section UUID. + - name: custom_assigned + type: boolean + - name: policy_id + type: keyword + - name: requirement + type: group + fields: + - name: description + type: keyword + - name: id + type: keyword + - name: name + type: keyword + - name: view_order + type: keyword + - name: section + type: group + fields: + - name: description + type: keyword + - name: id + type: keyword + - name: label + type: keyword + - name: view_order + type: long + - name: standard + type: group + fields: + - name: description + type: keyword + - name: id + type: keyword + - name: name + type: keyword + - name: system_default + type: boolean + - name: created_by + type: keyword + - name: created_on + type: date + - name: deleted + type: boolean + - name: description + type: keyword + - name: enabled + type: boolean + - name: finding_types + type: keyword + - name: id + type: keyword + - name: labels + type: keyword + - name: last_modified_by + type: keyword + - name: last_modified_on + type: date + - name: name + type: keyword + - name: recommendation + type: keyword + - name: remediable + type: boolean + - name: remediation + type: group + fields: + - name: actions + type: group + fields: + - name: operation + type: keyword + - name: payload + type: keyword + - name: cli_script_template + type: keyword + - name: description + type: keyword + - name: rule + type: group + fields: + - name: api_name + type: keyword + - name: cloud + type: group + fields: + - name: account + type: keyword + - name: type + type: keyword + - name: criteria + type: keyword + description: Saved search ID that defines the rule criteria. + - name: data_criteria + type: group + fields: + - name: classification_result + type: keyword + description: Data policy. Required for DLP rule criteria. + - name: exposure + type: keyword + description: Possible values [private, public, conditional]. + - name: extension + type: keyword + - name: last_modified_on + type: date + - name: name + type: keyword + - name: parameters + type: flattened + - name: resource + type: group + fields: + - name: id_path + type: keyword + - name: type + type: keyword + - name: type + type: keyword + description: Possible values [Config, Network, AuditEvent, DLP, IAM, NetworkConfig] Type of rule or RQL query. + - name: severity + type: keyword + description: Possible values [high, medium, low]. + - name: system_default + type: boolean + - name: type + type: keyword + description: "Possible values: [config, network, audit_event, anomaly, data, iam, workload_vulnerability, workload_incident, waas_event, attack_path] Policy type. Policy type anomaly is read-only." + - name: upi + type: keyword + - name: policy_id + type: keyword + - name: reason + type: keyword + - name: resource + type: group + fields: + - name: account + type: group + fields: + - name: id + type: keyword + - name: value + type: keyword + - name: additional_info + type: flattened + description: Additional info. + - name: api_name + type: keyword + - name: cloud + type: group + fields: + - name: account + type: group + fields: + - name: ancestors + type: keyword + - name: groups + type: keyword + - name: owners + type: keyword + - name: service_name + type: keyword + - name: type + type: keyword + - name: config_json_available + type: boolean + - name: data + type: flattened + - name: details_available + type: boolean + - name: id + type: keyword + - name: name + type: keyword + - name: region + type: group + fields: + - name: id + type: keyword + - name: value + type: keyword + - name: rrn + type: keyword + - name: tags + type: flattened + - name: ts + type: date + - name: type + type: keyword + - name: unified_asset_id + type: keyword + - name: url + type: keyword + - name: risk_detail + type: group + fields: + - name: policy_scores + type: group + fields: + - name: cloud_type + type: keyword + - name: compliance_metadata + type: group + fields: + - name: compliance + type: group + fields: + - name: id + type: keyword + - name: custom_assigned + type: boolean + - name: policy + type: group + fields: + - name: id + type: keyword + - name: requirement + type: group + fields: + - name: description + type: keyword + - name: id + type: keyword + - name: name + type: keyword + - name: section + type: group + fields: + - name: description + type: keyword + - name: id + type: keyword + - name: label + type: keyword + - name: standard + type: group + fields: + - name: description + type: keyword + - name: id + type: keyword + - name: name + type: keyword + - name: created + type: group + fields: + - name: by + type: keyword + - name: "on" + type: date + - name: deleted + type: boolean + - name: description + type: keyword + - name: enabled + type: boolean + - name: finding_types + type: keyword + - name: labels + type: keyword + - name: last_modified + type: group + fields: + - name: by + type: keyword + - name: "on" + type: date + - name: name + type: keyword + - name: overridden + type: boolean + - name: points + type: keyword + - name: policy + type: group + fields: + - name: id + type: keyword + - name: subtypes + type: keyword + - name: type + type: keyword + - name: upi + type: keyword + - name: recommendation + type: keyword + - name: remediable + type: boolean + - name: remediation + type: group + fields: + - name: actions + type: group + fields: + - name: operation + type: keyword + - name: payload + type: keyword + - name: cli_script_template + type: keyword + - name: description + type: keyword + - name: impact + type: keyword + - name: restrict_alert_dismissal + type: boolean + - name: risk_score + type: group + fields: + - name: max + type: long + - name: value + type: long + - name: rule + type: group + fields: + - name: api_name + type: keyword + - name: cloud + type: group + fields: + - name: account + type: keyword + - name: type + type: keyword + - name: criteria + type: keyword + - name: data_criteria + type: group + fields: + - name: classification_result + type: keyword + - name: exposure + type: keyword + - name: extension + type: keyword + - name: last_modified_on + type: date + - name: name + type: keyword + - name: parameters + type: flattened + - name: resource + type: group + fields: + - name: id_path + type: keyword + - name: type + type: keyword + - name: type + type: keyword + - name: severity + type: keyword + - name: system_default + type: boolean + - name: rating + type: keyword + - name: risk_score + type: group + fields: + - name: max + type: long + - name: value + type: long + - name: score + type: keyword + - name: save_search_id + type: keyword + - name: status + type: keyword + - name: time + type: date + description: Timestamp when alert was last reopened for resource update, or the same as firstSeen if there are no status changes. + - name: triggered_by + type: keyword diff --git a/packages/prisma_cloud/data_stream/alert/manifest.yml b/packages/prisma_cloud/data_stream/alert/manifest.yml new file mode 100644 index 00000000000..23aa8f0ef79 --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/manifest.yml @@ -0,0 +1,95 @@ +title: Collect Alert logs from Prisma Cloud Security Posture Management. +type: logs +streams: + - input: cel + title: Alert Logs + description: Collect Alert logs from Prisma Cloud Security Posture Management. + template_path: input.yml.hbs + enabled: false + vars: + - name: url + type: text + title: URL + description: Base URL of the Prisma Cloud Server API. + required: true + show_user: true + - name: interval + type: text + title: Interval + description: Interval between two REST API calls. Supported units for this parameter are h/m/s. + default: 1m + multi: false + required: true + show_user: true + - name: time_amount + type: integer + title: Time Amount + description: Number of Time Units. + default: 5 + multi: false + required: true + show_user: true + - name: time_unit + type: text + title: Time Unit + description: Possible Value for this parameter are minute/hour/day/week/month/year. + default: day + multi: false + required: true + show_user: true + - name: batch_size + type: integer + title: Batch Size + description: The maximum number of items that will be returned in one response. The maximum cannot exceed 10,000. + default: 10000 + multi: false + required: true + show_user: false + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Supported time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + required: false + show_user: false + description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/prisma_cloud/data_stream/alert/sample_event.json b/packages/prisma_cloud/data_stream/alert/sample_event.json new file mode 100644 index 00000000000..e7c8e4ca678 --- /dev/null +++ b/packages/prisma_cloud/data_stream/alert/sample_event.json @@ -0,0 +1,186 @@ +{ + "@timestamp": "2023-09-06T12:30:41.966Z", + "agent": { + "ephemeral_id": "7aae6130-635a-422f-ac2e-e40324e86921", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "cloud": { + "account": { + "id": "710002259376" + }, + "provider": "aws", + "service": { + "name": "Amazon EC2" + } + }, + "data_stream": { + "dataset": "prisma_cloud.alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "threat" + ], + "dataset": "prisma_cloud.alert", + "end": "2023-09-06T12:30:41.966Z", + "id": "N-3910", + "ingested": "2023-10-18T12:08:31Z", + "kind": "alert", + "original": "{\"alertAdditionalInfo\":{\"scannerVersion\":\"CS_2.0\"},\"alertAttribution\":{\"attributionEventList\":[{\"event\":\"first_event\",\"event_ts\":1694003441966,\"username\":\"alex123\"}],\"resourceCreatedBy\":\"string\",\"resourceCreatedOn\":0},\"alertRules\":[],\"alertTime\":1694003441966,\"firstSeen\":1694003441966,\"history\":[{\"modifiedBy\":\"alex123\",\"modifiedOn\":\"1694003441966\",\"reason\":\"Reason1\",\"status\":\"OPEN\"}],\"id\":\"N-3910\",\"investigateOptions\":{\"alertId\":\"N-3910\"},\"lastSeen\":1694003441966,\"lastUpdated\":1694003441966,\"metadata\":null,\"policy\":{\"complianceMetadata\":[{\"complianceId\":\"qwer345bv\",\"customAssigned\":true,\"policyId\":\"werf435tr\",\"requirementDescription\":\"Description of policy compliance.\",\"requirementId\":\"req-123-xyz\",\"requirementName\":\"rigidity\",\"sectionDescription\":\"Description of section.\",\"sectionId\":\"sect-453-abc\",\"sectionLabel\":\"label-1\",\"standardDescription\":\"Description of standard.\",\"standardId\":\"stand-543-pqr\",\"standardName\":\"Class 1\"}],\"deleted\":false,\"description\":\"This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.\",\"findingTypes\":[],\"labels\":[\"Prisma_Cloud\",\"Attack Path Rule\"],\"lastModifiedBy\":\"template@redlock.io\",\"lastModifiedOn\":1687474999057,\"name\":\"AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0/0)\",\"policyId\":\"ad23603d-754e-4499-8988-b8017xxxx98\",\"policyType\":\"network\",\"recommendation\":\"The following steps are recommended to restrict unrestricted access from the Internet:\\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\\n2. Identify the network component on which restrictive rules can be implemented.\\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\\n a) The overly permissive Security Group rules can be made more restrictive.\\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.\",\"remediable\":false,\"remediation\":{\"actions\":[{\"operation\":\"buy\",\"payload\":\"erefwsdf\"}],\"cliScriptTemplate\":\"temp1\",\"description\":\"Description of CLI Script Template.\"},\"severity\":\"high\",\"systemDefault\":true},\"policyId\":\"ad23603d-754e-4499-8988-b801xxx85898\",\"reason\":\"NEW_ALERT\",\"resource\":{\"account\":\"AWS Cloud Account\",\"accountId\":\"710002259376\",\"additionalInfo\":null,\"cloudAccountGroups\":[\"Default Account Group\"],\"cloudServiceName\":\"Amazon EC2\",\"cloudType\":\"aws\",\"data\":null,\"id\":\"i-04578exxxx8100947\",\"name\":\"IS-37133\",\"region\":\"AWS Virginia\",\"regionId\":\"us-east-1\",\"resourceApiName\":\"aws-ec2-describe-instances\",\"resourceConfigJsonAvailable\":false,\"resourceDetailsAvailable\":true,\"resourceTs\":1694003441915,\"resourceType\":\"INSTANCE\",\"rrn\":\"rrn:aws:instance:us-east-1:710000059376:e7ddce5a1ffcb47bxxxxxerf2635a3b4d9da3:i-04578e0008100947\",\"unifiedAssetId\":\"66c543b6261c4d9edxxxxxb42e15f4\",\"url\":\"https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947\"},\"status\":\"open\"}", + "start": "2023-09-06T12:30:41.966Z", + "type": [ + "indicator" + ] + }, + "input": { + "type": "cel" + }, + "prisma_cloud": { + "alert": { + "additional_info": { + "scanner_version": "CS_2.0" + }, + "attribution": { + "event_list": [ + { + "ts": "2023-09-06T12:30:41.966Z", + "username": "alex123", + "value": "first_event" + } + ], + "resource": { + "created_by": "string", + "created_on": "1970-01-01T00:00:00.000Z" + } + }, + "first_seen": "2023-09-06T12:30:41.966Z", + "history": [ + { + "modified_by": "alex123", + "modified_on": "2023-09-06T12:30:41.966Z", + "reason": "Reason1", + "status": "OPEN" + } + ], + "id": "N-3910", + "last": { + "seen": "2023-09-06T12:30:41.966Z", + "updated": "2023-09-06T12:30:41.966Z" + }, + "policy": { + "compliance_metadata": [ + { + "compliance_id": "qwer345bv", + "custom_assigned": true, + "policy_id": "werf435tr", + "requirement": { + "description": "Description of policy compliance.", + "id": "req-123-xyz", + "name": "rigidity" + }, + "section": { + "description": "Description of section.", + "id": "sect-453-abc", + "label": "label-1" + }, + "standard": { + "description": "Description of standard.", + "id": "stand-543-pqr", + "name": "Class 1" + } + } + ], + "deleted": false, + "description": "This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.", + "id": "ad23603d-754e-4499-8988-b8017xxxx98", + "labels": [ + "Prisma_Cloud", + "Attack Path Rule" + ], + "last_modified_by": "template@redlock.io", + "last_modified_on": "2023-06-22T23:03:19.057Z", + "name": "AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0/0)", + "recommendation": "The following steps are recommended to restrict unrestricted access from the Internet:\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\n2. Identify the network component on which restrictive rules can be implemented.\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\n a) The overly permissive Security Group rules can be made more restrictive.\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.", + "remediable": false, + "remediation": { + "actions": [ + { + "operation": "buy", + "payload": "erefwsdf" + } + ], + "cli_script_template": "temp1", + "description": "Description of CLI Script Template." + }, + "severity": "high", + "system_default": true, + "type": "network" + }, + "policy_id": "ad23603d-754e-4499-8988-b801xxx85898", + "reason": "NEW_ALERT", + "resource": { + "account": { + "id": "710002259376", + "value": "AWS Cloud Account" + }, + "api_name": "aws-ec2-describe-instances", + "cloud": { + "account": { + "groups": [ + "Default Account Group" + ] + }, + "service_name": "Amazon EC2", + "type": "aws" + }, + "config_json_available": false, + "details_available": true, + "id": "i-04578exxxx8100947", + "name": "IS-37133", + "region": { + "id": "us-east-1", + "value": "AWS Virginia" + }, + "rrn": "rrn:aws:instance:us-east-1:710000059376:e7ddce5a1ffcb47bxxxxxerf2635a3b4d9da3:i-04578e0008100947", + "ts": "2023-09-06T12:30:41.915Z", + "type": "INSTANCE", + "unified_asset_id": "66c543b6261c4d9edxxxxxb42e15f4", + "url": "https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947" + }, + "status": "open", + "time": "2023-09-06T12:30:41.966Z" + } + }, + "related": { + "user": [ + "alex123" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-alert" + ], + "url": { + "domain": "console.aws.amazon.com", + "fragment": "Instances:instanceId=i-0457xxxxx00947", + "original": "https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947", + "path": "/ec2/v2/home", + "query": "region=us-east-1", + "scheme": "https" + } +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-audit.log b/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-audit.log new file mode 100644 index 00000000000..ed7b90bb7d2 --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-audit.log @@ -0,0 +1 @@ +{"timestamp":1693819702240,"user":"john.user@google.com","ipAddress":"81.2.69.142","actionType":"LOGIN","resourceName":"john.user@google.com","action":"'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key.","resourceType":"Login","result":"fail"} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-audit.log-expected.json b/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-audit.log-expected.json new file mode 100644 index 00000000000..4ead130a4bd --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-audit.log-expected.json @@ -0,0 +1,58 @@ +{ + "expected": [ + { + "@timestamp": "2023-09-04T09:28:22.240Z", + "ecs": { + "version": "8.10.0" + }, + "event": { + "action": "login", + "category": [ + "authentication" + ], + "kind": "event", + "original": "{\"timestamp\":1693819702240,\"user\":\"john.user@google.com\",\"ipAddress\":\"81.2.69.142\",\"actionType\":\"LOGIN\",\"resourceName\":\"john.user@google.com\",\"action\":\"'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key.\",\"resourceType\":\"Login\",\"result\":\"fail\"}", + "outcome": "failure", + "type": [ + "info" + ] + }, + "host": { + "ip": [ + "81.2.69.142" + ] + }, + "prisma_cloud": { + "audit": { + "action": { + "type": "LOGIN", + "value": "'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key." + }, + "ip_address": "81.2.69.142", + "resource": { + "name": "john.user@google.com", + "type": "Login" + }, + "result": "fail", + "timestamp": "2023-09-04T09:28:22.240Z", + "user": "john.user@google.com" + } + }, + "related": { + "ip": [ + "81.2.69.142" + ], + "user": [ + "john.user@google.com" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "user": { + "email": "john.user@google.com" + } + } + ] +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-common-config.yml b/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/prisma_cloud/data_stream/audit/_dev/test/system/test-input-config.yml b/packages/prisma_cloud/data_stream/audit/_dev/test/system/test-input-config.yml new file mode 100644 index 00000000000..a2af63f2fcb --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/_dev/test/system/test-input-config.yml @@ -0,0 +1,12 @@ +input: cel +service: prisma_cloud +vars: + username: xxxx + password: xxxx +data_stream: + vars: + url: http://{{Hostname}}:{{Port}} + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/audit/agent/stream/input.yml.hbs b/packages/prisma_cloud/data_stream/audit/agent/stream/input.yml.hbs new file mode 100644 index 00000000000..103aa5ca360 --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/agent/stream/input.yml.hbs @@ -0,0 +1,93 @@ +config_version: 2 +interval: {{interval}} +{{#if enable_request_tracer}} +resource.tracer.filename: "../../logs/httpjson/http-request-trace-*.ndjson" +{{/if}} +{{#if proxy_url}} +resource.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +resource.url: {{url}} +state: + user: {{username}} + password: {{password}} + time_amount: {{time_amount}} + time_unit: {{time_unit}} + state_interval: {{interval}} +redact: + fields: + - password +program: | + ( + state.with( + post_request( + state.url + "/login", + "application/json", + {"username":state.user,"password":state.password}.encode_json() + ).do_request().as(resp, bytes(resp.Body).decode_json().as(body, { + "access_token": body.token, + })) + ).as(state, state.with( + request("GET", + state.url + + "/audit/redlock?timeType=relative&timeAmount=" + + (has(state.cursor) && has(state.cursor.time_amount_in_minute) && state.cursor.time_amount_in_minute != null + ? + string(state.cursor.time_amount_in_minute) + : + string(state.time_amount)) + + "&timeUnit=" + + (has(state.cursor) && has(state.cursor.time_amount_in_minute) && state.cursor.time_amount_in_minute != null + ? + "minute" + : + state.time_unit) + ).with({ + "Header":{ + "x-redlock-auth": [state.access_token], + } + }).do_request().as(resp, bytes(resp.Body).decode_json().as(inner_body, { + "events": inner_body.map(e, { + "message": e.encode_json(), + }), + "cursor": { + "time_amount_in_minute": ( + inner_body.size() > 0 + ? + (int((now() - timestamp(int(timestamp(0)+duration(string(int(inner_body.map(e, e.timestamp).max()))+"ms")))).getSeconds()) + int((duration(state.state_interval)).getSeconds()))/60 + : + ( + has(state.cursor) && has(state.cursor.time_amount_in_minute) + ? + state.cursor.time_amount_in_minute + : + null + ) + ) + } + })) + ) + ) + ) +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/audit/elasticsearch/ingest_pipeline/default.yml b/packages/prisma_cloud/data_stream/audit/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..869d8a944ea --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,213 @@ +--- +description: Pipeline for processing audit logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 8.10.0 + - set: + field: event.kind + tag: set_event_kind_1 + value: event + - append: + field: event.type + tag: append_event_type + value: info + - rename: + field: message + tag: rename_message + target_field: event.original + ignore_missing: true + - drop: + if: ctx.event?.original != null && ctx.event.original.isEmpty() + - json: + field: event.original + tag: json_message + target_field: json + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.ipAddress + tag: convert_ip_address_to_ip + target_field: prisma_cloud.audit.ip_address + type: ip + ignore_missing: true + if: ctx.json?.ipAddress != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: host.ip + tag: append_ip_address_to_host_ip + value: '{{{prisma_cloud.audit.ip_address}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.audit?.ip_address != null + - append: + field: related.ip + tag: convert_ip_address_to_related_ip + value: '{{{prisma_cloud.audit.ip_address}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.audit?.ip_address != null + - rename: + field: json.user + tag: rename_prisma_cloud_audit_user + target_field: prisma_cloud.audit.user + ignore_missing: true + - set: + field: user.email + tag: set_prisma_cloud_audit_user + copy_from: prisma_cloud.audit.user + ignore_empty_value: true + - append: + field: related.user + tag: append_prisma_cloud_audit_user_to_related_user + value: '{{{prisma_cloud.audit.user}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.audit?.user != null + - rename: + field: json.action + tag: rename_prisma_cloud_audit_action + target_field: prisma_cloud.audit.action.value + ignore_missing: true + - rename: + field: json.actionType + tag: rename_prisma_cloud_audit_action_type + target_field: prisma_cloud.audit.action.type + ignore_missing: true + - lowercase: + field: prisma_cloud.audit.action.type + tag: 'lowercase_action_type' + ignore_missing: true + target_field: event.action + if: ctx.prisma_cloud?.audit?.action?.type != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - split: + field: event.action + tag: split_action_type + separator: '\s+' + if: ctx.event?.action != null && ctx.event?.action != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - join: + field: event.action + tag: join_action_type + separator: "-" + if: ctx.event?.action instanceof List + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: event.category + tag: append_event_category + value: authentication + if: ctx.prisma_cloud?.audit?.action?.type != null && (ctx.prisma_cloud.audit.action.type.toLowerCase().contains('login') || ctx.prisma_cloud.audit.action.type.toLowerCase().contains('logout')) + allow_duplicates: false + - rename: + field: json.resourceName + tag: rename_prisma_cloud_audit_resource_name + target_field: prisma_cloud.audit.resource.name + ignore_missing: true + - rename: + field: json.resourceType + tag: rename_prisma_cloud_audit_resource_type + target_field: prisma_cloud.audit.resource.type + ignore_missing: true + - rename: + field: json.result + tag: rename_json_result + target_field: prisma_cloud.audit.result + ignore_missing: true + - script: + lang: painless + tag: script_to_set_result + description: Script to set result for different ranges. + if: ctx.prisma_cloud?.audit?.result != null + source: >- + if (ctx.prisma_cloud?.audit?.result != null && ctx.prisma_cloud.audit.result.toLowerCase().contains("success")){ + ctx.event.outcome = "success"; + } else if (ctx.prisma_cloud?.audit?.result != null && ctx.prisma_cloud.audit.result.toLowerCase().contains("fail")){ + ctx.event.outcome = "failure"; + } else { + ctx.event.outcome = "unknown"; + } + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.timestamp + tag: 'date_rename_timestamp_to_custom_name' + formats: + - UNIX_MS + if: ctx.json?.timestamp != null && ctx.json.timestamp != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.timestamp + tag: 'date_rename_timestamp_to_custom_name' + target_field: prisma_cloud.audit.timestamp + formats: + - UNIX_MS + if: ctx.json?.timestamp != null && ctx.json.timestamp != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - remove: + field: + - json + ignore_missing: true + - remove: + field: + - event.original + ignore_missing: true + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + - remove: + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + ignore_missing: true + tag: remove_preserve_duplicate_custom_fields + field: + - prisma_cloud.audit.ip_address + - prisma_cloud.audit.user + - prisma_cloud.audit.action.type + - prisma_cloud.audit.result + - prisma_cloud.audit.timestamp + - script: + lang: painless + description: Drops null/empty values recursively. + source: |- + boolean drop(Object object) { + if (object == null || object == '') { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(v -> drop(v)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(v -> drop(v)); + return (((List) object).length == 0); + } + return false; + } + drop(ctx); + - set: + field: event.kind + value: pipeline_error + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + value: pipeline_error diff --git a/packages/prisma_cloud/data_stream/audit/fields/base-fields.yml b/packages/prisma_cloud/data_stream/audit/fields/base-fields.yml new file mode 100644 index 00000000000..3228f592fcd --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: prisma_cloud +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: prisma_cloud.audit +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/prisma_cloud/data_stream/audit/fields/beats.yml b/packages/prisma_cloud/data_stream/audit/fields/beats.yml new file mode 100644 index 00000000000..b3701b581cf --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/fields/beats.yml @@ -0,0 +1,9 @@ +- name: input.type + type: keyword + description: Type of filebeat input. +- name: log.offset + type: long + description: Log offset. +- name: tags + type: keyword + description: User defined tags. diff --git a/packages/prisma_cloud/data_stream/audit/fields/fields.yml b/packages/prisma_cloud/data_stream/audit/fields/fields.yml new file mode 100644 index 00000000000..81191b5e609 --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/fields/fields.yml @@ -0,0 +1,32 @@ +- name: prisma_cloud + type: group + fields: + - name: audit + type: group + fields: + - name: action + type: group + fields: + - name: type + type: keyword + description: Action Type. + - name: value + type: keyword + - name: ip_address + type: ip + description: IP Address. + - name: resource + type: group + fields: + - name: name + type: keyword + - name: type + type: keyword + - name: result + type: keyword + - name: timestamp + type: date + description: Timestamp. + - name: user + type: keyword + description: User. diff --git a/packages/prisma_cloud/data_stream/audit/manifest.yml b/packages/prisma_cloud/data_stream/audit/manifest.yml new file mode 100644 index 00000000000..4cc9b62eaa7 --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/manifest.yml @@ -0,0 +1,87 @@ +title: Collect Audit logs from Prisma Cloud Security Posture Management. +type: logs +streams: + - input: cel + title: Audit Logs + description: Collect Audit logs from Prisma Cloud Security Posture Management. + template_path: input.yml.hbs + enabled: false + vars: + - name: url + type: text + title: URL + description: Base URL of the Prisma Cloud Server API. + required: true + show_user: true + - name: interval + type: text + title: Interval + description: Interval between two REST API calls. Supported units for this parameter are h/m/s. + default: 1m + multi: false + required: true + show_user: true + - name: time_amount + type: integer + title: Time Amount + description: Number of Time Units. + default: 5 + multi: false + required: true + show_user: true + - name: time_unit + type: text + title: Time Unit + description: Possible Value for this parameter are minute/hour/day/week/month/year. + default: day + multi: false + required: true + show_user: true + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Supported time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + required: false + show_user: false + description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-audit + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.audit fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/prisma_cloud/data_stream/audit/sample_event.json b/packages/prisma_cloud/data_stream/audit/sample_event.json new file mode 100644 index 00000000000..feb29ae7767 --- /dev/null +++ b/packages/prisma_cloud/data_stream/audit/sample_event.json @@ -0,0 +1,79 @@ +{ + "@timestamp": "2023-09-13T08:40:39.068Z", + "agent": { + "ephemeral_id": "7aae6130-635a-422f-ac2e-e40324e86921", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "data_stream": { + "dataset": "prisma_cloud.audit", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "action": "login", + "agent_id_status": "verified", + "category": [ + "authentication" + ], + "dataset": "prisma_cloud.audit", + "ingested": "2023-10-18T12:09:30Z", + "kind": "event", + "original": "{\"action\":\"'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key.\",\"actionType\":\"LOGIN\",\"ipAddress\":\"81.2.69.192\",\"resourceName\":\"john.user@google.com\",\"resourceType\":\"Login\",\"result\":\"Successful\",\"timestamp\":1694594439068,\"user\":\"john.user@google.com\"}", + "outcome": "success", + "type": [ + "info" + ] + }, + "host": { + "ip": [ + "81.2.69.192" + ] + }, + "input": { + "type": "cel" + }, + "prisma_cloud": { + "audit": { + "action": { + "type": "LOGIN", + "value": "'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key." + }, + "ip_address": "81.2.69.192", + "resource": { + "name": "john.user@google.com", + "type": "Login" + }, + "result": "Successful", + "timestamp": "2023-09-13T08:40:39.068Z", + "user": "john.user@google.com" + } + }, + "related": { + "ip": [ + "81.2.69.192" + ], + "user": [ + "john.user@google.com" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-audit" + ], + "user": { + "email": "john.user@google.com" + } +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-common-config.yml b/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-host.log b/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-host.log new file mode 100644 index 00000000000..1e01f48b88e --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-host.log @@ -0,0 +1 @@ +{"_id":"DESKTOP-6PQJAMS","binaries":[{"altered":true,"cveCount":0,"deps":["string"],"fileMode":0,"functionLayer":"string","md5":"string","missingPkg":true,"name":"string","path":"string","pkgRootDir":"string","services":["string"],"version":"string"}],"startupBinaries":[{"altered":true,"cveCount":0,"deps":["string"],"fileMode":0,"functionLayer":"string","md5":"string","missingPkg":true,"name":"string","path":"string","pkgRootDir":"string","services":["string"],"version":"string"}],"cloudMetadata":{"accountID":"Non-onboarded cloud accounts","awsExecutionEnv":"string","image":"string","labels":[{"key":"string","sourceName":"string","sourceType":["namespace","deployment","aws","azure","gcp","oci"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"name":"string","provider":["aws"],"region":"string","resourceID":"string","resourceURL":"string","type":"string","vmID":"string","vmImageID":"string"},"type":"host","hostname":"DESKTOP-6PQJAMS","scanTime":"2023-08-23T11:48:41.803Z","Secrets":[],"osDistro":"windows","osDistroVersion":"string","osDistroRelease":"Windows","distro":"Microsoft Windows [Version 10.0.19045.2006]","packageManager":true,"packages":[{"pkgs":[{"binaryIdx":[0],"binaryPkgs":["string"],"cveCount":0,"defaultGem":true,"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"functionLayer":"string","goPkg":true,"jarIdentifier":"string","layerTime":0,"license":"string","name":"string","osPackage":true,"path":"string","version":"string"}],"pkgsType":"nodejs"}],"isARM64":false,"packageCorrelationDone":true,"redHatNonRPMImage":false,"image":{"created":"0001-01-01T00:00:00Z","entrypoint":["string"],"env":["string"],"healthcheck":true,"history":[{"baseLayer":true,"created":0,"emptyLayer":true,"id":"string","instruction":"string","sizeBytes":0,"tags":["string"],"vulnerabilities":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.950Z","exploit":["","exploit-db","exploit-windows","cisa-kev"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db","exploit-windows","cisa-kev"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI","HIPAA","NIST SP 800-190","GDPR","DISA STIG"]],"text":"string","title":"string","twistlock":true,"type":["container","image","host_config","daemon_config","daemon_config_files","security_operations","k8s_master","k8s_worker","k8s_federation","linux","windows","istio","serverless","custom","docker_stig","openshift_master","openshift_worker","application_control_linux","gke_worker","image_malware","host_malware"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}]}],"id":"string","labels":{},"layers":["string"],"os":"string","repoDigest":["string"],"repoTags":["string"],"user":"string","workingDir":"string"},"allCompliance":{"compliance":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.949Z","exploit":["","exploit-db","exploit-windows","cisa-kev"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db","exploit-windows","cisa-kev"]},{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db","exploit-windows","cisa-kev"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI","HIPAA","NIST SP 800-190","GDPR","DISA STIG"]],"text":"string","title":"string","twistlock":true,"type":["container","image","host_config","daemon_config","daemon_config_files","security_operations","k8s_master","k8s_worker","k8s_federation","linux","windows","istio","serverless","custom","docker_stig","openshift_master","openshift_worker","application_control_linux","gke_worker","image_malware","host_malware"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}],"enabled":"true"},"clusters":["string"],"repoTag":null,"tags":[{"digest":"string","id":"string","registry":"string","repo":"string","tag":"string"}],"trustResult":{"groups":[{"_id":"string","disabled":true,"images":["string"],"layers":["string"],"modified":"2023-09-08T04:01:49.951Z","name":"string","notes":"string","owner":"string","previousName":"string"}],"hostsStatuses":[{"host":"string","status":"trusted"}]},"repoDigests":[],"creationTime":"0001-01-01T00:00:00Z","pushTime":"0001-01-01T00:00:00Z","vulnerabilities":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.951Z","exploit":["","exploit-db","exploit-windows","cisa-kev"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db","exploit-windows","cisa-kev"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI","HIPAA","NIST SP 800-190","GDPR","DISA STIG"]],"text":"string","title":"string","twistlock":true,"type":["container","image","host_config","daemon_config","daemon_config_files","security_operations","k8s_master","k8s_worker","k8s_federation","linux","windows","istio","serverless","custom","docker_stig","openshift_master","openshift_worker","application_control_linux","gke_worker","image_malware","host_malware"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}],"vulnerabilitiesCount":0,"complianceIssues":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":"5.2","description":"string","discovered":"2023-09-08T04:01:49.949Z","exploit":["","exploit-db","exploit-windows","cisa-kev"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db","exploit-windows","cisa-kev"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI","HIPAA","NIST SP 800-190","GDPR","DISA STIG"]],"text":"string","title":"string","twistlock":true,"type":["container","image","host_config","daemon_config","daemon_config_files","security_operations","k8s_master","k8s_worker","k8s_federation","linux","windows","istio","serverless","custom","docker_stig","openshift_master","openshift_worker","application_control_linux","gke_worker","image_malware","host_malware"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}],"complianceIssuesCount":4,"vulnerabilityDistribution":{"critical":0,"high":0,"medium":0,"low":0,"total":0},"complianceDistribution":{"critical":4,"high":0,"medium":0,"low":0,"total":4},"vulnerabilityRiskScore":0,"complianceRiskScore":4000000,"riskFactors":{},"installedProducts":{"agentless":true,"apache":"string","awsCloud":true,"clusterType":["AKS","ECS","EKS","GKE","Kubernetes"],"crio":true,"docker":"string","dockerEnterprise":true,"hasPackageManager":true,"k8sApiServer":true,"k8sControllerManager":true,"k8sEtcd":true,"k8sFederationApiServer":true,"k8sFederationControllerManager":true,"k8sKubelet":true,"k8sProxy":true,"k8sScheduler":true,"kubernetes":"string","managedClusterVersion":"string","openshift":true,"openshiftVersion":"string","osDistro":"Windows","serverless":true,"swarmManager":true,"swarmNode":true},"firstScanTime":"2023-08-11T06:53:57.456Z","history":[{"baseLayer":true,"created":0,"emptyLayer":true,"id":"string","instruction":"string","sizeBytes":0,"tags":["string"],"vulnerabilities":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.950Z","exploit":["","exploit-db","exploit-windows","cisa-kev"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db","exploit-windows","cisa-kev"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI","HIPAA","NIST SP 800-190","GDPR","DISA STIG"]],"text":"string","title":"string","twistlock":true,"type":["container","image","host_config","daemon_config","daemon_config_files","security_operations","k8s_master","k8s_worker","k8s_federation","linux","windows","istio","serverless","custom","docker_stig","openshift_master","openshift_worker","application_control_linux","gke_worker","image_malware","host_malware"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}]}],"hostDevices":[{"ip":"0.0.0.0","name":"string"}],"hosts":{},"id":"string","err":"","collections":["All"],"instances":[{"host":"string","image":"string","modified":"2023-09-08T04:01:49.951Z","registry":"string","repo":"string","tag":"string"}],"scanID":0,"trustStatus":"","externalLabels":[{"key":"string","sourceName":"string","sourceType":["namespace","deployment","aws","azure","gcp","oci"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"firewallProtection":{"enabled":false,"supported":false,"outOfBandMode":"Observation","ports":[0],"tlsPorts":[0],"unprotectedProcesses":[{"port":0,"process":"string","tls":true}]},"applications":[{"installedFromPackage":true,"knownVulnerabilities":0,"layerTime":0,"name":"string","path":"string","service":true,"version":"string"}],"appEmbedded":false,"wildFireUsage":null,"agentless":false,"malwareAnalyzedTime":"0001-01-01T00:00:00Z"} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-host.log-expected.json b/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-host.log-expected.json new file mode 100644 index 00000000000..fa329e96891 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/_dev/test/pipeline/test-host.log-expected.json @@ -0,0 +1,992 @@ +{ + "expected": [ + { + "cloud": { + "account": { + "id": "Non-onboarded cloud accounts" + }, + "instance": { + "id": "string", + "name": "string" + }, + "machine": { + "type": "string" + }, + "provider": [ + "aws" + ], + "region": "string" + }, + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "host" + ], + "id": "DESKTOP-6PQJAMS", + "kind": "event", + "original": "{\"_id\":\"DESKTOP-6PQJAMS\",\"binaries\":[{\"altered\":true,\"cveCount\":0,\"deps\":[\"string\"],\"fileMode\":0,\"functionLayer\":\"string\",\"md5\":\"string\",\"missingPkg\":true,\"name\":\"string\",\"path\":\"string\",\"pkgRootDir\":\"string\",\"services\":[\"string\"],\"version\":\"string\"}],\"startupBinaries\":[{\"altered\":true,\"cveCount\":0,\"deps\":[\"string\"],\"fileMode\":0,\"functionLayer\":\"string\",\"md5\":\"string\",\"missingPkg\":true,\"name\":\"string\",\"path\":\"string\",\"pkgRootDir\":\"string\",\"services\":[\"string\"],\"version\":\"string\"}],\"cloudMetadata\":{\"accountID\":\"Non-onboarded cloud accounts\",\"awsExecutionEnv\":\"string\",\"image\":\"string\",\"labels\":[{\"key\":\"string\",\"sourceName\":\"string\",\"sourceType\":[\"namespace\",\"deployment\",\"aws\",\"azure\",\"gcp\",\"oci\"],\"timestamp\":\"2023-09-08T04:01:49.949Z\",\"value\":\"string\"}],\"name\":\"string\",\"provider\":[\"aws\"],\"region\":\"string\",\"resourceID\":\"string\",\"resourceURL\":\"string\",\"type\":\"string\",\"vmID\":\"string\",\"vmImageID\":\"string\"},\"type\":\"host\",\"hostname\":\"DESKTOP-6PQJAMS\",\"scanTime\":\"2023-08-23T11:48:41.803Z\",\"Secrets\":[],\"osDistro\":\"windows\",\"osDistroVersion\":\"string\",\"osDistroRelease\":\"Windows\",\"distro\":\"Microsoft Windows [Version 10.0.19045.2006]\",\"packageManager\":true,\"packages\":[{\"pkgs\":[{\"binaryIdx\":[0],\"binaryPkgs\":[\"string\"],\"cveCount\":0,\"defaultGem\":true,\"files\":[{\"md5\":\"string\",\"path\":\"string\",\"sha1\":\"string\",\"sha256\":\"string\"}],\"functionLayer\":\"string\",\"goPkg\":true,\"jarIdentifier\":\"string\",\"layerTime\":0,\"license\":\"string\",\"name\":\"string\",\"osPackage\":true,\"path\":\"string\",\"version\":\"string\"}],\"pkgsType\":\"nodejs\"}],\"isARM64\":false,\"packageCorrelationDone\":true,\"redHatNonRPMImage\":false,\"image\":{\"created\":\"0001-01-01T00:00:00Z\",\"entrypoint\":[\"string\"],\"env\":[\"string\"],\"healthcheck\":true,\"history\":[{\"baseLayer\":true,\"created\":0,\"emptyLayer\":true,\"id\":\"string\",\"instruction\":\"string\",\"sizeBytes\":0,\"tags\":[\"string\"],\"vulnerabilities\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.950Z\",\"exploit\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\",\"HIPAA\",\"NIST SP 800-190\",\"GDPR\",\"DISA STIG\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\",\"image\",\"host_config\",\"daemon_config\",\"daemon_config_files\",\"security_operations\",\"k8s_master\",\"k8s_worker\",\"k8s_federation\",\"linux\",\"windows\",\"istio\",\"serverless\",\"custom\",\"docker_stig\",\"openshift_master\",\"openshift_worker\",\"application_control_linux\",\"gke_worker\",\"image_malware\",\"host_malware\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}]}],\"id\":\"string\",\"labels\":{},\"layers\":[\"string\"],\"os\":\"string\",\"repoDigest\":[\"string\"],\"repoTags\":[\"string\"],\"user\":\"string\",\"workingDir\":\"string\"},\"allCompliance\":{\"compliance\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.949Z\",\"exploit\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"]},{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\",\"HIPAA\",\"NIST SP 800-190\",\"GDPR\",\"DISA STIG\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\",\"image\",\"host_config\",\"daemon_config\",\"daemon_config_files\",\"security_operations\",\"k8s_master\",\"k8s_worker\",\"k8s_federation\",\"linux\",\"windows\",\"istio\",\"serverless\",\"custom\",\"docker_stig\",\"openshift_master\",\"openshift_worker\",\"application_control_linux\",\"gke_worker\",\"image_malware\",\"host_malware\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}],\"enabled\":\"true\"},\"clusters\":[\"string\"],\"repoTag\":null,\"tags\":[{\"digest\":\"string\",\"id\":\"string\",\"registry\":\"string\",\"repo\":\"string\",\"tag\":\"string\"}],\"trustResult\":{\"groups\":[{\"_id\":\"string\",\"disabled\":true,\"images\":[\"string\"],\"layers\":[\"string\"],\"modified\":\"2023-09-08T04:01:49.951Z\",\"name\":\"string\",\"notes\":\"string\",\"owner\":\"string\",\"previousName\":\"string\"}],\"hostsStatuses\":[{\"host\":\"string\",\"status\":\"trusted\"}]},\"repoDigests\":[],\"creationTime\":\"0001-01-01T00:00:00Z\",\"pushTime\":\"0001-01-01T00:00:00Z\",\"vulnerabilities\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.951Z\",\"exploit\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\",\"HIPAA\",\"NIST SP 800-190\",\"GDPR\",\"DISA STIG\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\",\"image\",\"host_config\",\"daemon_config\",\"daemon_config_files\",\"security_operations\",\"k8s_master\",\"k8s_worker\",\"k8s_federation\",\"linux\",\"windows\",\"istio\",\"serverless\",\"custom\",\"docker_stig\",\"openshift_master\",\"openshift_worker\",\"application_control_linux\",\"gke_worker\",\"image_malware\",\"host_malware\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}],\"vulnerabilitiesCount\":0,\"complianceIssues\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":\"5.2\",\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.949Z\",\"exploit\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\",\"HIPAA\",\"NIST SP 800-190\",\"GDPR\",\"DISA STIG\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\",\"image\",\"host_config\",\"daemon_config\",\"daemon_config_files\",\"security_operations\",\"k8s_master\",\"k8s_worker\",\"k8s_federation\",\"linux\",\"windows\",\"istio\",\"serverless\",\"custom\",\"docker_stig\",\"openshift_master\",\"openshift_worker\",\"application_control_linux\",\"gke_worker\",\"image_malware\",\"host_malware\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}],\"complianceIssuesCount\":4,\"vulnerabilityDistribution\":{\"critical\":0,\"high\":0,\"medium\":0,\"low\":0,\"total\":0},\"complianceDistribution\":{\"critical\":4,\"high\":0,\"medium\":0,\"low\":0,\"total\":4},\"vulnerabilityRiskScore\":0,\"complianceRiskScore\":4000000,\"riskFactors\":{},\"installedProducts\":{\"agentless\":true,\"apache\":\"string\",\"awsCloud\":true,\"clusterType\":[\"AKS\",\"ECS\",\"EKS\",\"GKE\",\"Kubernetes\"],\"crio\":true,\"docker\":\"string\",\"dockerEnterprise\":true,\"hasPackageManager\":true,\"k8sApiServer\":true,\"k8sControllerManager\":true,\"k8sEtcd\":true,\"k8sFederationApiServer\":true,\"k8sFederationControllerManager\":true,\"k8sKubelet\":true,\"k8sProxy\":true,\"k8sScheduler\":true,\"kubernetes\":\"string\",\"managedClusterVersion\":\"string\",\"openshift\":true,\"openshiftVersion\":\"string\",\"osDistro\":\"Windows\",\"serverless\":true,\"swarmManager\":true,\"swarmNode\":true},\"firstScanTime\":\"2023-08-11T06:53:57.456Z\",\"history\":[{\"baseLayer\":true,\"created\":0,\"emptyLayer\":true,\"id\":\"string\",\"instruction\":\"string\",\"sizeBytes\":0,\"tags\":[\"string\"],\"vulnerabilities\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.950Z\",\"exploit\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\",\"exploit-windows\",\"cisa-kev\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\",\"HIPAA\",\"NIST SP 800-190\",\"GDPR\",\"DISA STIG\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\",\"image\",\"host_config\",\"daemon_config\",\"daemon_config_files\",\"security_operations\",\"k8s_master\",\"k8s_worker\",\"k8s_federation\",\"linux\",\"windows\",\"istio\",\"serverless\",\"custom\",\"docker_stig\",\"openshift_master\",\"openshift_worker\",\"application_control_linux\",\"gke_worker\",\"image_malware\",\"host_malware\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}]}],\"hostDevices\":[{\"ip\":\"0.0.0.0\",\"name\":\"string\"}],\"hosts\":{},\"id\":\"string\",\"err\":\"\",\"collections\":[\"All\"],\"instances\":[{\"host\":\"string\",\"image\":\"string\",\"modified\":\"2023-09-08T04:01:49.951Z\",\"registry\":\"string\",\"repo\":\"string\",\"tag\":\"string\"}],\"scanID\":0,\"trustStatus\":\"\",\"externalLabels\":[{\"key\":\"string\",\"sourceName\":\"string\",\"sourceType\":[\"namespace\",\"deployment\",\"aws\",\"azure\",\"gcp\",\"oci\"],\"timestamp\":\"2023-09-08T04:01:49.949Z\",\"value\":\"string\"}],\"files\":[{\"md5\":\"string\",\"path\":\"string\",\"sha1\":\"string\",\"sha256\":\"string\"}],\"firewallProtection\":{\"enabled\":false,\"supported\":false,\"outOfBandMode\":\"Observation\",\"ports\":[0],\"tlsPorts\":[0],\"unprotectedProcesses\":[{\"port\":0,\"process\":\"string\",\"tls\":true}]},\"applications\":[{\"installedFromPackage\":true,\"knownVulnerabilities\":0,\"layerTime\":0,\"name\":\"string\",\"path\":\"string\",\"service\":true,\"version\":\"string\"}],\"appEmbedded\":false,\"wildFireUsage\":null,\"agentless\":false,\"malwareAnalyzedTime\":\"0001-01-01T00:00:00Z\"}", + "start": "0001-01-01T00:00:00.000Z", + "type": [ + "info" + ] + }, + "file": { + "hash": { + "md5": [ + "string" + ], + "sha1": [ + "string" + ], + "sha256": [ + "string" + ] + }, + "path": [ + "string" + ] + }, + "group": { + "name": [ + "string" + ] + }, + "host": { + "hostname": "DESKTOP-6PQJAMS", + "ip": [ + "0.0.0.0" + ], + "type": "host" + }, + "os": { + "family": "windows", + "name": "Windows", + "version": "string" + }, + "package": { + "license": [ + "string" + ], + "name": [ + "string" + ], + "path": [ + "string" + ], + "type": [ + "nodejs" + ], + "version": [ + "string" + ] + }, + "prisma_cloud": { + "host": { + "_id": "DESKTOP-6PQJAMS", + "agentless": false, + "all_compliance": { + "data": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0.0, + "description": "string", + "discovered": "2023-09-08T04:01:49.949Z", + "exploit": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ] + }, + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI", + "HIPAA", + "NIST SP 800-190", + "GDPR", + "DISA STIG" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container", + "image", + "host_config", + "daemon_config", + "daemon_config_files", + "security_operations", + "k8s_master", + "k8s_worker", + "k8s_federation", + "linux", + "windows", + "istio", + "serverless", + "custom", + "docker_stig", + "openshift_master", + "openshift_worker", + "application_control_linux", + "gke_worker", + "image_malware", + "host_malware" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ], + "enabled": true + }, + "app_embedded": false, + "applications": [ + { + "installed_from_package": true, + "known_vulnerabilities": 0, + "layer_time": "1970-01-01T00:00:00.000Z", + "name": "string", + "path": "string", + "service": true, + "version": "string" + } + ], + "binaries": [ + { + "altered": true, + "cve_count": 0, + "deps": [ + "string" + ], + "file_mode": 0, + "function_layer": "string", + "md5": "string", + "missing_pkg": true, + "name": "string", + "path": "string", + "pkg_root_dir": "string", + "services": [ + "string" + ], + "version": "string" + } + ], + "cloud_metadata": { + "account_id": "Non-onboarded cloud accounts", + "aws_execution_env": "string", + "image": "string", + "labels": [ + { + "key": "string", + "source": { + "name": "string", + "type": [ + "namespace", + "deployment", + "aws", + "azure", + "gcp", + "oci" + ] + }, + "timestamp": "2023-09-08T04:01:49.949Z", + "value": "string" + } + ], + "name": "string", + "provider": [ + "aws" + ], + "region": "string", + "resource": { + "id": "string", + "url": "string" + }, + "type": "string", + "vm": { + "id": "string", + "image_id": "string" + } + }, + "clusters": [ + "string" + ], + "collections": [ + "All" + ], + "compliance_distribution": { + "critical": 4, + "high": 0, + "low": 0, + "medium": 0, + "total": 4 + }, + "compliance_issues": { + "count": 4, + "data": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 5.2, + "description": "string", + "discovered": "2023-09-08T04:01:49.949Z", + "exploit": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI", + "HIPAA", + "NIST SP 800-190", + "GDPR", + "DISA STIG" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container", + "image", + "host_config", + "daemon_config", + "daemon_config_files", + "security_operations", + "k8s_master", + "k8s_worker", + "k8s_federation", + "linux", + "windows", + "istio", + "serverless", + "custom", + "docker_stig", + "openshift_master", + "openshift_worker", + "application_control_linux", + "gke_worker", + "image_malware", + "host_malware" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ] + }, + "compliance_risk_score": 4000000.0, + "creation_time": "0001-01-01T00:00:00.000Z", + "devices": [ + { + "ip": "0.0.0.0", + "name": "string" + } + ], + "distro": "Microsoft Windows [Version 10.0.19045.2006]", + "external_labels": [ + { + "key": "string", + "source": { + "name": "string", + "type": [ + "namespace", + "deployment", + "aws", + "azure", + "gcp", + "oci" + ] + }, + "timestamp": "2023-09-08T04:01:49.949Z", + "value": "string" + } + ], + "files": [ + { + "md5": "string", + "path": "string", + "sha1": "string", + "sha256": "string" + } + ], + "firewall_protection": { + "enabled": false, + "out_of_band_mode": "Observation", + "ports": [ + 0 + ], + "supported": false, + "tls_ports": [ + 0 + ], + "unprotected_processes": [ + { + "port": 0, + "process": "string", + "tls": true + } + ] + }, + "first_scan_time": "2023-08-11T06:53:57.456Z", + "history": [ + { + "base_layer": true, + "created": "1970-01-01T00:00:00.000Z", + "empty_layer": true, + "id": "string", + "instruction": "string", + "size_bytes": 0, + "tags": [ + "string" + ], + "vulnerabilities": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0.0, + "description": "string", + "discovered": "2023-09-08T04:01:49.950Z", + "exploit": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI", + "HIPAA", + "NIST SP 800-190", + "GDPR", + "DISA STIG" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container", + "image", + "host_config", + "daemon_config", + "daemon_config_files", + "security_operations", + "k8s_master", + "k8s_worker", + "k8s_federation", + "linux", + "windows", + "istio", + "serverless", + "custom", + "docker_stig", + "openshift_master", + "openshift_worker", + "application_control_linux", + "gke_worker", + "image_malware", + "host_malware" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ] + } + ], + "hostname": "DESKTOP-6PQJAMS", + "id": "string", + "image": { + "created": "0001-01-01T00:00:00.000Z", + "entrypoint": [ + "string" + ], + "env": [ + "string" + ], + "healthcheck": true, + "history": [ + { + "base_layer": true, + "created": "1970-01-01T00:00:00.000Z", + "empty_layer": true, + "id": "string", + "instruction": "string", + "size_bytes": 0, + "tags": [ + "string" + ], + "vulnerabilities": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0.0, + "description": "string", + "discovered": "2023-09-08T04:01:49.950Z", + "exploit": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI", + "HIPAA", + "NIST SP 800-190", + "GDPR", + "DISA STIG" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container", + "image", + "host_config", + "daemon_config", + "daemon_config_files", + "security_operations", + "k8s_master", + "k8s_worker", + "k8s_federation", + "linux", + "windows", + "istio", + "serverless", + "custom", + "docker_stig", + "openshift_master", + "openshift_worker", + "application_control_linux", + "gke_worker", + "image_malware", + "host_malware" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ] + } + ], + "id": "string", + "layers": [ + "string" + ], + "os": "string", + "repo": { + "digest": [ + "string" + ], + "tags": [ + "string" + ] + }, + "user": "string", + "working_dir": "string" + }, + "installed_products": { + "agentless": true, + "apache": "string", + "aws_cloud": true, + "cluster_type": [ + "AKS", + "ECS", + "EKS", + "GKE", + "Kubernetes" + ], + "crio": true, + "docker": "string", + "docker_enterprise": true, + "has_package_manager": true, + "k8s_api_server": true, + "k8s_controller_manager": true, + "k8s_etcd": true, + "k8s_federation_api_server": true, + "k8s_federation_controller_manager": true, + "k8s_kubelet": true, + "k8s_proxy": true, + "k8s_scheduler": true, + "kubernetes": "string", + "managed_cluster_version": "string", + "openshift": true, + "openshift_version": "string", + "os_distro": "Windows", + "serverless": true, + "swarm": { + "manager": true, + "node": true + } + }, + "instances": [ + { + "host": "string", + "image": "string", + "modified": "2023-09-08T04:01:49.951Z", + "registry": "string", + "repo": "string", + "tag": "string" + } + ], + "is_arm64": false, + "malware_analyzed_time": "0001-01-01T00:00:00.000Z", + "os_distro": { + "release": "Windows", + "value": "windows", + "version": "string" + }, + "package": { + "correlation_done": true, + "manager": true + }, + "packages": [ + { + "pkgs": [ + { + "binary_idx": [ + 0 + ], + "binary_pkgs": [ + "string" + ], + "cve_count": 0, + "default_gem": true, + "files": [ + { + "md5": "string", + "path": "string", + "sha1": "string", + "sha256": "string" + } + ], + "function_layer": "string", + "go_pkg": true, + "jar_identifier": "string", + "layer_time": "1970-01-01T00:00:00.000Z", + "license": "string", + "name": "string", + "os_package": true, + "path": "string", + "version": "string" + } + ], + "pkgs_type": "nodejs" + } + ], + "push_time": "0001-01-01T00:00:00.000Z", + "red_hat_non_rpm_image": false, + "scan": { + "time": "2023-08-23T11:48:41.803Z" + }, + "startup_binaries": [ + { + "altered": true, + "cve_count": 0, + "deps": [ + "string" + ], + "file_mode": 0, + "function_layer": "string", + "md5": "string", + "missing_pkg": true, + "name": "string", + "path": "string", + "pkg_root_dir": "string", + "services": [ + "string" + ], + "version": "string" + } + ], + "tags": [ + { + "digest": "string", + "id": "string", + "registry": "string", + "repo": "string", + "tag": "string" + } + ], + "trust_result": { + "groups": [ + { + "_id": "string", + "disabled": true, + "images": [ + "string" + ], + "layers": [ + "string" + ], + "modified": "2023-09-08T04:01:49.951Z", + "name": "string", + "notes": "string", + "owner": "string", + "previous_name": "string" + } + ], + "hosts_statuses": [ + { + "host": "string", + "status": "trusted" + } + ] + }, + "type": "host", + "vulnerabilities": { + "count": 0, + "data": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0.0, + "description": "string", + "discovered": "2023-09-08T04:01:49.951Z", + "exploit": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db", + "exploit-windows", + "cisa-kev" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI", + "HIPAA", + "NIST SP 800-190", + "GDPR", + "DISA STIG" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container", + "image", + "host_config", + "daemon_config", + "daemon_config_files", + "security_operations", + "k8s_master", + "k8s_worker", + "k8s_federation", + "linux", + "windows", + "istio", + "serverless", + "custom", + "docker_stig", + "openshift_master", + "openshift_worker", + "application_control_linux", + "gke_worker", + "image_malware", + "host_malware" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ] + }, + "vulnerability": { + "distribution": { + "critical": 0, + "high": 0, + "low": 0, + "medium": 0, + "total": 0 + }, + "risk_score": 0 + } + } + }, + "related": { + "hash": [ + "string" + ], + "hosts": [ + "string", + "DESKTOP-6PQJAMS" + ], + "ip": [ + "0.0.0.0" + ] + }, + "rule": { + "author": [ + "string" + ], + "name": [ + "string" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "vulnerability": { + "description": [ + "string" + ], + "id": [ + "string" + ], + "severity": [ + "string" + ] + } + } + ] +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/host/_dev/test/system/test-input-config.yml b/packages/prisma_cloud/data_stream/host/_dev/test/system/test-input-config.yml new file mode 100644 index 00000000000..a2af63f2fcb --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/_dev/test/system/test-input-config.yml @@ -0,0 +1,12 @@ +input: cel +service: prisma_cloud +vars: + username: xxxx + password: xxxx +data_stream: + vars: + url: http://{{Hostname}}:{{Port}} + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/host/_dev/test/system/test-tcp-config.yml b/packages/prisma_cloud/data_stream/host/_dev/test/system/test-tcp-config.yml new file mode 100644 index 00000000000..097079a9f28 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/_dev/test/system/test-tcp-config.yml @@ -0,0 +1,11 @@ +service: prisma_cloud-host-tcp +service_notify_signal: SIGHUP +input: tcp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9508 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/host/_dev/test/system/test-udp-config.yml b/packages/prisma_cloud/data_stream/host/_dev/test/system/test-udp-config.yml new file mode 100644 index 00000000000..5c0f0114a28 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/_dev/test/system/test-udp-config.yml @@ -0,0 +1,11 @@ +service: prisma_cloud-host-udp +service_notify_signal: SIGHUP +input: udp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9509 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/host/agent/stream/input.yml.hbs b/packages/prisma_cloud/data_stream/host/agent/stream/input.yml.hbs new file mode 100644 index 00000000000..e1526419d89 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/agent/stream/input.yml.hbs @@ -0,0 +1,100 @@ +config_version: 2 +interval: {{interval}} +{{#if enable_request_tracer}} +resource.tracer.filename: "../../logs/httpjson/http-request-trace-*.ndjson" +{{/if}} +{{#if proxy_url}} +resource.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +resource.url: {{url}} +state: + user: {{username}} + password: {{password}} + batch_size: {{batch_size}} + offset: {{offset}} + want_more: false +redact: + fields: + - password +program: | + ( + state.with(has(state.want_more) && !(state.want_more) + ? + post_request( + state.url + "/authenticate", + "application/json", + {"username":state.user,"password":state.password}.encode_json() + ).do_request().as(resp, bytes(resp.Body).decode_json().as(body, { + "access_token": body.token, + })) + : + {} + ).as(state, + request("GET", + ( + has(state.cursor) && has(state.cursor.new_offset) && state.cursor.new_offset != null + ? + state.url + "/hosts?limit=" + string(state.batch_size) + "&offset=" + string(state.cursor.new_offset) + : + state.url + "/hosts?limit=" + string(state.batch_size) + "&offset=" + string(state.offset) + )).with({ + "Header":{ + "Authorization": ["Bearer " + state.access_token], + } + }).do_request().as(resp, bytes(resp.Body).decode_json().as(inner_body, { + "events": (inner_body != null ? + inner_body.map(e, { + "message": e.encode_json(), + }) + : + [] + ), + "url": state.url, + "want_more": inner_body != null && inner_body.size() > 0, + "user": state.user, + "password": state.password, + "batch_size": string(state.batch_size), + "access_token": state.access_token, + "cursor": + { + "new_offset": + ( + has(state.cursor) && has(state.cursor.new_offset) && state.cursor.new_offset != null + ? + ( + inner_body != null && inner_body.size() > 0 + ? + string(int(state.cursor.new_offset) + int(inner_body.size())) + : + state.cursor.new_offset + ) + : + string(int(state.offset) + (inner_body != null ? int(inner_body.size()) : 0)) + ) + }, + })) + ) + ) +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/host/agent/stream/tcp.yml.hbs b/packages/prisma_cloud/data_stream/host/agent/stream/tcp.yml.hbs new file mode 100644 index 00000000000..c29dbc0fc55 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/agent/stream/tcp.yml.hbs @@ -0,0 +1,24 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if tcp_options}} +{{tcp_options}} +{{/if}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/host/agent/stream/udp.yml.hbs b/packages/prisma_cloud/data_stream/host/agent/stream/udp.yml.hbs new file mode 100644 index 00000000000..3e79cc97e77 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/agent/stream/udp.yml.hbs @@ -0,0 +1,21 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if udp_options}} +{{udp_options}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/host/elasticsearch/ingest_pipeline/default.yml b/packages/prisma_cloud/data_stream/host/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..508389bf8e3 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,4193 @@ +--- +description: Pipeline for processing host logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 8.10.0 + - set: + field: event.kind + tag: set_event_kind + value: event + - append: + field: event.category + tag: append_event_category + value: host + - append: + field: event.type + tag: append_event_type + value: info + - rename: + field: message + tag: rename_message + target_field: event.original + ignore_missing: true + - drop: + if: ctx.event?.original != null && ctx.event.original.isEmpty() + - json: + field: event.original + tag: json_event_original + target_field: json + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json._id + tag: rename_id + target_field: prisma_cloud.host._id + ignore_missing: true + - set: + field: event.id + tag: set_id_to_event_id + copy_from: prisma_cloud.host._id + ignore_empty_value: true + - convert: + field: json.agentless + tag: convert_agentless + target_field: prisma_cloud.host.agentless + type: boolean + ignore_missing: true + if: ctx.json?.agentless != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.allCompliance + tag: rename_allCompliance + target_field: prisma_cloud.host.all_compliance + ignore_missing: true + - rename: + field: prisma_cloud.host.all_compliance.compliance + tag: rename_compliance_to_data + target_field: prisma_cloud.host.all_compliance.data + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.applicableRules + tag: rename_prisma_cloud_host_all_compliance_data_applicableRules + target_field: _ingest._value.applicable_rules + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.binaryPkgs + tag: rename_prisma_cloud_host_all_compliance_data_binaryPkgs + target_field: _ingest._value.binary_pkgs + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vecStr + tag: rename_prisma_cloud_host_all_compliance_data_vecStr + target_field: _ingest._value.vec_str + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.block + tag: convert_prisma_cloud_host_all_compliance_data_block + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.block + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cri + tag: convert_prisma_cloud_host_all_compliance_data_cri + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cri + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.custom + tag: convert_prisma_cloud_host_all_compliance_data_custom + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.custom + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.id + tag: append_prisma_cloud_host_all_compliance_data_cve + value: '{{{_ingest._value.cve}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.severity + tag: append_prisma_cloud_host_all_compliance_data_severity + value: '{{{_ingest._value.severity}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cvss + tag: convert_prisma_cloud_host_all_compliance_data_cvss + type: float + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cvss + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.description + tag: append_description_to_vulnerability_description + value: '{{{_ingest._value.description}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cve + - _ingest._value.severity + - _ingest._value.description + tag: remove_prisma_cloud_host_all_compliance_data_fields_ecs_mapped + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.discovered + target_field: _ingest._value.discovered + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.discovered + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.fixDate + target_field: _ingest._value.fix_date + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.fixDate + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.fixLink + tag: rename_prisma_cloud_host_all_compliance_data_fixLink + target_field: _ingest._value.fix_link + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_prisma_cloud_host_all_compliance_data_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.gracePeriodDays + tag: convert_prisma_cloud_host_all_compliance_data_gracePeriodDays + target_field: _ingest._value.grace_period_days + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.gracePeriodDays + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.id + tag: convert_prisma_cloud_host_all_compliance_data_id + type: string + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.id + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.packageName + tag: rename_prisma_cloud_host_all_compliance_data_packageName + target_field: _ingest._value.package.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.packageVersion + tag: rename_prisma_cloud_host_all_compliance_data_packageVersion + target_field: _ingest._value.package.version + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.riskFactors + tag: rename_prisma_cloud_host_all_compliance_data_riskFactors + target_field: _ingest._value.risk_factors + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.published + target_field: _ingest._value.published + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.published + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.twistlock + tag: convert_prisma_cloud_host_all_compliance_data_twistlock + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.twistlock + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vulnTagInfos + tag: rename_prisma_cloud_host_all_compliance_data_vulnTagInfos + target_field: _ingest._value.vuln_tag_infos + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.wildfireMalware + tag: rename_prisma_cloud_host_all_compliance_data_wildfireMalware + target_field: _ingest._value.wild_fire_malware + ignore_missing: true + - foreach: + field: prisma_cloud.host.all_compliance.data + if: ctx.prisma_cloud?.host?.all_compliance?.data instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.fixDate + - _ingest._value.gracePeriodDays + - _ingest._value.layerTime + tag: foreach_remove_prisma_cloud_host_all_compliance_data_fields + ignore_missing: true + - convert: + field: prisma_cloud.host.all_compliance.enabled + tag: convert_prisma_cloud_host_all_compliance_enabled + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.all_compliance?.enabled != '' + on_failure: + - remove: + field: prisma_cloud.host.all_compliance.enabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.appEmbedded + tag: convert_appEmbedded + target_field: prisma_cloud.host.app_embedded + type: boolean + ignore_missing: true + if: ctx.json?.appEmbedded != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.applications + tag: rename_applications + target_field: prisma_cloud.host.applications + ignore_missing: true + - foreach: + field: prisma_cloud.host.applications + if: ctx.prisma_cloud?.host?.applications instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.installedFromPackage + tag: convert_prisma_cloud_host_applications_installedFromPackage + target_field: _ingest._value.installed_from_package + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.installedFromPackage + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.applications + if: ctx.prisma_cloud?.host?.applications instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.knownVulnerabilities + tag: convert_prisma_cloud_host_applications_knownVulnerabilities + target_field: _ingest._value.known_vulnerabilities + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.knownVulnerabilities + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.applications + if: ctx.prisma_cloud?.host?.applications instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.applications + if: ctx.prisma_cloud?.host?.applications instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.service + tag: convert_prisma_cloud_host_applications_service + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.service + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.applications + if: ctx.prisma_cloud?.host?.applications instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.installedFromPackage + - _ingest._value.knownVulnerabilities + - _ingest._value.layerTime + tag: foreach_remove_prisma_cloud_host_applications_fields + ignore_missing: true + - rename: + field: json.baseImage + tag: rename_baseImage + target_field: prisma_cloud.host.base_image + ignore_missing: true + - rename: + field: json.binaries + tag: rename_binaries + target_field: prisma_cloud.host.binaries + ignore_missing: true + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.altered + tag: convert_binaries_altered + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.altered + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cveCount + tag: convert_prisma_cloud_host_binaries_cveCount + target_field: _ingest._value.cve_count + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cveCount + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.fileMode + tag: convert_prisma_cloud_host_binaries_fileMode + target_field: _ingest._value.file_mode + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.fileMode + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_prisma_cloud_host_binaries_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.missingPkg + tag: convert_prisma_cloud_host_binaries_missingPkg + target_field: _ingest._value.missing_pkg + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.missingPkg + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.pkgRootDir + tag: rename_prisma_cloud_host_binaries_pkgRootDir + target_field: _ingest._value.pkg_root_dir + ignore_missing: true + - foreach: + field: prisma_cloud.host.binaries + if: ctx.prisma_cloud?.host?.binaries instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cveCount + - _ingest._value.fileMode + - _ingest._value.missingPkg + tag: foreach_remove_prisma_cloud_host_binaries_fields + ignore_missing: true + - rename: + field: json.cloudMetadata + tag: rename_cloudMetadata + target_field: prisma_cloud.host.cloud_metadata + ignore_missing: true + - rename: + field: prisma_cloud.host.cloud_metadata.accountID + tag: rename_accountID + target_field: prisma_cloud.host.cloud_metadata.account_id + ignore_missing: true + - set: + field: cloud.account.id + tag: set_account_id + copy_from: prisma_cloud.host.cloud_metadata.account_id + ignore_empty_value: true + - rename: + field: prisma_cloud.host.cloud_metadata.awsExecutionEnv + tag: rename_awsExecutionEnv + target_field: prisma_cloud.host.cloud_metadata.aws_execution_env + ignore_missing: true + - rename: + field: json.cloudMetadata.labels + tag: rename_cloudMetadata_labels + target_field: prisma_cloud.host.cloud_metadata.labels + ignore_missing: true + - foreach: + field: prisma_cloud.host.cloud_metadata.labels + if: ctx.prisma_cloud?.host?.cloud_metadata?.labels instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sourceName + tag: rename_cloud_metadata_labels_sourceName + target_field: _ingest._value.source.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.cloud_metadata.labels + if: ctx.prisma_cloud?.host?.cloud_metadata?.labels instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sourceType + tag: foreach_cloud_metadata_labels_sourceType + target_field: _ingest._value.source.type + ignore_missing: true + - foreach: + field: prisma_cloud.host.cloud_metadata.labels + if: ctx.prisma_cloud?.host?.cloud_metadata?.labels instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.timestamp + target_field: _ingest._value.timestamp + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.timestamp + ignore_missing: true + - set: + field: cloud.instance.name + tag: set_cloud_instance_name + copy_from: prisma_cloud.host.cloud_metadata.name + ignore_empty_value: true + - set: + field: cloud.provider + tag: set_cloud_provider + copy_from: prisma_cloud.host.cloud_metadata.provider + ignore_empty_value: true + - set: + field: cloud.region + tag: set_cloud_region + copy_from: prisma_cloud.host.cloud_metadata.region + ignore_empty_value: true + - rename: + field: prisma_cloud.host.cloud_metadata.resourceID + tag: rename_cloud_metadata_resourceID + target_field: prisma_cloud.host.cloud_metadata.resource.id + ignore_missing: true + - set: + field: cloud.instance.id + tag: set_cloud_instance_id + copy_from: prisma_cloud.host.cloud_metadata.resource.id + ignore_empty_value: true + - rename: + field: prisma_cloud.host.cloud_metadata.resourceURL + tag: rename_cloud_metadata_resourceURL + target_field: prisma_cloud.host.cloud_metadata.resource.url + ignore_missing: true + - set: + field: cloud.machine.type + tag: set_cloud_machine_type + copy_from: prisma_cloud.host.cloud_metadata.type + ignore_empty_value: true + - rename: + field: prisma_cloud.host.cloud_metadata.vmID + tag: rename_cloud_metadata_vmID + target_field: prisma_cloud.host.cloud_metadata.vm.id + ignore_missing: true + - rename: + field: prisma_cloud.host.cloud_metadata.vmImageID + tag: rename_cloud_metadata_vmImageID + target_field: prisma_cloud.host.cloud_metadata.vm.image_id + ignore_missing: true + - rename: + field: json.clusters + tag: rename_clusters + target_field: prisma_cloud.host.clusters + ignore_missing: true + - rename: + field: json.clusterType + tag: rename_clusterType + target_field: prisma_cloud.host.cluster_type + ignore_missing: true + - rename: + field: json.collections + tag: rename_collections + target_field: prisma_cloud.host.collections + ignore_missing: true + - convert: + field: json.complianceDistribution.critical + tag: convert_critical + target_field: prisma_cloud.host.compliance_distribution.critical + type: long + ignore_missing: true + if: ctx.json?.complianceDistribution?.critical != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.complianceDistribution.high + tag: convert_high + target_field: prisma_cloud.host.compliance_distribution.high + type: long + ignore_missing: true + if: ctx.json?.complianceDistribution?.high != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.complianceDistribution.low + tag: convert_low + target_field: prisma_cloud.host.compliance_distribution.low + type: long + ignore_missing: true + if: ctx.json?.complianceDistribution?.low != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.complianceDistribution.medium + tag: convert_medium + target_field: prisma_cloud.host.compliance_distribution.medium + type: long + ignore_missing: true + if: ctx.json?.complianceDistribution?.medium != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.complianceDistribution.total + tag: convert_total + target_field: prisma_cloud.host.compliance_distribution.total + type: long + ignore_missing: true + if: ctx.json?.complianceDistribution?.total != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.complianceIssues + tag: rename_complianceIssues + target_field: prisma_cloud.host.compliance_issues.data + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.applicableRules + tag: rename_prisma_cloud_host_compliance_issues_data_applicableRules + target_field: _ingest._value.applicable_rules + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.binaryPkgs + tag: rename_prisma_cloud_host_compliance_issues_data_binaryPkgs + target_field: _ingest._value.binary_pkgs + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vecStr + tag: rename_prisma_cloud_host_compliance_issues_data_vecStr + target_field: _ingest._value.vec_str + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.block + tag: convert_prisma_cloud_host_compliance_issues_data_block + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.block + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cri + tag: convert_prisma_cloud_host_compliance_issues_data_cri + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cri + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.custom + tag: convert_prisma_cloud_host_compliance_issues_data_custom + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.custom + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.id + tag: append_prisma_cloud_host_compliance_issues_data_cve_to_vulnerability_id + value: '{{{_ingest._value.cve}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.severity + value: '{{{_ingest._value.severity}}}' + tag: append_compliance_issues_serevity + allow_duplicates: false + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cvss + tag: convert_cvss_prisma_cloud_host_compliance_issues_data + type: float + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cvss + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.description + tag: append_prisma_cloud_host_compliance_issues_data_description + value: '{{{_ingest._value.description}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cve + - _ingest._value.severity + - _ingest._value.description + ignore_missing: true + tag: remove_compliance_issues_ecs_duplicated_fields + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.discovered + target_field: _ingest._value.discovered + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.discovered + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.fixDate + target_field: _ingest._value.fix_date + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.fixDate + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.fixLink + tag: rename_prisma_cloud_host_compliance_issues_data_fixLink + target_field: _ingest._value.fix_link + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_prisma_cloud_host_compliance_issues_data_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.gracePeriodDays + tag: convert_gracePeriodDays_prisma_cloud_host_compliance_issues_data + target_field: _ingest._value.grace_period_days + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.gracePeriodDays + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.id + tag: convert_prisma_cloud_host_compliance_issues_data_id + type: string + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.id + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.packageName + tag: rename_packageName_prisma_cloud_host_compliance_issues_data + target_field: _ingest._value.package.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.packageVersion + tag: rename_packageVersion_prisma_cloud_host_compliance_issues_data + target_field: _ingest._value.package.version + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.riskFactors + tag: rename_riskFactors_prisma_cloud_host_compliance_issues_data + target_field: _ingest._value.risk_factors + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.published + target_field: _ingest._value.published + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.published + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.twistlock + tag: convert_twistlock_prisma_cloud_host_compliance_issues_data + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.twistlock + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vulnTagInfos + tag: rename_vulnTagInfos_prisma_cloud_host_compliance_issues_data + target_field: _ingest._value.vuln_tag_infos + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.wildfireMalware + tag: rename_wildfireMalware_prisma_cloud_host_compliance_issues_data + target_field: _ingest._value.wild_fire_malware + ignore_missing: true + - foreach: + field: prisma_cloud.host.compliance_issues.data + if: ctx.prisma_cloud?.host?.compliance_issues?.data instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.fixDate + - _ingest._value.gracePeriodDays + - _ingest._value.layerTime + ignore_missing: true + tag: foreach_prisma_cloud_host_compliance_issues_data_remove + - convert: + field: json.complianceIssuesCount + tag: convert_complianceIssueCount + target_field: prisma_cloud.host.compliance_issues.count + type: long + ignore_missing: true + if: ctx.json?.complianceIssuesCount != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.complianceRiskScore + tag: convert_complianceRiskScore + target_field: prisma_cloud.host.compliance_risk_score + type: float + ignore_missing: true + if: ctx.json?.complianceRiskScore != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.creationTime + tag: date_creationTime + target_field: prisma_cloud.host.creation_time + formats: + - ISO8601 + if: ctx.json?.creationTime != null && ctx.json.creationTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.start + tag: set_creation_time + copy_from: prisma_cloud.host.creation_time + ignore_empty_value: true + - rename: + field: json.distro + tag: rename_distro + target_field: prisma_cloud.host.distro + ignore_missing: true + - rename: + field: json.ecsClusterName + tag: rename_ecsClusterName + target_field: prisma_cloud.host.ecs_cluster_name + ignore_missing: true + - rename: + field: json.err + tag: rename_err + target_field: prisma_cloud.host.err + ignore_missing: true + - rename: + field: json.externalLabels + tag: rename_externalLabels + target_field: prisma_cloud.host.external_labels + ignore_missing: true + - foreach: + field: prisma_cloud.host.external_labels + if: ctx.prisma_cloud?.host?.external_labels instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sourceName + tag: rename_externalLabels_sourceName + target_field: _ingest._value.source.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.external_labels + if: ctx.prisma_cloud?.host?.external_labels instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.sourceType + tag: rename_sourceType_externalLabels + target_field: _ingest._value.source.type + ignore_missing: true + - foreach: + field: prisma_cloud.host.external_labels + if: ctx.prisma_cloud?.host?.external_labels instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.timestamp + target_field: _ingest._value.timestamp + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.timestamp + ignore_missing: true + - rename: + field: json.files + tag: rename_files + target_field: prisma_cloud.host.files + ignore_missing: true + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: file.hash.md5 + tag: append_file_md5 + value: '{{{_ingest._value.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: related.hash + tag: append_related_hash_md5 + value: '{{{_ingest._value.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: file.path + tag: append_file_path + value: '{{{_ingest._value.path}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: file.hash.sha1 + tag: append_file_hash_sha1 + value: '{{{_ingest._value.sha1}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: related.hash + tag: append_related_hash_sha1 + value: '{{{_ingest._value.sha1}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: file.hash.sha256 + tag: append_file_sha256 + value: '{{{_ingest._value.sha256}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List + ignore_failure: true + processor: + append: + field: related.hash + tag: append_related_hash_sha256 + value: '{{{_ingest._value.sha256}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.files + if: ctx.prisma_cloud?.host?.files instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.md5 + - _ingest._value.path + - _ingest._value.sha1 + - _ingest._value.sha256 + ignore_missing: true + tag: remove_file_hash_fields_ecs_duplicate + - rename: + field: json.firewallProtection + tag: rename_firewallProtection + target_field: prisma_cloud.host.firewall_protection + ignore_missing: true + - convert: + field: prisma_cloud.host.firewall_protection.enabled + tag: convert_enabled + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.firewall_protection?.enabled != '' + on_failure: + - remove: + field: prisma_cloud.host.firewall_protection.enabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: prisma_cloud.host.firewall_protection.outOfBandMode + tag: rename_out_of_band_mode + target_field: prisma_cloud.host.firewall_protection.out_of_band_mode + ignore_missing: true + - convert: + field: prisma_cloud.host.firewall_protection.ports + tag: convert_ports + type: long + ignore_missing: true + if: ctx.prisma_cloud?.host?.firewall_protection?.ports != '' + on_failure: + - remove: + field: prisma_cloud.host.firewall_protection.ports + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: prisma_cloud.host.firewall_protection.supported + tag: convert_supported + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.firewall_protection?.supported != '' + on_failure: + - remove: + field: prisma_cloud.host.firewall_protection.supported + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: prisma_cloud.host.firewall_protection.tlsPorts + tag: convert_tlsPorts + target_field: prisma_cloud.host.firewall_protection.tls_ports + type: long + ignore_missing: true + if: ctx.prisma_cloud?.host?.firewall_protection?.tlsPorts != '' + on_failure: + - remove: + field: prisma_cloud.host.firewall_protection.tlsPorts + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: prisma_cloud.host.firewall_protection.unprotectedProcesses + tag: rename_unprotectedProcesses + target_field: prisma_cloud.host.firewall_protection.unprotected_processes + ignore_missing: true + - foreach: + field: prisma_cloud.host.firewall_protection.unprotected_processes + if: ctx.prisma_cloud?.host?.firewall_protection?.unprotected_processes instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.port + tag: convert_port_firewall_protection_unprotected_processes + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.port + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.firewall_protection.unprotected_processes + if: ctx.prisma_cloud?.host?.firewall_protection?.unprotected_processes instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.tls + tag: convert_tls_firewall_protection.unprotected_processes + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.tls + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.firstScanTime + tag: date_firstScanTime + target_field: prisma_cloud.host.first_scan_time + formats: + - ISO8601 + if: ctx.json?.firstScanTime != null && ctx.json.firstScanTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.history + tag: rename_history + target_field: prisma_cloud.host.history + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.baseLayer + tag: convert_baseLayer_history + target_field: _ingest._value.base_layer + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.baseLayer + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.emptyLayer + tag: convert_emptyLayer_history + target_field: _ingest._value.empty_layer + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.emptyLayer + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.created + target_field: _ingest._value.created + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.created + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.sizeBytes + tag: convert_sizeBytes_history + target_field: _ingest._value.size_bytes + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.sizeBytes + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.baseLayer + - _ingest._value.emptyLayer + - _ingest._value.sizeBytes + tag: remove_history_fields_duplicate + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.applicableRules + tag: rename_history_vulnerabilities_applicableRules + target_field: _ingest._value.applicable_rules + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.binaryPkgs + tag: rename_history_vulnerabilities_binaryPkgs + target_field: _ingest._value.binary_pkgs + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.vecStr + tag: rename_history_vulnerabilities_vecStr + target_field: _ingest._value.vec_str + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.block + tag: convert_history_vulnerabilities_block + target_field: _ingest._value.block + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.block + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.cri + tag: convert_history_vulnerabilities_cri + target_field: _ingest._value.cri + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cri + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.custom + tag: convert_history_vulnerabilities_custom + target_field: _ingest._value.custom + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.custom + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + append: + field: vulnerability.id + tag: append_history_vulnerabilities_cve + value: '{{{_ingest._value.cve}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + append: + field: vulnerability.severity + value: '{{{_ingest._value.severity}}}' + tag: nested_append_history_vulnerabilities + allow_duplicates: false + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.cvss + tag: convert_cvss_1 + target_field: _ingest._value.cvss + type: float + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cvss + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + append: + field: vulnerability.description + tag: append_description_2 + value: '{{{_ingest._value.description}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cve + - _ingest._value.severity + - _ingest._value.description + ignore_missing: true + tag: remove_cve_severity_description_fields + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.discovered + target_field: _ingest._value.discovered + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.discovered + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.fixDate + target_field: _ingest._value.fix_date + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.fixDate + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.fixLink + tag: rename_fixLink + target_field: _ingest._value.fix_link + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.gracePeriodDays + tag: convert_gracePeriodDays + target_field: _ingest._value.grace_period_days + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.graceperiodDays + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.id + tag: convert_id + target_field: _ingest._value.id + type: string + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.id + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.packageName + tag: rename_vulnerabilities_packageName + target_field: _ingest._value.package.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.packageVersion + tag: rename_vulnerabilities_packageVersion + target_field: _ingest._value.package.version + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.riskFactors + tag: rename_vulnerabilities_riskFactors + target_field: _ingest._value.risk_factors + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.published + target_field: _ingest._value.published + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.published + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.twistlock + tag: convert_vulnerabilities_twistlock + target_field: _ingest._value.twistlock + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.twistlock + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.vulnTagInfos + tag: rename_vulnerabilities_vulnTagInfos + target_field: _ingest._value.vuln_tag_infos + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.wildfireMalware + tag: rename_vulnerabilities_wildfireMalware + target_field: _ingest._value.wild_fire_malware + ignore_missing: true + - foreach: + field: prisma_cloud.host.history + if: ctx.prisma_cloud?.host?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + remove: + field: + - _ingest._value.fixDate + - _ingest._value.gracePeriodDays + - _ingest._value.layerTime + ignore_missing: true + tag: remove_history_vulnerability_duplicate_fields + - rename: + field: json.hostDevices + tag: rename_hostDevices + target_field: prisma_cloud.host.devices + ignore_missing: true + - foreach: + field: prisma_cloud.host.devices + if: ctx.prisma_cloud?.host?.devices instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + tag: convert__devices_ip + type: ip + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.ip + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.devices + if: ctx.prisma_cloud?.host?.devices instanceof List + ignore_failure: true + processor: + append: + field: related.ip + tag: append_devices_ip_to_related_ip + value: '{{{_ingest._value.ip}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.devices + if: ctx.prisma_cloud?.host?.devices instanceof List + ignore_failure: true + processor: + append: + field: host.ip + tag: append_devices_ip_to_host_ip + value: '{{{_ingest._value.ip}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.devices + if: ctx.prisma_cloud?.host?.devices instanceof List + ignore_failure: true + processor: + append: + field: related.ip + tag: append_devices_ip_to_related_ip + value: '{{{_ingest._value.ip}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.devices + if: ctx.prisma_cloud?.host?.devices instanceof List + ignore_failure: true + processor: + append: + field: related.hosts + tag: append_devices_name_to_related_hosts + value: '{{{_ingest._value.name}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.devices + if: ctx.prisma_cloud?.host?.devices instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.ip + - _ingest._value.name + ignore_missing: true + tag: foreach_remove_ecs_duplicate_devices_fields + - rename: + field: json.hostname + tag: rename_hostname + target_field: prisma_cloud.host.hostname + ignore_missing: true + - set: + field: host.hostname + tag: set_hostname + copy_from: prisma_cloud.host.hostname + ignore_empty_value: true + - append: + field: related.hosts + value: '{{{prisma_cloud.host.hostname}}}' + tag: append_hostname_into_hosts + allow_duplicates: false + if: ctx.prisma_cloud?.host?.hostname != null + - convert: + field: json.hostRuntimeEnabled + tag: convert_hostRuntimeEnabled + target_field: prisma_cloud.host.runtime_enabled + type: boolean + ignore_missing: true + if: ctx.json?.hostRuntimeEnabled != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.hosts + tag: rename_hosts + target_field: prisma_cloud.host.hosts + ignore_missing: true + - rename: + field: json.id + tag: rename_id + target_field: prisma_cloud.host.id + ignore_missing: true + - date: + field: json.image.created + tag: date_created + target_field: prisma_cloud.host.image.created + formats: + - ISO8601 + if: ctx.json?.image?.created != null && ctx.json.image.created != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.image.entrypoint + tag: rename_entrypoint + target_field: prisma_cloud.host.image.entrypoint + ignore_missing: true + - rename: + field: json.image.env + tag: rename_env + target_field: prisma_cloud.host.image.env + ignore_missing: true + - convert: + field: json.image.healthcheck + tag: convert_healthcheck + target_field: prisma_cloud.host.image.healthcheck + type: boolean + ignore_missing: true + if: ctx.json?.image?.healthcheck != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.image.history + tag: rename_image_history + target_field: prisma_cloud.host.image.history + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.baseLayer + tag: convert_image_history_baseLayer + target_field: _ingest._value.base_layer + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.baseLayer + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.emptyLayer + tag: convert_image_history_emptylayer + target_field: _ingest._value.empty_layer + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.emptyLayer + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.created + target_field: _ingest._value.created + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.created + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.sizeBytes + tag: convert_image_history_sizeBytes + target_field: _ingest._value.size_bytes + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.sizeBytes + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.baseLayer + - _ingest._value.emptyLayer + - _ingest._value.sizeBytes + tag: foreach_remove_image_history_duplicate_fields + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.applicableRules + tag: rename_image_history_vulnerabilities_applicableRules + target_field: _ingest._value.applicable_rules + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.binaryPkgs + tag: rename_image_history_vulnerabilities_binaryPkgs + target_field: _ingest._value.binary_pkgs + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.vecStr + tag: rename_image_history_vulnerabilities_vecStr + target_field: _ingest._value.vec_str + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.block + tag: convert_image_history_vulnerabilities_block + target_field: _ingest._value.block + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.block + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.cri + tag: convert_image_history_vulnerabilities_cri + target_field: _ingest._value.cri + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cri + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.custom + tag: convert_image_history_vulnerabilities_custom + target_field: _ingest._value.custom + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.custom + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + append: + field: vulnerability.id + tag: append_image_history_vulnerabilities_cve_to_vulnerability_id + value: '{{{_ingest._value.cve}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + append: + field: vulnerability.severity + tag: append_image_history_vulnerabilities_severity + value: '{{{_ingest._value.severity}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.cvss + tag: convert_image_history_vulnerabilities_cvss + target_field: _ingest._value.cvss + type: float + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cvss + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + append: + field: vulnerability.description + tag: append__image_history_vulnerabilities_description + value: '{{{_ingest._value.description}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cve + - _ingest._value.severity + - _ingest._value.description + ignore_missing: true + tag: remove_image_history_vulnerabilites_ecs_duplicate_fields + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.discovered + target_field: _ingest._value.discovered + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.discovered + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.fixDate + target_field: _ingest._value.fix_date + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.fixDate + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.fixLink + tag: rename_image_history_vulnerabilities_fixLink + target_field: _ingest._value.fix_link + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_image_history_vulnerabilities_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.gracePeriodDays + tag: convert_image_history_vulnerabilities_gracePeriodDays + target_field: _ingest._value.grace_period_days + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.gracePeriodDays + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.id + tag: convert_image_history_vulnerabilities_id + target_field: _ingest._value.id + type: string + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.id + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.packageName + tag: rename_image_history_vulnerabilities_packageName + target_field: _ingest._value.package.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.packageVersion + tag: rename_image_history_vulnerabilities_packageVersion + target_field: _ingest._value.package.version + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.riskFactors + tag: rename_image_history_vulnerabilities_riskFactors + target_field: _ingest._value.risk_factors + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + date: + field: _ingest._value.published + target_field: _ingest._value.published + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.published + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + convert: + field: _ingest._value.twistlock + tag: convert_image_history_vulnerabilities_twistlock + target_field: _ingest._value.twistlock + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.twistlock + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.vulnTagInfos + tag: rename_image_history_vulnerabilities_vulnTagInfos + target_field: _ingest._value.vuln_tag_infos + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + rename: + field: _ingest._value.wildfireMalware + tag: rename_image_history_vulnerabilities_wildfireMalware + target_field: _ingest._value.wild_fire_malware + ignore_missing: true + - foreach: + field: prisma_cloud.host.image.history + if: ctx.prisma_cloud?.host?.image?.history instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.vulnerabilities + ignore_failure: true + processor: + remove: + field: + - _ingest._value.fixDate + - _ingest._value.gracePeriodDays + - _ingest._value.layerTime + ignore_missing: true + tag: remove_custom_ecs_mapped_fields + - rename: + field: json.image.id + tag: rename_image_id + target_field: prisma_cloud.host.image.id + ignore_missing: true + - rename: + field: json.image.labels + tag: rename_image_labels + target_field: prisma_cloud.host.image.labels + ignore_missing: true + - rename: + field: json.image.layers + tag: rename_image_layers + target_field: prisma_cloud.host.image.layers + ignore_missing: true + - rename: + field: json.image.os + tag: rename_image_os + target_field: prisma_cloud.host.image.os + ignore_missing: true + - rename: + field: json.image.repoDigest + tag: rename_repoDigest + target_field: prisma_cloud.host.image.repo.digest + ignore_missing: true + - rename: + field: json.image.repoTags + tag: rename_repoTags + target_field: prisma_cloud.host.image.repo.tags + ignore_missing: true + - rename: + field: json.image.user + tag: rename_image_user + target_field: prisma_cloud.host.image.user + ignore_missing: true + - rename: + field: json.image.workingDir + tag: rename_image_workingDir + target_field: prisma_cloud.host.image.working_dir + ignore_missing: true + - convert: + field: json.installedProducts.agentless + tag: convert_prisma_cloud_host_installed_products_agentless + target_field: prisma_cloud.host.installed_products.agentless + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.agentless != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.installedProducts.apache + tag: rename_installedProducts_apache + target_field: prisma_cloud.host.installed_products.apache + ignore_missing: true + - rename: + field: json.installedProducts.docker + tag: rename_installedProducts_docker + target_field: prisma_cloud.host.installed_products.docker + ignore_missing: true + - rename: + field: json.installedProducts.kubernetes + tag: rename_installedProducts_kubernetes + target_field: prisma_cloud.host.installed_products.kubernetes + ignore_missing: true + - convert: + field: json.installedProducts.awsCloud + tag: convert_awsClous + target_field: prisma_cloud.host.installed_products.aws_cloud + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.awsCloud != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.installedProducts.clusterType + tag: rename_clusterType + target_field: prisma_cloud.host.installed_products.cluster_type + ignore_missing: true + - convert: + field: json.installedProducts.crio + tag: convert_crio + target_field: prisma_cloud.host.installed_products.crio + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.crio != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.dockerEnterprise + tag: convert_dockerEnterprise + target_field: prisma_cloud.host.installed_products.docker_enterprise + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.dockerEnterprise != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.hasPackageManager + tag: convert_hasPackageManager + target_field: prisma_cloud.host.installed_products.has_package_manager + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.hasPackageManager != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sApiServer + tag: convert_k8sApiserver + target_field: prisma_cloud.host.installed_products.k8s_api_server + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sApiServer != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sControllerManager + tag: convert_k8sControllerManager + target_field: prisma_cloud.host.installed_products.k8s_controller_manager + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sControllerManager != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sEtcd + tag: convert_k8sEtcd + target_field: prisma_cloud.host.installed_products.k8s_etcd + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sEtcd != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sFederationApiServer + tag: convert_k8sFederationApiServer + target_field: prisma_cloud.host.installed_products.k8s_federation_api_server + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sFederationApiServer != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sFederationControllerManager + tag: convert_k8sFederationControllerManager + target_field: prisma_cloud.host.installed_products.k8s_federation_controller_manager + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sFederationControllerManager != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sKubelet + tag: convert_k8sKubelet + target_field: prisma_cloud.host.installed_products.k8s_kubelet + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sKubelet != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sProxy + tag: convert_k8sProxy + target_field: prisma_cloud.host.installed_products.k8s_proxy + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sProxy != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.k8sScheduler + tag: convert_k8sScheduler + target_field: prisma_cloud.host.installed_products.k8s_scheduler + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.k8sScheduler != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.installedProducts.managedClusterVersion + tag: rename_managedClusterVersion + target_field: prisma_cloud.host.installed_products.managed_cluster_version + ignore_missing: true + - convert: + field: json.installedProducts.openshift + tag: convert_openshift + target_field: prisma_cloud.host.installed_products.openshift + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.openshift != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.installedProducts.openshiftVersion + tag: rename_openshiftVersion + target_field: prisma_cloud.host.installed_products.openshift_version + ignore_missing: true + - rename: + field: json.installedProducts.osDistro + tag: rename_osDistro + target_field: prisma_cloud.host.installed_products.os_distro + ignore_missing: true + - convert: + field: json.installedProducts.serverless + tag: convert_serverless + target_field: prisma_cloud.host.installed_products.serverless + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.serverless != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.swarmManager + tag: convert_swarmManager + target_field: prisma_cloud.host.installed_products.swarm.manager + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.swarmManager != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.installedProducts.swarmNode + tag: convert_swarmNode + target_field: prisma_cloud.host.installed_products.swarm.node + type: boolean + ignore_missing: true + if: ctx.prisma_cloud?.host?.installed_products?.swarmNode != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.instances + tag: rename_instances + target_field: prisma_cloud.host.instances + ignore_missing: true + - foreach: + field: prisma_cloud.host.instances + if: ctx.prisma_cloud?.host?.instances instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.modified + target_field: _ingest._value.modified + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - convert: + field: json.isARM64 + target_field: prisma_cloud.host.is_arm64 + tag: convert_isARM64 + type: boolean + ignore_missing: true + if: ctx.json?.isARM64 != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.k8sClusterAddr + tag: rename_k8sClusterAddr + target_field: prisma_cloud.host.k8s_cluster_addr + ignore_missing: true + - rename: + field: json.labels + tag: rename_labels + target_field: prisma_cloud.host.labels + ignore_missing: true + - date: + field: json.malwareAnalyzedTime + tag: date_malwareAnalyzedTime + target_field: prisma_cloud.host.malware_analyzed_time + formats: + - ISO8601 + if: ctx.json?.malwareAnalyzedTime != null && ctx.json.malwareAnalyzedTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.missingDistroVulnCoverage + tag: convert_missingDistroVulnCoverage + target_field: prisma_cloud.host.missing_distro_vuln_coverage + type: boolean + ignore_missing: true + if: ctx.json?.missingDistroVulnCoverage != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.namespaces + tag: rename_namespaces + target_field: prisma_cloud.host.namespaces + ignore_missing: true + - rename: + field: json.osDistro + tag: rename_osDistro + target_field: prisma_cloud.host.os_distro.value + ignore_missing: true + - set: + field: os.family + tag: set_os_distro_value_to_os_family + copy_from: prisma_cloud.host.os_distro.value + ignore_empty_value: true + - rename: + field: json.osDistroRelease + tag: rename_osDistroRelease + target_field: prisma_cloud.host.os_distro.release + ignore_missing: true + - set: + field: os.name + tag: set_os_distro_release_to_os_name + copy_from: prisma_cloud.host.os_distro.release + ignore_empty_value: true + - rename: + field: json.osDistroVersion + tag: rename_osDistroVersion + target_field: prisma_cloud.host.os_distro.version + ignore_missing: true + - set: + field: os.version + tag: set_os_distro_version_to_os_version + copy_from: prisma_cloud.host.os_distro.version + ignore_empty_value: true + - convert: + field: json.packageCorrelationDone + tag: convert_packageCorrelationDone + target_field: prisma_cloud.host.package.correlation_done + type: boolean + ignore_missing: true + if: ctx.json?.packageCorrelationDone != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.packageManager + tag: convert_packageManager + target_field: prisma_cloud.host.package.manager + type: boolean + ignore_missing: true + if: ctx.json?.packageManager != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.packages + tag: rename_packages + target_field: prisma_cloud.host.packages + ignore_missing: true + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.pkgsType + tag: rename_packages_pkgsType + target_field: _ingest._value.pkgs_type + ignore_missing: true + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + append: + field: package.type + tag: append_packages_pkgs_type_to_package_type + value: '{{{_ingest._value.pkgs_type}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.pkgs_type + ignore_missing: true + tag: foreach_remove_packages_ecs_duplicate_fields + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + convert: + field: _ingest._value.binaryIdx + tag: convert_packages_binaryIdx + target_field: _ingest._value.binary_idx + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.binaryIdx + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + rename: + field: _ingest._value.binaryPkgs + tag: rename_packages_pkgs_binaryPkgs + target_field: _ingest._value.binary_pkgs + ignore_missing: true + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + convert: + field: _ingest._value.cveCount + tag: convert_packages_pkgs_cveCount + target_field: _ingest._value.cve_count + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cveCount + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + convert: + field: _ingest._value.defaultGem + tag: convert_packages_pkgs_defaultGem + target_field: _ingest._value.default_gem + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.defaultGem + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_packages_pkgs_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + convert: + field: _ingest._value.goPkg + tag: convert_packages_pkgs_goPkg + target_field: _ingest._value.go_pkg + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.goPkg + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + rename: + field: _ingest._value.jarIdentifier + tag: rename_packages_pkgs_jarIdentifier + target_field: _ingest._value.jar_identifier + ignore_missing: true + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + append: + field: package.license + tag: append_packages_pkgs_license + value: '{{{_ingest._value.license}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + append: + field: package.name + tag: append_packages_pkgs_name_to_package_name + value: '{{{_ingest._value.name}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + append: + field: package.path + tag: append_packages_pkgs_path + value: '{{{_ingest._value.path}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + append: + field: package.version + tag: append_packages_pkgs_version_to_package_version + value: '{{{_ingest._value.version}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + remove: + field: + - _ingest._value.license + - _ingest._value.name + - _ingest._value.path + - _ingest._value.version + ignore_missing: true + tag: remove_packages_ecs_duplicate_fields_custom + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + convert: + field: _ingest._value.osPackage + tag: convert_packages_pkgs_osPackage + target_field: _ingest._value.os_package + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.osPackage + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.packages + if: ctx.prisma_cloud?.host?.packages instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.pkgs + ignore_failure: true + processor: + remove: + field: + - _ingest._value.binaryIdx + - _ingest._value.cveCount + - _ingest._value.defaultGem + - _ingest._value.goPkg + - _ingest._value.layerTime + - _ingest._value.osPackage + ignore_missing: true + tag: remove_pacages_custom_duplicate + - convert: + field: json.pullDuration + tag: convert_pullDuration + target_field: prisma_cloud.host.pull_duration + type: long + ignore_missing: true + if: ctx.json?.pullDuration != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.pushTime + tag: date_pushTime + target_field: prisma_cloud.host.push_time + formats: + - ISO8601 + if: ctx.json?.pushTime != null && ctx.json.pushTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.redHatNonRPMImage + tag: convert_redHatNonRPMImage + target_field: prisma_cloud.host.red_hat_non_rpm_image + type: boolean + ignore_missing: true + if: ctx.json?.redHatNonRPMImage != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.registryNamespace + tag: rename_registryNamespace + target_field: prisma_cloud.host.registry.namespace + ignore_missing: true + - rename: + field: json.registryTags + tag: rename_registryTags + target_field: prisma_cloud.host.registry.tags + ignore_missing: true + - rename: + field: json.registryType + tag: rename_registryType + target_field: prisma_cloud.host.registry.type + ignore_missing: true + - rename: + field: json.repoDigests + tag: rename_repoDigests + target_field: prisma_cloud.host.repo_digests + ignore_missing: true + - rename: + field: json.repoTag.digest + tag: rename_repoTag_digest + target_field: prisma_cloud.host.repo_tag.digest + ignore_missing: true + - rename: + field: json.repoTag.id + tag: rename_repoTag_id + target_field: prisma_cloud.host.repo_tag.id + ignore_missing: true + - rename: + field: json.repoTag.registry + tag: rename_repoTag_registry + target_field: prisma_cloud.host.repo_tag.registry + ignore_missing: true + - rename: + field: json.repoTag.repo + tag: rename_repoTag_repo + target_field: prisma_cloud.host.repo_tag.repo + ignore_missing: true + - rename: + field: json.repoTag.tag + tag: rename_repoTag_tag + target_field: prisma_cloud.host.repo_tag.value + ignore_missing: true + - rename: + field: json.riskFactors + tag: rename_riskFactors + target_field: prisma_cloud.host.risk_factors + ignore_missing: true + - rename: + field: json.rhelRepos + tag: rename_rhelRepos + target_field: prisma_cloud.host.rhel_repos + ignore_missing: true + - date: + field: json.scanBuildDate + tag: date_scanBuildDate + target_field: prisma_cloud.host.scan.build_date + formats: + - ISO8601 + - basic_date + if: ctx.json?.scanBuildDate != null && ctx.json.scanBuildDate != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.scanDuration + tag: convert_scanDuration + target_field: prisma_cloud.host.scan.duration + type: long + ignore_missing: true + if: ctx.json?.scanDuration != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.scanId + tag: rename_scanId + target_field: prisma_cloud.host.scan.id + ignore_missing: true + - date: + field: json.scanTime + tag: date_scanTime + target_field: prisma_cloud.host.scan.time + formats: + - ISO8601 + if: ctx.json?.scanTime != null && ctx.json.scanTime != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.scanVersion + tag: rename_scanVersion + target_field: prisma_cloud.host.scan.version + ignore_missing: true + - rename: + field: json.Secrets + tag: rename_Secrets + target_field: prisma_cloud.host.secrets + ignore_missing: true + - rename: + field: json.startupBinaries + tag: rename_startupBinaries + target_field: prisma_cloud.host.startup_binaries + ignore_missing: true + - foreach: + field: prisma_cloud.host.startup_binaries + tag: foreach_convert_startup_binaries_altered + if: ctx.prisma_cloud?.host?.startup_binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.altered + tag: convert_startup_binaries_altered + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.altered + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.startup_binaries + if: ctx.prisma_cloud?.host?.startup_binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cveCount + tag: convert_startup_binaries_cveCount + target_field: _ingest._value.cve_count + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cveCount + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.startup_binaries + if: ctx.prisma_cloud?.host?.startup_binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.fileMode + tag: convert_startup_binaries_fileMode + target_field: _ingest._value.file_mode + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.fileMode + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.startup_binaries + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_startup_binaries_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.startup_binaries + if: ctx.prisma_cloud?.host?.startup_binaries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.missingPkg + tag: convert_startup_binaries_missingPkg + target_field: _ingest._value.missing_pkg + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.missingPkg + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.startup_binaries + ignore_failure: true + processor: + rename: + field: _ingest._value.pkgRootDir + tag: rename_startup_binaries_pkgRootDir + target_field: _ingest._value.pkg_root_dir + ignore_missing: true + - foreach: + field: prisma_cloud.host.startup_binaries + if: ctx.prisma_cloud?.host?.startup_binaries instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cveCount + - _ingest._value.fileMode + - _ingest._value.missingPkg + ignore_missing: true + tag: foreach_remove_startupBinaries_custom_duplicate_fields + - convert: + field: json.stopped + tag: convert_stopped + target_field: prisma_cloud.host.stopped + type: boolean + ignore_missing: true + if: ctx.json?.stopped != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.tags + tag: rename_tags + target_field: prisma_cloud.host.tags + ignore_missing: true + - rename: + field: json.topLayer + tag: rename_topLayer + target_field: prisma_cloud.host.top_layer + ignore_missing: true + - rename: + field: json.trustResult.hostsStatuses + tag: rename_hostsStatuses + target_field: prisma_cloud.host.trust_result.hosts_statuses + ignore_missing: true + - rename: + field: json.trustResult.groups + tag: rename_groups + target_field: prisma_cloud.host.trust_result.groups + ignore_missing: true + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List + ignore_failure: true + processor: + append: + field: group.name + tag: append_trust_result_groups_id_to_group_name + value: '{{{_ingest._value._id}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.disabled + tag: convert_trust_result_groups_disabled + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.disabled + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.modified + target_field: _ingest._value.modified + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List + ignore_failure: true + processor: + append: + field: rule.name + tag: append_trust_result_groups_name_to_rule_name + value: '{{{_ingest._value.name}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List + ignore_failure: true + processor: + append: + field: rule.author + tag: append_trust_result_groups_owner_to_rule_author + value: '{{{_ingest._value.owner}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.id + - _ingest._value.name + - _ingest._value.owner + ignore_missing: true + tag: foreach_remove_ecs_mapped_trustResult_fields + - foreach: + field: prisma_cloud.host.trust_result.groups + if: ctx.prisma_cloud?.host?.trust_result?.groups instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.previousName + tag: rename_trust_result_groups_previousName + target_field: _ingest._value.previous_name + ignore_missing: true + - rename: + field: json.trustStatus + tag: rename_trustStatus + target_field: prisma_cloud.host.trust_status + ignore_missing: true + - convert: + field: json.twistlockImage + tag: convert_twistlockImage + target_field: prisma_cloud.host.twistlock_image + type: boolean + ignore_missing: true + if: ctx.json?.twistlockImage != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.type + tag: rename_type + target_field: prisma_cloud.host.type + ignore_missing: true + - set: + field: host.type + tag: set_type_to_host_type + copy_from: prisma_cloud.host.type + ignore_empty_value: true + - rename: + field: json.vulnerabilities + tag: rename_vulnerabilities + target_field: prisma_cloud.host.vulnerabilities.data + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.applicableRules + tag: rename_vulnerabilities_data_applicableRules + target_field: _ingest._value.applicable_rules + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.binaryPkgs + tag: rename_vulnerabilities_data_binaryPkgs + target_field: _ingest._value.binary_pkgs + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vecStr + tag: rename_vulnerabilities_data_vecStr + target_field: _ingest._value.vec_str + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.block + tag: convert_vulnerabilities_data_block + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.block + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cri + tag: convert_vulnerabilities_data_cri + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cri + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.custom + tag: convert_vulnerabilities_data_custom + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.custom + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.id + tag: append_vulnerabilities_data_cve_to_vulnerability_id + value: '{{{_ingest._value.cve}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.severity + tag: append__vulnerabilities_data_severity + value: '{{{_ingest._value.severity}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cvss + tag: convert_vulnerabilities_data_cvss + type: float + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cvss + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + append: + field: vulnerability.description + tag: append_vulnerabilities_data_description + value: '{{{_ingest._value.description}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.cve + - _ingest._value.severity + - _ingest._value.description + ignore_missing: true + tag: foreach_remove_ecs_mapped_vulnerabilities_data_fields + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.discovered + target_field: _ingest._value.discovered + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.discovered + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.fixDate + target_field: _ingest._value.fix_date + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.fixDate + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.fixLink + tag: rename_vulnerabilities_data_fixLink + target_field: _ingest._value.fix_link + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.functionLayer + tag: rename_vulnerabilities_data_functionLayer + target_field: _ingest._value.function_layer + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.gracePeriodDays + tag: convert_vulnerabilities_data_gracePeriodDays + target_field: _ingest._value.grace_period_days + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.gracePeriodDays + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.id + tag: convert_vulnerabilities_data_id + type: string + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.id + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.layerTime + target_field: _ingest._value.layer_time + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.layerTime + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.packageName + tag: rename_vulnerabilities_data_packageName + target_field: _ingest._value.package.name + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.packageVersion + tag: rename_vulnerabilities_data_packageVersion + target_field: _ingest._value.package.version + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.riskFactors + tag: rename_vulnerabilities_data_riskFactors + target_field: _ingest._value.risk_factors + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.published + target_field: _ingest._value.published + formats: + - UNIX_MS + on_failure: + - remove: + field: _ingest._value.published + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.twistlock + tag: convert_vulnerabilities_data_twistlock + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.twistlock + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vulnTagInfos + tag: rename_vulnerabilities_data_vulnTagInfos + target_field: _ingest._value.vuln_tag_infos + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.wildfireMalware + tag: rename_vulnerabilities_data_wildfireMalware + target_field: _ingest._value.wild_fire_malware + ignore_missing: true + - foreach: + field: prisma_cloud.host.vulnerabilities.data + if: ctx.prisma_cloud?.host?.vulnerabilities?.data instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.fixDate + - _ingest._value.gracePeriodDays + - _ingest._value.layerTime + ignore_missing: true + tag: foreach_remove_ecs_dupliacte_fields + - convert: + field: json.vulnerabilitiesCount + tag: convert_vulnerabilitiesCount + target_field: prisma_cloud.host.vulnerabilities.count + type: long + ignore_missing: true + if: ctx.json?.vulnerabilitiesCount != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.vulnerabilityDistribution.critical + tag: convert_vulnerabilityDistribution_critical + target_field: prisma_cloud.host.vulnerability.distribution.critical + type: long + ignore_missing: true + if: ctx.json?.vulnerabilityDistribution?.critical != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.vulnerabilityDistribution.high + tag: convert_vulnerabilityDistribution_high + target_field: prisma_cloud.host.vulnerability.distribution.high + type: long + ignore_missing: true + if: ctx.json?.vulnerabilityDistribution?.high != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.vulnerabilityDistribution.low + tag: convert_vulnerabilityDistribution_low + target_field: prisma_cloud.host.vulnerability.distribution.low + type: long + ignore_missing: true + if: ctx.json?.vulnerabilityDistribution?.low != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.vulnerabilityDistribution.medium + tag: convert_vulnerabilityDistribution_medium + target_field: prisma_cloud.host.vulnerability.distribution.medium + type: long + ignore_missing: true + if: ctx.json?.vulnerabilityDistribution?.medium != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.vulnerabilityDistribution.total + tag: convert_vulnerabilityDistribution_total + target_field: prisma_cloud.host.vulnerability.distribution.total + type: long + ignore_missing: true + if: ctx.json?.vulnerabilityDistribution?.total != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.vulnerabilityRiskScore + tag: convert_vulnerabilityRiskScore + target_field: prisma_cloud.host.vulnerability.risk_score + type: long + ignore_missing: true + if: ctx.json?.vulnerabilityRiskScore != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.wildFireUsage.bytes + tag: convert_wildFireUsage_bytes + target_field: prisma_cloud.host.wild_fire_usage.bytes + type: long + ignore_missing: true + if: ctx.json?.wildFireUsage?.bytes != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.wildFireUsage.queries + tag: convert_wildFireUsage_queries + target_field: prisma_cloud.host.wild_fire_usage.queries + type: long + ignore_missing: true + if: ctx.json?.wildFireUsage?.queries != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.wildFireUsage.uploads + tag: convert_wildFireUsage_uploads + target_field: prisma_cloud.host.wild_fire_usage.uploads + type: long + ignore_missing: true + if: ctx.json?.wildFireUsage?.uploads != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - remove: + field: + - json + - prisma_cloud.host.firewall_protection.tlsPorts + - prisma_cloud.host.installed_products.awsCloud + - prisma_cloud.host.installed_products.dockerEnterprise + - prisma_cloud.host.installed_products.hasPackageManager + - prisma_cloud.host.installed_products.k8sApiServer + - prisma_cloud.host.installed_products.k8sControllerManager + - prisma_cloud.host.installed_products.k8sEtcd + - prisma_cloud.host.installed_products.k8sFederationApiServer + - prisma_cloud.host.installed_products.k8sFederationControllerManager + - prisma_cloud.host.installed_products.k8sKubelet + - prisma_cloud.host.installed_products.k8sProxy + - prisma_cloud.host.installed_products.k8sScheduler + - prisma_cloud.host.installed_products.swarmManager + - prisma_cloud.host.installed_products.swarmNode + ignore_missing: true + tag: remove_json + - remove: + field: + - prisma_cloud.host._id + - prisma_cloud.host.cloud_metadata.account_id + - prisma_cloud.host.cloud_metadata.name + - prisma_cloud.host.cloud_metadata.provider + - prisma_cloud.host.cloud_metadata.region + - prisma_cloud.host.cloud_metadata.resource.id + - prisma_cloud.host.cloud_metadata.type + - prisma_cloud.host.creation_time + - prisma_cloud.host.hostname + - prisma_cloud.host.os_distro.version + - prisma_cloud.host.os_distro.release + - prisma_cloud.host.os_distro.value + - prisma_cloud.host.type + ignore_missing: true + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + tag: remove_preserve_duplicate_custom_fields + - remove: + field: event.original + ignore_missing: true + tag: remove_event_original + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + - script: + lang: painless + source: |- + boolean drop(Object o) { + if (o == null || o == '') { + return true; + } else if (o instanceof Map) { + ((Map) o).values().removeIf(v -> drop(v)); + return (((Map) o).size() == 0); + } else if (o instanceof List) { + ((List) o).removeIf(v -> drop(v)); + return (((List) o).length == 0); + } + return false; + } + drop(ctx); + description: Drops null/empty values recursively. + tag: script_drop_null_values + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + tag: set_pipeline_error_in_event_kind + value: pipeline_error + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + value: pipeline_error diff --git a/packages/prisma_cloud/data_stream/host/fields/base-fields.yml b/packages/prisma_cloud/data_stream/host/fields/base-fields.yml new file mode 100644 index 00000000000..e58ee0689e0 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: prisma_cloud +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: prisma_cloud.host +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/prisma_cloud/data_stream/host/fields/beats.yml b/packages/prisma_cloud/data_stream/host/fields/beats.yml new file mode 100644 index 00000000000..b3701b581cf --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/fields/beats.yml @@ -0,0 +1,9 @@ +- name: input.type + type: keyword + description: Type of filebeat input. +- name: log.offset + type: long + description: Log offset. +- name: tags + type: keyword + description: User defined tags. diff --git a/packages/prisma_cloud/data_stream/host/fields/fields.yml b/packages/prisma_cloud/data_stream/host/fields/fields.yml new file mode 100644 index 00000000000..38997b03d3b --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/fields/fields.yml @@ -0,0 +1,1444 @@ +- name: prisma_cloud + type: group + fields: + - name: host + type: group + fields: + - name: _id + type: keyword + description: Image identifier (image ID or repo:tag). + - name: agentless + type: boolean + description: Agentless indicates that the host was scanned with the agentless scanner. + - name: all_compliance + type: group + fields: + - name: data + type: group + fields: + - name: applicable_rules + type: keyword + description: Rules applied on the package. + - name: binary_pkgs + type: keyword + description: Names of the distro binary package names (packages which are built from the source of the package). + - name: block + type: boolean + description: Indicates if the vulnerability has a block effect (true) or not (false). + - name: cause + type: keyword + description: Additional information regarding the root cause for the vulnerability. + - name: cri + type: boolean + description: Indicates if this is a CRI-specific vulnerability (true) or not (false). + - name: custom + type: boolean + description: Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). + - name: cve + type: keyword + description: CVE ID of the vulnerability (if applied). + - name: cvss + type: float + description: CVSS score of the vulnerability. + - name: description + type: keyword + description: Description of the vulnerability. + - name: discovered + type: date + description: Specifies the time of discovery for the vulnerability. + - name: exploit + type: keyword + description: ExploitType represents the source of an exploit. + - name: exploits + type: group + fields: + - name: kind + type: keyword + description: ExploitKind represents the kind of the exploit. + - name: link + type: keyword + description: Link is a link to information about the exploit. + - name: source + type: keyword + description: ExploitType represents the source of an exploit. + - name: fix_date + type: date + description: Date/time when the vulnerability was fixed (in Unix time). + - name: fix_link + type: keyword + description: Link to the vendor's fixed-version information. + - name: function_layer + type: keyword + description: Specifies the serverless layer ID in which the vulnerability was discovered. + - name: grace_period_days + type: long + description: Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. + - name: id + type: keyword + description: ID of the violation. + - name: layer_time + type: date + description: Date/time of the image layer to which the CVE belongs. + - name: link + type: keyword + description: Vendor link to the CVE. + - name: package + type: group + fields: + - name: name + type: keyword + description: Name of the package that caused the vulnerability. + - name: version + type: keyword + description: Version of the package that caused the vulnerability (or null). + - name: published + type: date + description: Date/time when the vulnerability was published (in Unix time). + - name: risk_factors + type: flattened + description: RiskFactors maps the existence of vulnerability risk factors. + - name: severity + type: keyword + description: Textual representation of the vulnerability's severity. + - name: status + type: keyword + description: Vendor status for the vulnerability. + - name: templates + type: keyword + description: List of templates with which the vulnerability is associated. + - name: text + type: keyword + description: Description of the violation. + - name: title + type: keyword + description: Compliance title. + - name: twistlock + type: boolean + description: Indicates if this is a Twistlock-specific vulnerability (true) or not (false). + - name: type + type: keyword + description: Type represents the vulnerability type. + - name: vec_str + type: keyword + description: Textual representation of the metric values used to score the vulnerability. + - name: vuln_tag_infos + type: group + fields: + - name: color + type: keyword + description: Color is a hexadecimal representation of color code value. + - name: comment + type: keyword + description: Tag comment in a specific vulnerability context. + - name: name + type: keyword + description: Name of the tag. + - name: wild_fire_malware + type: group + fields: + - name: md5 + type: keyword + description: MD5 is the hash of the malicious binary. + - name: path + type: keyword + description: Path is the path to malicious binary. + - name: verdict + type: keyword + description: Verdict is the malicious source like grayware, malware and phishing. + - name: enabled + type: boolean + description: Enabled indicates whether passed compliance checks is enabled by policy. + - name: app_embedded + type: boolean + description: Indicates that this image was scanned by an App-Embedded Defender. + - name: applications + type: group + fields: + - name: installed_from_package + type: boolean + description: Indicates that the app was installed as an OS package. + - name: known_vulnerabilities + type: long + description: Total number of vulnerabilities for this application. + - name: layer_time + type: date + description: Image layer to which the application belongs - layer creation time. + - name: name + type: keyword + description: Name of the application. + - name: path + type: keyword + description: Path of the detected application. + - name: service + type: boolean + description: Service indicates whether the application is installed as a service. + - name: version + type: keyword + description: Version of the application. + - name: base_image + type: keyword + description: Image’s base image name. Used when filtering the vulnerabilities by base images. + - name: binaries + type: group + fields: + - name: altered + type: boolean + description: Indicates if the binary was installed from a package manager and modified/replaced (true) or not (false). + - name: cve_count + type: long + description: Total number of CVEs for this specific binary. + - name: deps + type: keyword + description: Third-party package files which are used by the binary. + - name: file_mode + type: long + description: Represents the file's mode and permission bits. + - name: function_layer + type: keyword + description: ID of the serverless layer in which the package was discovered. + - name: md5 + type: keyword + description: Md5 hashset of the binary. + - name: missing_pkg + type: boolean + description: Indicates if this binary is not related to any package (true) or not (false). + - name: name + type: keyword + description: Name of the binary. + - name: path + type: keyword + description: Path is the path of the binary. + - name: pkg_root_dir + type: keyword + description: Path for searching packages used by the binary. + - name: services + type: keyword + description: Names of services which use the binary. + - name: version + type: keyword + description: Version of the binary. + - name: cloud_metadata + type: group + fields: + - name: account_id + type: keyword + description: Cloud account ID. + - name: aws_execution_env + type: keyword + description: AWS execution environment (e.g. EC2/Fargate). + - name: image + type: keyword + description: Image name. + - name: labels + type: group + fields: + - name: key + type: keyword + description: Label key. + - name: source + type: group + fields: + - name: name + type: keyword + description: Source name (e.g., for a namespace, the source name can be 'twistlock'). + - name: type + type: keyword + description: ExternalLabelSourceType indicates the source of the labels. + - name: timestamp + type: date + description: Time when the label was fetched. + - name: value + type: keyword + description: Value of the label. + - name: name + type: keyword + description: Instance name. + - name: provider + type: keyword + description: CloudProvider specifies the cloud provider name. + - name: region + type: keyword + description: Instance region. + - name: resource + type: group + fields: + - name: id + type: keyword + description: Unique ID of the resource. + - name: url + type: keyword + description: Server-defined URL for the resource. + - name: type + type: keyword + description: Instance type. + - name: vm + type: group + fields: + - name: id + type: keyword + description: Azure unique vm ID. + - name: image_id + type: keyword + description: VMImageID holds the VM image ID. + - name: cluster_type + type: keyword + description: ClusterType is the cluster type. + - name: clusters + type: keyword + description: Cluster names. + - name: collections + type: keyword + description: Collections to which this result applies. + - name: compliance_distribution + type: group + fields: + - name: critical + type: long + - name: high + type: long + - name: low + type: long + - name: medium + type: long + - name: total + type: long + - name: compliance_issues + type: group + fields: + - name: count + type: long + description: Number of compliance issues. + - name: data + type: group + fields: + - name: applicable_rules + type: keyword + description: Rules applied on the package. + - name: binary_pkgs + type: keyword + description: Names of the distro binary package names (packages which are built from the source of the package). + - name: block + type: boolean + description: Indicates if the vulnerability has a block effect (true) or not (false). + - name: cause + type: keyword + description: Additional information regarding the root cause for the vulnerability. + - name: cri + type: boolean + description: Indicates if this is a CRI-specific vulnerability (true) or not (false). + - name: custom + type: boolean + description: Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). + - name: cve + type: keyword + description: CVE ID of the vulnerability (if applied). + - name: cvss + type: float + description: CVSS score of the vulnerability. + - name: description + type: keyword + description: Description of the vulnerability. + - name: discovered + type: date + description: Specifies the time of discovery for the vulnerability. + - name: exploit + type: keyword + description: ExploitType represents the source of an exploit. + - name: exploits + type: group + fields: + - name: kind + type: keyword + description: ExploitKind represents the kind of the exploit. + - name: link + type: keyword + description: Link is a link to information about the exploit. + - name: source + type: keyword + description: ExploitType represents the source of an exploit. + - name: fix_date + type: date + description: Date/time when the vulnerability was fixed (in Unix time). + - name: fix_link + type: keyword + description: Link to the vendor's fixed-version information. + - name: function_layer + type: keyword + description: Specifies the serverless layer ID in which the vulnerability was discovered. + - name: grace_period_days + type: long + description: Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. + - name: id + type: keyword + description: ID of the violation. + - name: layer_time + type: date + description: Date/time of the image layer to which the CVE belongs. + - name: link + type: keyword + description: Vendor link to the CVE. + - name: package + type: group + fields: + - name: name + type: keyword + description: Name of the package that caused the vulnerability. + - name: version + type: keyword + description: Version of the package that caused the vulnerability (or null). + - name: published + type: date + description: Date/time when the vulnerability was published (in Unix time). + - name: risk_factors + type: flattened + description: RiskFactors maps the existence of vulnerability risk factors. + - name: severity + type: keyword + description: Textual representation of the vulnerability's severity. + - name: status + type: keyword + description: Vendor status for the vulnerability. + - name: text + type: keyword + description: Description of the violation. + - name: title + type: keyword + description: Compliance title. + - name: twistlock + type: boolean + description: Indicates if this is a Twistlock-specific vulnerability (true) or not (false). + - name: type + type: keyword + description: Type represents the vulnerability type. + - name: vec_str + type: keyword + description: Textual representation of the metric values used to score the vulnerability. + - name: vuln_tag_infos + type: group + fields: + - name: color + type: keyword + description: Color is a hexadecimal representation of color code value. + - name: comment + type: keyword + description: Tag comment in a specific vulnerability context. + - name: name + type: keyword + description: Name of the tag. + - name: wildfire_malware + type: group + fields: + - name: md5 + type: keyword + description: MD5 is the hash of the malicious binary. + - name: path + type: keyword + description: Path is the path to malicious binary. + - name: verdict + type: keyword + description: Verdict is the malicious source like grayware, malware and phishing. + - name: compliance_risk_score + type: float + description: Compliance risk score for the image. + - name: creation_time + type: date + description: Specifies the time of creation for the latest version of the image. + - name: devices + type: group + fields: + - name: ip + type: ip + description: Network device IPv4 address. + - name: name + type: keyword + description: Network device name. + - name: distro + type: keyword + description: Full name of the distribution. + - name: ecs_cluster_name + type: keyword + description: ECS cluster name. + - name: err + type: keyword + description: Description of an error that occurred during image health scan. + - name: external_labels + type: group + fields: + - name: key + type: keyword + description: Label key. + - name: source + type: group + fields: + - name: name + type: keyword + description: Source name (e.g., for a namespace, the source name can be 'twistlock'). + - name: type + type: keyword + description: ExternalLabelSourceType indicates the source of the labels. + - name: timestamp + type: keyword + description: Time when the label was fetched. + - name: value + type: keyword + description: Value of the label. + - name: files + type: group + fields: + - name: md5 + type: keyword + description: Hash sum of the file using md5. + - name: path + type: keyword + description: Path of the file. + - name: sha1 + type: keyword + description: Hash sum of the file using SHA-1. + - name: sha256 + type: keyword + description: Hash sum of the file using SHA256. + - name: firewall_protection + type: group + fields: + - name: enabled + type: boolean + description: Enabled indicates if WAAS proxy protection is enabled (true) or not (false). + - name: out_of_band_mode + type: keyword + description: OutOfBandMode holds the app firewall out-of-band mode. + - name: ports + type: long + description: Ports indicates http open ports associated with the container. + - name: supported + type: boolean + description: Supported indicates if WAAS protection is supported (true) or not (false). + - name: tls_ports + type: long + description: TLSPorts indicates https open ports associated with the container. + - name: unprotected_processes + type: group + fields: + - name: port + type: long + description: Port is the process port. + - name: process + type: keyword + description: Process is the process name. + - name: tls + type: boolean + description: TLS is the port TLS indication. + - name: first_scan_time + type: date + description: Specifies the time of the scan for the first version of the image. This time is preserved even after the version update. + - name: history + type: group + fields: + - name: base_layer + type: boolean + description: Indicates if this layer originated from the base image (true) or not (false). + - name: created + type: date + description: Date/time when the image layer was created. + - name: empty_layer + type: boolean + description: Indicates if this instruction didn't create a separate layer (true) or not. + - name: id + type: keyword + description: ID of the layer. + - name: instruction + type: keyword + description: Docker file instruction and arguments used to create this layer. + - name: size_bytes + type: long + description: Size of the layer (in bytes). + - name: tags + type: keyword + description: Holds the image tags. + - name: vulnerabilities + type: group + fields: + - name: applicable_rules + type: keyword + description: Rules applied on the package. + - name: binary_pkgs + type: keyword + description: Names of the distro binary package names (packages which are built from the source of the package). + - name: block + type: boolean + description: Indicates if the vulnerability has a block effect (true) or not (false). + - name: cause + type: keyword + description: Additional information regarding the root cause for the vulnerability. + - name: cri + type: boolean + description: Indicates if this is a CRI-specific vulnerability (true) or not (false). + - name: custom + type: boolean + description: Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). + - name: cve + type: keyword + description: CVE ID of the vulnerability (if applied). + - name: cvss + type: float + description: CVSS score of the vulnerability. + - name: description + type: keyword + description: Description of the vulnerability. + - name: discovered + type: date + description: Specifies the time of discovery for the vulnerability. + - name: exploit + type: keyword + description: ExploitType represents the source of an exploit. + - name: exploits + type: group + fields: + - name: kind + type: keyword + description: ExploitKind represents the kind of the exploit. + - name: link + type: keyword + description: Link is a link to information about the exploit. + - name: source + type: keyword + description: ExploitType represents the source of an exploit. + - name: fix_date + type: date + description: Date/time when the vulnerability was fixed (in Unix time). + - name: fix_link + type: keyword + description: Link to the vendor's fixed-version information. + - name: function_layer + type: keyword + description: Specifies the serverless layer ID in which the vulnerability was discovered. + - name: grace_period_days + type: long + description: Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. + - name: id + type: keyword + description: ID of the violation. + - name: layer_time + type: date + description: Date/time of the image layer to which the CVE belongs. + - name: link + type: keyword + description: Vendor link to the CVE. + - name: package + type: group + fields: + - name: name + type: keyword + description: Name of the package that caused the vulnerability. + - name: version + type: keyword + description: Version of the package that caused the vulnerability (or null). + - name: published + type: date + description: Date/time when the vulnerability was published (in Unix time). + - name: risk_factors + type: flattened + description: RiskFactors maps the existence of vulnerability risk factors. + - name: severity + type: keyword + description: Textual representation of the vulnerability's severity. + - name: status + type: keyword + description: Vendor status for the vulnerability. + - name: templates + type: keyword + description: List of templates with which the vulnerability is associated. + - name: text + type: keyword + description: Description of the violation. + - name: title + type: keyword + description: Compliance title. + - name: twistlock + type: boolean + description: Indicates if this is a Twistlock-specific vulnerability (true) or not (false). + - name: type + type: keyword + description: Type represents the vulnerability type. + - name: vec_str + type: keyword + description: Textual representation of the metric values used to score the vulnerability. + - name: vuln_tag_infos + type: group + fields: + - name: color + type: keyword + description: Color is a hexadecimal representation of color code value. + - name: comment + type: keyword + description: Tag comment in a specific vulnerability context. + - name: name + type: keyword + description: Name of the tag. + - name: wild_fire_malware + type: group + fields: + - name: md5 + type: keyword + description: MD5 is the hash of the malicious binary. + - name: path + type: keyword + description: Path is the path to malicious binary. + - name: verdict + type: keyword + description: Verdict is the malicious source like grayware, malware and phishing. + - name: hostname + type: keyword + description: Name of the host that was scanned. + - name: hosts + type: flattened + description: ImageHosts is a fast index for image scan results metadata per host. + - name: id + type: keyword + description: Image ID. + - name: image + type: group + fields: + - name: created + type: date + description: Date/time when the image was created. + - name: entrypoint + type: keyword + description: Combined entrypoint of the image (entrypoint + CMD). + - name: env + type: keyword + description: Image environment variables. + - name: healthcheck + type: boolean + description: Indicates if health checks are enabled (true) or not (false). + - name: history + type: group + fields: + - name: base_layer + type: boolean + description: Indicates if this layer originated from the base image (true) or not (false). + - name: created + type: date + description: Date/time when the image layer was created. + - name: empty_layer + type: boolean + description: Indicates if this instruction didn't create a separate layer (true) or not. + - name: id + type: keyword + description: ID of the layer. + - name: instruction + type: keyword + description: Docker file instruction and arguments used to create this layer. + - name: size_bytes + type: long + description: Size of the layer (in bytes). + - name: tags + type: keyword + description: Holds the image tags. + - name: vulnerabilities + type: group + fields: + - name: applicable_rules + type: keyword + description: Rules applied on the package. + - name: binaryPkgs + type: keyword + description: Names of the distro binary package names (packages which are built from the source of the package). + - name: block + type: boolean + description: Indicates if the vulnerability has a block effect (true) or not (false). + - name: cause + type: keyword + description: Additional information regarding the root cause for the vulnerability. + - name: cri + type: boolean + description: Indicates if this is a CRI-specific vulnerability (true) or not (false). + - name: custom + type: boolean + description: Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). + - name: cve + type: keyword + description: CVE ID of the vulnerability (if applied). + - name: cvss + type: float + description: CVSS score of the vulnerability. + - name: description + type: keyword + description: Description of the vulnerability. + - name: discovered + type: date + description: Specifies the time of discovery for the vulnerability. + - name: exploit + type: keyword + description: ExploitType represents the source of an exploit. + - name: exploits + type: group + fields: + - name: kind + type: keyword + description: ExploitKind represents the kind of the exploit. + - name: link + type: keyword + description: Link is a link to information about the exploit. + - name: source + type: keyword + description: ExploitType represents the source of an exploit. + - name: fix_date + type: date + description: Date/time when the vulnerability was fixed (in Unix time). + - name: fix_link + type: keyword + description: Link to the vendor's fixed-version information. + - name: function_layer + type: keyword + description: Specifies the serverless layer ID in which the vulnerability was discovered. + - name: grace_period_days + type: long + description: Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. + - name: id + type: keyword + description: ID of the violation. + - name: layer_time + type: date + description: Date/time of the image layer to which the CVE belongs. + - name: link + type: keyword + description: Vendor link to the CVE. + - name: package + type: group + fields: + - name: name + type: keyword + description: Name of the package that caused the vulnerability. + - name: version + type: keyword + description: Version of the package that caused the vulnerability (or null). + - name: published + type: date + description: Date/time when the vulnerability was published (in Unix time). + - name: risk_factors + type: flattened + description: RiskFactors maps the existence of vulnerability risk factors. + - name: severity + type: keyword + description: Textual representation of the vulnerability's severity. + - name: status + type: keyword + description: Vendor status for the vulnerability. + - name: templates + type: keyword + description: List of templates with which the vulnerability is associated. + - name: text + type: keyword + description: Description of the violation. + - name: title + type: keyword + description: Compliance title. + - name: twistlock + type: boolean + description: Indicates if this is a Twistlock-specific vulnerability (true) or not (false). + - name: type + type: keyword + description: Type represents the vulnerability type. + - name: vec_str + type: keyword + description: Textual representation of the metric values used to score the vulnerability. + - name: vuln_tag_infos + type: group + fields: + - name: color + type: keyword + description: Color is a hexadecimal representation of color code value. + - name: comment + type: keyword + description: Tag comment in a specific vulnerability context. + - name: name + type: keyword + description: Name of the tag. + - name: wild_fire_malware + type: group + fields: + - name: md5 + type: keyword + description: MD5 is the hash of the malicious binary. + - name: path + type: keyword + description: Path is the path to malicious binary. + - name: verdict + type: keyword + description: Verdict is the malicious source like grayware, malware and phishing. + - name: id + type: keyword + description: ID of the image. + - name: labels + type: flattened + description: Image labels. + - name: layers + type: keyword + description: Image filesystem layers. + - name: os + type: keyword + description: Image os type. + - name: repo + type: group + fields: + - name: digest + type: keyword + description: Image repo digests. + - name: tags + type: keyword + description: Image repo tags. + - name: user + type: keyword + description: Image user. + - name: working_dir + type: keyword + description: Base working directory of the image. + - name: installed_products + type: group + fields: + - name: agentless + type: boolean + description: Agentless indicates whether the scan was performed with agentless approach. + - name: apache + type: keyword + description: Apache indicates the apache server version, empty in case apache not running. + - name: aws_cloud + type: boolean + description: AWSCloud indicates whether AWS cloud is used. + - name: cluster_type + type: keyword + description: ClusterType is the cluster type. + - name: crio + type: boolean + description: CRI indicates whether the container runtime is CRI (and not docker). + - name: docker + type: keyword + description: Docker represents the docker daemon version. + - name: docker_enterprise + type: boolean + description: DockerEnterprise indicates whether the enterprise version of Docker is installed. + - name: has_package_manager + type: boolean + description: HasPackageManager indicates whether package manager is installed on the OS. + - name: k8s_api_server + type: boolean + description: K8sAPIServer indicates whether a kubernetes API server is running. + - name: k8s_controller_manager + type: boolean + description: K8sControllerManager indicates whether a kubernetes controller manager is running. + - name: k8s_etcd + type: boolean + description: K8sEtcd indicates whether etcd is running. + - name: k8s_federation_api_server + type: boolean + description: K8sFederationAPIServer indicates whether a federation API server is running. + - name: k8s_federation_controller_manager + type: boolean + description: K8sFederationControllerManager indicates whether a federation controller manager is running. + - name: k8s_kubelet + type: boolean + description: K8sKubelet indicates whether kubelet is running. + - name: k8s_proxy + type: boolean + description: K8sProxy indicates whether a kubernetes proxy is running. + - name: k8s_scheduler + type: boolean + description: K8sScheduler indicates whether the kubernetes scheduler is running. + - name: kubernetes + type: keyword + description: Kubernetes represents the kubernetes version. + - name: managed_cluster_version + type: keyword + description: ManagedClusterVersion is the version of the managed Kubernetes service, e.g. AKS/EKS/GKE/etc. + - name: openshift + type: boolean + description: Openshift indicates whether openshift is deployed. + - name: openshift_version + type: keyword + description: OpenshiftVersion represents the running openshift version. + - name: os_distro + type: keyword + description: OSDistro specifies the os distribution. + - name: serverless + type: boolean + description: Serverless indicates whether evaluated on a serverless environment. + - name: swarm + type: group + fields: + - name: manager + type: boolean + description: SwarmManager indicates whether a swarm manager is running. + - name: node + type: boolean + description: SwarmNode indicates whether the node is part of an active swarm. + - name: instances + type: group + fields: + - name: host + type: keyword + - name: image + type: keyword + - name: modified + type: date + - name: registry + type: keyword + - name: repo + type: keyword + - name: tag + type: keyword + - name: is_arm64 + type: boolean + description: IsARM64 indicates if the architecture of the image is aarch64. + - name: k8s_cluster_addr + type: keyword + description: Endpoint of the Kubernetes API server. + - name: labels + type: keyword + description: Image labels. + - name: malware_analyzed_time + type: date + description: MalwareAnalyzedTime is the WildFire evaluator analyzing time shown as progress in UI and cannot to be overwritten by a new scan result. + - name: missing_distro_vuln_coverage + type: boolean + description: Indicates if the image OS is covered in the IS (true) or not (false). + - name: namespaces + type: keyword + description: k8s namespaces of all the containers running this image. + - name: os_distro + type: group + fields: + - name: release + type: keyword + description: OS distribution release. + - name: value + type: keyword + description: Name of the OS distribution. + - name: version + type: keyword + description: OS distribution version. + - name: package + type: group + fields: + - name: correlation_done + type: boolean + description: PackageCorrelationDone indicates that the correlation to OS packages has been done. + - name: manager + type: boolean + description: Indicates if the package manager is installed for the OS. + - name: packages + type: group + fields: + - name: pkgs + type: group + fields: + - name: binary_idx + type: long + description: Indexes of the top binaries which use the package. + - name: binary_pkgs + type: keyword + description: Names of the distro binary packages (packages which are built on the source of the package). + - name: cve_count + type: long + description: Total number of CVEs for this specific package. + - name: default_gem + type: boolean + description: DefaultGem indicates this is a gem default package (and not a bundled package). + - name: files + type: group + fields: + - name: md5 + type: keyword + description: Hash sum of the file using md5. + - name: path + type: keyword + description: Path of the file. + - name: sha1 + type: keyword + description: Hash sum of the file using SHA-1. + - name: sha256 + type: keyword + description: Hash sum of the file using SHA256. + - name: function_layer + type: keyword + description: ID of the serverless layer in which the package was discovered. + - name: go_pkg + type: boolean + description: GoPkg indicates this is a Go package (and not module). + - name: jar_identifier + type: keyword + description: JarIdentifier holds an additional identification detail of a JAR package. + - name: layer_time + type: date + description: Image layer to which the package belongs (layer creation time). + - name: license + type: keyword + description: License information for the package. + - name: name + type: keyword + description: Name of the package. + - name: os_package + type: boolean + description: OSPackage indicates that a python/java package was installed as an OS package. + - name: path + type: keyword + description: Full package path (e.g., JAR or Node.js package path). + - name: version + type: keyword + description: Package version. + - name: pkgs_type + type: keyword + description: PackageType describes the package type. + - name: pull_duration + type: long + description: PullDuration is the time it took to pull the image. + - name: push_time + type: date + description: PushTime is the image push time to the registry. + - name: red_hat_non_rpm_image + type: boolean + description: RedHatNonRPMImage indicates whether the image is a Red Hat image with non-RPM content. + - name: registry + type: group + fields: + - name: namespace + type: keyword + description: IBM cloud namespace to which the image belongs. + - name: tags + type: keyword + description: RegistryTags are the tags of the registry this image is stored. + - name: type + type: keyword + description: RegistryType indicates the registry type where the image is stored. + - name: repo_digests + type: keyword + description: Digests of the image. Used for content trust (notary). Has one digest per tag. + - name: repo_tag + type: group + fields: + - name: digest + type: keyword + description: Image digest (requires V2 or later registry). + - name: id + type: keyword + description: ID of the image. + - name: registry + type: keyword + description: Registry name to which the image belongs. + - name: repo + type: keyword + description: Repository name to which the image belongs. + - name: value + type: keyword + description: Image tag. + - name: rhel_repos + type: keyword + description: RhelRepositories are the (RPM) repositories IDs from which the packages in this image were installed Used for matching vulnerabilities by Red Hat CPEs. + - name: risk_factors + type: flattened + description: RiskFactors maps the existence of vulnerability risk factors. + - name: runtime_enabled + type: boolean + description: HostRuntimeEnabled indicates if any runtime rule applies to the host. + - name: scan + type: group + fields: + - name: build_date + type: date + description: Scanner build date that published the image. + - name: duration + type: long + description: ScanDuration is the total time it took to scan the image. + - name: id + type: keyword + description: ScanID is the ID of the scan. + - name: time + type: date + description: Specifies the time of the last scan of the image. + - name: version + type: keyword + description: Scanner version that published the image. + - name: secrets + type: keyword + description: 'Secrets are paths to embedded secrets inside the image Note: capital letter JSON annotation is kept to avoid converting all images for backward-compatibility support.' + - name: startup_binaries + type: group + fields: + - name: altered + type: boolean + description: Indicates if the binary was installed from a package manager and modified/replaced (true) or not (false). + - name: cve_count + type: long + description: Total number of CVEs for this specific binary. + - name: deps + type: keyword + description: Third-party package files which are used by the binary. + - name: file_mode + type: long + description: Represents the file's mode and permission bits. + - name: function_layer + type: keyword + description: ID of the serverless layer in which the package was discovered. + - name: md5 + type: keyword + description: Md5 hashset of the binary. + - name: missing_pkg + type: boolean + description: Indicates if this binary is not related to any package (true) or not (false). + - name: name + type: keyword + description: Name of the binary. + - name: path + type: keyword + description: Path is the path of the binary. + - name: pkg_root_dir + type: keyword + description: Path for searching packages used by the binary. + - name: services + type: keyword + description: Names of services which use the binary. + - name: version + type: keyword + description: Version of the binary. + - name: stopped + type: boolean + description: Stopped indicates whether the host was running during the agentless scan. + - name: tags + type: group + fields: + - name: digest + type: keyword + description: Image digest (requires V2 or later registry). + - name: id + type: keyword + description: ID of the image. + - name: registry + type: keyword + description: Registry name to which the image belongs. + - name: repo + type: keyword + description: Repository name to which the image belongs. + - name: tag + type: keyword + description: Image tag. + - name: top_layer + type: keyword + description: SHA256 of the image's last layer that is the last element of the Layers field. + - name: trust_result + type: group + fields: + - name: groups + type: group + fields: + - name: _id + type: keyword + description: Name of the group. + - name: disabled + type: boolean + description: Indicates if the rule is currently disabled (true) or not (false). + - name: images + type: keyword + description: Image names or IDs (e.g., docker.io/library/ubuntu:16.04 / SHA264@...). + - name: layers + type: keyword + description: Filesystem layers. The image is trusted if its layers have a prefix of the trusted groups layer in the same order. + - name: modified + type: date + description: Datetime when the rule was last modified. + - name: name + type: keyword + description: Name of the rule. + - name: notes + type: keyword + description: Free-form text. + - name: owner + type: keyword + description: User who created or last modified the rule. + - name: previous_name + type: keyword + description: Previous name of the rule. Required for rule renaming. + - name: hosts_statuses + type: group + fields: + - name: host + type: keyword + description: Host name. + - name: status + type: keyword + description: Status is the trust status for an image. + - name: trust_status + type: keyword + description: Status is the trust status for an image. + - name: twistlock_image + type: boolean + description: Indicates if the image is a Twistlock image (true) or not (false). + - name: type + type: keyword + description: ScanType displays the components for an ongoing scan. + - name: vulnerabilities + type: group + fields: + - name: count + type: long + description: Total number of vulnerabilities. + - name: data + type: group + fields: + - name: applicable_rules + type: keyword + description: Rules applied on the package. + - name: binary_pkgs + type: keyword + description: Names of the distro binary package names (packages which are built from the source of the package). + - name: block + type: boolean + description: Indicates if the vulnerability has a block effect (true) or not (false). + - name: cause + type: keyword + description: Additional information regarding the root cause for the vulnerability. + - name: cri + type: boolean + description: Indicates if this is a CRI-specific vulnerability (true) or not (false). + - name: custom + type: boolean + description: Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). + - name: cve + type: keyword + description: CVE ID of the vulnerability (if applied). + - name: cvss + type: float + description: CVSS score of the vulnerability. + - name: description + type: keyword + description: Description of the vulnerability. + - name: discovered + type: date + description: Specifies the time of discovery for the vulnerability. + - name: exploit + type: keyword + description: ExploitType represents the source of an exploit. + - name: exploits + type: group + fields: + - name: kind + type: keyword + description: ExploitKind represents the kind of the exploit. + - name: source + type: keyword + description: ExploitType represents the source of an exploit. + - name: fix_date + type: date + description: Date/time when the vulnerability was fixed (in Unix time). + - name: fix_link + type: keyword + description: Link to the vendor's fixed-version information. + - name: function_layer + type: keyword + description: Specifies the serverless layer ID in which the vulnerability was discovered. + - name: grace_period_days + type: long + description: Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. + - name: id + type: keyword + description: ID of the violation. + - name: layer_time + type: date + description: Date/time of the image layer to which the CVE belongs. + - name: link + type: keyword + description: Vendor link to the CVE. + - name: package + type: group + fields: + - name: name + type: keyword + description: Name of the package that caused the vulnerability. + - name: version + type: keyword + description: Version of the package that caused the vulnerability (or null). + - name: published + type: date + description: Date/time when the vulnerability was published (in Unix time). + - name: risk_factors + type: flattened + description: RiskFactors maps the existence of vulnerability risk factors. + - name: severity + type: keyword + description: Textual representation of the vulnerability's severity. + - name: status + type: keyword + description: Vendor status for the vulnerability. + - name: templates + type: keyword + description: List of templates with which the vulnerability is associated. + - name: text + type: keyword + description: Description of the violation. + - name: title + type: keyword + - name: twistlock + type: boolean + description: Indicates if this is a Twistlock-specific vulnerability (true) or not (false). + - name: type + type: keyword + description: Type represents the vulnerability type. + - name: vec_str + type: keyword + description: Textual representation of the metric values used to score the vulnerability. + - name: vuln_tag_infos + type: group + fields: + - name: color + type: keyword + description: Color is a hexadecimal representation of color code value. + - name: comment + type: keyword + description: Tag comment in a specific vulnerability context. + - name: name + type: keyword + description: Name of the tag. + - name: wild_fire_malware + type: group + fields: + - name: md5 + type: keyword + description: MD5 is the hash of the malicious binary. + - name: path + type: keyword + description: Path is the path to malicious binary. + - name: verdict + type: keyword + description: Verdict is the malicious source like grayware, malware and phishing. + - name: vulnerability + type: group + fields: + - name: distribution + type: group + fields: + - name: critical + type: long + - name: high + type: long + - name: low + type: long + - name: medium + type: long + - name: total + type: long + - name: risk_score + type: long + description: Image's CVE risk score. + - name: wild_fire_usage + type: group + fields: + - name: bytes + type: long + description: Bytes is the total number of bytes uploaded to the WildFire API. + - name: queries + type: long + description: Queries is the number of queries to the WildFire API. + - name: uploads + type: long + description: Uploads is the number of uploads to the WildFire API. +- name: log.source.address + type: keyword + description: Source address from which the log event was read / sent from. diff --git a/packages/prisma_cloud/data_stream/host/manifest.yml b/packages/prisma_cloud/data_stream/host/manifest.yml new file mode 100644 index 00000000000..e7e97a6dca8 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/manifest.yml @@ -0,0 +1,219 @@ +title: Collect Host logs from Prisma Cloud Workload Protection. +type: logs +streams: + - input: cel + title: Host Logs + description: Collect Host logs from Prisma Cloud Workload Protection. + template_path: input.yml.hbs + enabled: false + vars: + - name: url + type: text + title: URL + description: Base URL of the Prisma Cloud Server API. + required: true + show_user: true + - name: interval + type: text + title: Interval + description: Interval between two REST API calls. Supported units for this parameter are h/m/s. + default: 1m + multi: false + required: true + show_user: true + - name: offset + type: integer + title: Offset + description: Offsets the result to a specific report count. Offset starts from 0. + default: 0 + multi: false + required: true + show_user: false + - name: batch_size + type: integer + title: Batch Size + description: Number of reports to retrieve in a page. For PCCE, the maximum limit is 250. For PCEE, the maximum limit is 50. + default: 50 + multi: false + required: true + show_user: false + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Supported time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + required: false + show_user: false + description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-host + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.host fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: tcp + template_path: tcp.yml.hbs + title: Host logs + description: Collect Host logs via TCP input. + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for TCP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The TCP port number to listen on. + multi: false + required: true + show_user: true + default: 9508 + - name: tcp_options + type: yaml + title: Custom TCP Options + multi: false + required: false + show_user: false + default: | + #max_message_size: 20MiB + #max_connections: 1 + #framing: delimiter + #line_delimiter: "\n" + description: Specify custom configuration options for the TCP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-host + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.host fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + template_path: udp.yml.hbs + title: Host logs + description: Collect Host logs via UDP input. + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for UDP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The UDP port number to listen on. + multi: false + required: true + show_user: true + default: 9508 + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + #max_message_size: 10KiB + #timeout: 300s + description: Specify custom configuration options for the UDP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-host + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.host fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/prisma_cloud/data_stream/host/sample_event.json b/packages/prisma_cloud/data_stream/host/sample_event.json new file mode 100644 index 00000000000..d55c9a2aeec --- /dev/null +++ b/packages/prisma_cloud/data_stream/host/sample_event.json @@ -0,0 +1,530 @@ +{ + "@timestamp": "2023-10-18T12:12:26.324Z", + "agent": { + "ephemeral_id": "b495d34c-84f2-4dde-abdf-838c08e654af", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "cloud": { + "account": { + "id": "Non-onboarded cloud accounts" + }, + "instance": { + "id": "string", + "name": "string" + }, + "machine": { + "type": "string" + }, + "provider": [ + "aws" + ], + "region": "string" + }, + "data_stream": { + "dataset": "prisma_cloud.host", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "host" + ], + "dataset": "prisma_cloud.host", + "id": "DESKTOP-6PQXXMS", + "ingested": "2023-10-18T12:12:27Z", + "kind": "event", + "original": "{\"Secrets\":[],\"_id\":\"DESKTOP-6PQXXMS\",\"agentless\":false,\"allCompliance\":{\"compliance\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.949Z\",\"exploit\":[\"exploit-db\"],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}],\"enabled\":\"true\"},\"appEmbedded\":false,\"applications\":[{\"installedFromPackage\":true,\"knownVulnerabilities\":0,\"layerTime\":0,\"name\":\"string\",\"path\":\"string\",\"service\":true,\"version\":\"string\"}],\"binaries\":[{\"altered\":true,\"cveCount\":0,\"deps\":[\"string\"],\"fileMode\":0,\"functionLayer\":\"string\",\"md5\":\"string\",\"missingPkg\":true,\"name\":\"string\",\"path\":\"string\",\"pkgRootDir\":\"string\",\"services\":[\"string\"],\"version\":\"string\"}],\"cloudMetadata\":{\"accountID\":\"Non-onboarded cloud accounts\",\"awsExecutionEnv\":\"string\",\"image\":\"string\",\"labels\":[{\"key\":\"string\",\"sourceName\":\"string\",\"sourceType\":[\"namespace\"],\"timestamp\":\"2023-09-08T04:01:49.949Z\",\"value\":\"string\"}],\"name\":\"string\",\"provider\":[\"aws\"],\"region\":\"string\",\"resourceID\":\"string\",\"resourceURL\":\"string\",\"type\":\"string\",\"vmID\":\"string\",\"vmImageID\":\"string\"},\"clusters\":[\"string\"],\"collections\":[\"All\"],\"complianceDistribution\":{\"critical\":4,\"high\":0,\"low\":0,\"medium\":0,\"total\":4},\"complianceIssuesCount\":4,\"complianceRiskScore\":4000000,\"creationTime\":\"0001-01-01T00:00:00Z\",\"distro\":\"Microsoft Windows [Version 10.0.19045.2006]\",\"err\":\"\",\"externalLabels\":[{\"key\":\"string\",\"sourceName\":\"string\",\"sourceType\":[\"namespace\"],\"timestamp\":\"2023-09-08T04:01:49.949Z\",\"value\":\"string\"}],\"files\":[{\"md5\":\"string\",\"path\":\"string\",\"sha1\":\"string\",\"sha256\":\"string\"}],\"firewallProtection\":{\"enabled\":false,\"outOfBandMode\":\"Observation\",\"ports\":[0],\"supported\":false,\"tlsPorts\":[0],\"unprotectedProcesses\":[{\"port\":0,\"process\":\"string\",\"tls\":true}]},\"firstScanTime\":\"2023-08-11T06:53:57.456Z\",\"history\":[{\"baseLayer\":true,\"created\":0,\"emptyLayer\":true,\"id\":\"string\",\"instruction\":\"string\",\"sizeBytes\":0,\"tags\":[\"string\"],\"vulnerabilities\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.950Z\",\"exploit\":[\"exploit-db\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}]}],\"hostDevices\":[{\"ip\":\"0.0.0.0\",\"name\":\"string\"}],\"hostname\":\"DESKTOP-6PQXXMS\",\"hosts\":{},\"id\":\"string\",\"image\":{\"created\":\"0001-01-01T00:00:00Z\",\"entrypoint\":[\"string\"],\"env\":[\"string\"],\"healthcheck\":true,\"id\":\"string\",\"labels\":{},\"layers\":[\"string\"],\"os\":\"string\",\"repoDigest\":[\"string\"],\"repoTags\":[\"string\"],\"user\":\"string\",\"workingDir\":\"string\"},\"instances\":[{\"host\":\"string\",\"image\":\"string\",\"modified\":\"2023-09-08T04:01:49.951Z\",\"registry\":\"string\",\"repo\":\"string\",\"tag\":\"string\"}],\"isARM64\":false,\"malwareAnalyzedTime\":\"0001-01-01T00:00:00Z\",\"osDistro\":\"windows\",\"osDistroRelease\":\"Windows\",\"osDistroVersion\":\"string\",\"packageCorrelationDone\":true,\"packageManager\":true,\"packages\":[{\"pkgs\":[{\"binaryIdx\":[0],\"binaryPkgs\":[\"string\"],\"cveCount\":0,\"defaultGem\":true,\"files\":[{\"md5\":\"string\",\"path\":\"string\",\"sha1\":\"string\",\"sha256\":\"string\"}],\"functionLayer\":\"string\",\"goPkg\":true,\"jarIdentifier\":\"string\",\"layerTime\":0,\"license\":\"string\",\"name\":\"string\",\"osPackage\":true,\"path\":\"string\",\"version\":\"string\"}],\"pkgsType\":\"nodejs\"}],\"pushTime\":\"0001-01-01T00:00:00Z\",\"redHatNonRPMImage\":false,\"repoDigests\":[],\"repoTag\":null,\"riskFactors\":{},\"scanID\":0,\"scanTime\":\"2023-08-23T11:48:41.803Z\",\"tags\":[{\"digest\":\"string\",\"id\":\"string\",\"registry\":\"string\",\"repo\":\"string\",\"tag\":\"string\"}],\"trustResult\":{\"hostsStatuses\":[{\"host\":\"string\",\"status\":\"trusted\"}]},\"trustStatus\":\"\",\"type\":\"host\",\"vulnerabilitiesCount\":0,\"vulnerabilityDistribution\":{\"critical\":0,\"high\":0,\"low\":0,\"medium\":0,\"total\":0},\"vulnerabilityRiskScore\":0,\"wildFireUsage\":null}", + "start": "0001-01-01T00:00:00.000Z", + "type": [ + "info" + ] + }, + "file": { + "hash": { + "md5": [ + "string" + ], + "sha1": [ + "string" + ], + "sha256": [ + "string" + ] + }, + "path": [ + "string" + ] + }, + "host": { + "hostname": "DESKTOP-6PQXXMS", + "ip": [ + "0.0.0.0" + ], + "type": "host" + }, + "input": { + "type": "cel" + }, + "os": { + "family": "windows", + "name": "Windows", + "version": "string" + }, + "package": { + "license": [ + "string" + ], + "name": [ + "string" + ], + "path": [ + "string" + ], + "type": [ + "nodejs" + ], + "version": [ + "string" + ] + }, + "prisma_cloud": { + "host": { + "_id": "DESKTOP-6PQXXMS", + "agentless": false, + "all_compliance": { + "data": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0, + "description": "string", + "discovered": "2023-09-08T04:01:49.949Z", + "exploit": [ + "exploit-db" + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ], + "enabled": true + }, + "app_embedded": false, + "applications": [ + { + "installed_from_package": true, + "known_vulnerabilities": 0, + "layer_time": "1970-01-01T00:00:00.000Z", + "name": "string", + "path": "string", + "service": true, + "version": "string" + } + ], + "binaries": [ + { + "altered": true, + "cve_count": 0, + "deps": [ + "string" + ], + "file_mode": 0, + "function_layer": "string", + "md5": "string", + "missing_pkg": true, + "name": "string", + "path": "string", + "pkg_root_dir": "string", + "services": [ + "string" + ], + "version": "string" + } + ], + "cloud_metadata": { + "account_id": "Non-onboarded cloud accounts", + "aws_execution_env": "string", + "image": "string", + "labels": [ + { + "key": "string", + "source": { + "name": "string", + "type": [ + "namespace" + ] + }, + "timestamp": "2023-09-08T04:01:49.949Z", + "value": "string" + } + ], + "name": "string", + "provider": [ + "aws" + ], + "region": "string", + "resource": { + "id": "string", + "url": "string" + }, + "type": "string", + "vm": { + "id": "string", + "image_id": "string" + } + }, + "clusters": [ + "string" + ], + "collections": [ + "All" + ], + "compliance_distribution": { + "critical": 4, + "high": 0, + "low": 0, + "medium": 0, + "total": 4 + }, + "compliance_issues": { + "count": 4 + }, + "compliance_risk_score": 4000000, + "creation_time": "0001-01-01T00:00:00.000Z", + "devices": [ + { + "ip": "0.0.0.0", + "name": "string" + } + ], + "distro": "Microsoft Windows [Version 10.0.19045.2006]", + "external_labels": [ + { + "key": "string", + "source": { + "name": "string", + "type": [ + "namespace" + ] + }, + "timestamp": "2023-09-08T04:01:49.949Z", + "value": "string" + } + ], + "files": [ + { + "md5": "string", + "path": "string", + "sha1": "string", + "sha256": "string" + } + ], + "firewall_protection": { + "enabled": false, + "out_of_band_mode": "Observation", + "ports": [ + 0 + ], + "supported": false, + "tls_ports": [ + 0 + ], + "unprotected_processes": [ + { + "port": 0, + "process": "string", + "tls": true + } + ] + }, + "first_scan_time": "2023-08-11T06:53:57.456Z", + "history": [ + { + "base_layer": true, + "created": "1970-01-01T00:00:00.000Z", + "empty_layer": true, + "id": "string", + "instruction": "string", + "size_bytes": 0, + "tags": [ + "string" + ], + "vulnerabilities": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0, + "description": "string", + "discovered": "2023-09-08T04:01:49.950Z", + "exploit": [ + "exploit-db" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ] + } + ], + "hostname": "DESKTOP-6PQXXMS", + "id": "string", + "image": { + "created": "0001-01-01T00:00:00.000Z", + "entrypoint": [ + "string" + ], + "env": [ + "string" + ], + "healthcheck": true, + "id": "string", + "layers": [ + "string" + ], + "os": "string", + "repo": { + "digest": [ + "string" + ], + "tags": [ + "string" + ] + }, + "user": "string", + "working_dir": "string" + }, + "instances": [ + { + "host": "string", + "image": "string", + "modified": "2023-09-08T04:01:49.951Z", + "registry": "string", + "repo": "string", + "tag": "string" + } + ], + "is_arm64": false, + "malware_analyzed_time": "0001-01-01T00:00:00.000Z", + "os_distro": { + "release": "Windows", + "value": "windows", + "version": "string" + }, + "package": { + "correlation_done": true, + "manager": true + }, + "packages": [ + { + "pkgs": [ + { + "binary_idx": [ + 0 + ], + "binary_pkgs": [ + "string" + ], + "cve_count": 0, + "default_gem": true, + "files": [ + { + "md5": "string", + "path": "string", + "sha1": "string", + "sha256": "string" + } + ], + "function_layer": "string", + "go_pkg": true, + "jar_identifier": "string", + "layer_time": "1970-01-01T00:00:00.000Z", + "license": "string", + "name": "string", + "os_package": true, + "path": "string", + "version": "string" + } + ], + "pkgs_type": "nodejs" + } + ], + "push_time": "0001-01-01T00:00:00.000Z", + "red_hat_non_rpm_image": false, + "scan": { + "time": "2023-08-23T11:48:41.803Z" + }, + "tags": [ + { + "digest": "string", + "id": "string", + "registry": "string", + "repo": "string", + "tag": "string" + } + ], + "trust_result": { + "hosts_statuses": [ + { + "host": "string", + "status": "trusted" + } + ] + }, + "type": "host", + "vulnerabilities": { + "count": 0 + }, + "vulnerability": { + "distribution": { + "critical": 0, + "high": 0, + "low": 0, + "medium": 0, + "total": 0 + }, + "risk_score": 0 + } + } + }, + "related": { + "hash": [ + "string" + ], + "hosts": [ + "string", + "DESKTOP-6PQXXMS" + ], + "ip": [ + "0.0.0.0" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-host" + ], + "vulnerability": { + "description": [ + "string" + ], + "id": [ + "string" + ], + "severity": [ + "string" + ] + } +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-common-config.yml b/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-host-profile.log b/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-host-profile.log new file mode 100644 index 00000000000..84596e8f70f --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-host-profile.log @@ -0,0 +1 @@ +{"_id":"localhost","hash":9,"created":"2023-09-05T09:19:52.991Z","time":"0001-01-01T00:00:00Z","apps":[{"name":"ServiceUser","startupProcess":{"user":"root","path":"\/opt\/Service\/ServiceUser\/ServiceUser","command":"\/opt\/Service\/ServiceUser\/ServiceUser -d","ppath":"\/usr\/lib\/systemd\/systemd","md5":"aaa","time":"2023-09-05T09:19:54.613Z"},"processes":[{"user":"root","interactive": true, "path":"\/opt\/Service\/ServiceUser\/ServiceUser","command":"\/opt\/Service\/ServiceUser\/ServiceUser -d","ppath":"\/usr\/lib\/systemd\/systemd","md5":"i98ujh","time":"2023-09-05T09:19:54.613Z"}],"outgoingPorts":[{"port":25,"processPath":"\/opt\/Service\/ServiceUser\/python38\/bin\/pybin","command":"\/opt\/Service\/ServiceUser\/python38\/bin\/pybin -E ..\/..\/Tools\/tantls\/py\/sslserveraudit\/auditpackage.py ","modified":"2023-09-05T09:48:28.472Z","ip":"127.0.0.1"}],"listeningPorts":[{"port":17473,"processPath":"\/opt\/Service\/ServiceUser\/ServiceUser","command":"\/opt\/Service\/ServiceUser\/ServiceUser","modified":"2023-09-05T09:20:42.893Z"}]},{"name":"filebeat","startupProcess":{"user":"root","path":"\/usr\/share\/filebeat\/bin\/filebeat","command":"\/usr\/share\/filebeat\/bin\/filebeat --environment systemd -c \/etc\/filebeat\/filebeat.yml --path.home \/us","ppath":"\/usr\/lib\/systemd\/systemd","md5":"mnj87","time":"2023-09-05T09:21:13.92Z", "interactive": true, "modified": true},"processes":[{"user":"root","path":"\/usr\/share\/filebeat\/bin\/filebeat","command":"\/usr\/share\/filebeat\/bin\/filebeat --environment systemd -c \/etc\/filebeat\/filebeat.yml --path.home \/us","ppath":"\/usr\/lib\/systemd\/systemd","md5":"kjj","time":"2023-09-05T09:21:13.92Z", "modified": true}],"listeningPorts":[{"port":9001,"processPath":"\/usr\/share\/filebeat\/bin\/filebeat","command":"\/usr\/share\/filebeat\/bin\/filebeat","modified":"2023-09-05T09:21:16.37Z"}]}],"sshEvents":[{"loginTime":1694016061,"ip":"0.0.0.0","user":"root","path":"\/usr\/libexec\/docker\/cli-plugins\/docker-app","command":"\/usr\/libexec\/docker\/cli-plugins\/docker-app docker-cli-plugin-metadata - High rate of events, throttling started","ppath":"\/usr\/bin\/docker","md5":"dhdj","time":"2023-09-06T16:05:43.768Z","interactive":false, "modified": true}],"geoip":{"countries":[{"ip":"0.0.0.0","code":"US","modified":"2023-09-06T16:03:30.578Z"},{"ip":"81.2.69.142","code":"CA","modified":"2023-09-06T16:03:30.229Z"}],"modified":"2023-09-06T16:03:30.703Z"},"labels":["osDistro:redhat","osVersion:7"],"accountID":"Non-onboarded cloud accounts","collections":["All"]} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-host-profile.log-expected.json b/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-host-profile.log-expected.json new file mode 100644 index 00000000000..7d56761aac4 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/_dev/test/pipeline/test-host-profile.log-expected.json @@ -0,0 +1,192 @@ +{ + "expected": [ + { + "cloud": { + "account": { + "id": "Non-onboarded cloud accounts" + } + }, + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "host" + ], + "created": "2023-09-05T09:19:52.991Z", + "kind": "asset", + "original": "{\"_id\":\"localhost\",\"hash\":9,\"created\":\"2023-09-05T09:19:52.991Z\",\"time\":\"0001-01-01T00:00:00Z\",\"apps\":[{\"name\":\"ServiceUser\",\"startupProcess\":{\"user\":\"root\",\"path\":\"\\/opt\\/Service\\/ServiceUser\\/ServiceUser\",\"command\":\"\\/opt\\/Service\\/ServiceUser\\/ServiceUser -d\",\"ppath\":\"\\/usr\\/lib\\/systemd\\/systemd\",\"md5\":\"aaa\",\"time\":\"2023-09-05T09:19:54.613Z\"},\"processes\":[{\"user\":\"root\",\"interactive\": true, \"path\":\"\\/opt\\/Service\\/ServiceUser\\/ServiceUser\",\"command\":\"\\/opt\\/Service\\/ServiceUser\\/ServiceUser -d\",\"ppath\":\"\\/usr\\/lib\\/systemd\\/systemd\",\"md5\":\"i98ujh\",\"time\":\"2023-09-05T09:19:54.613Z\"}],\"outgoingPorts\":[{\"port\":25,\"processPath\":\"\\/opt\\/Service\\/ServiceUser\\/python38\\/bin\\/pybin\",\"command\":\"\\/opt\\/Service\\/ServiceUser\\/python38\\/bin\\/pybin -E ..\\/..\\/Tools\\/tantls\\/py\\/sslserveraudit\\/auditpackage.py \",\"modified\":\"2023-09-05T09:48:28.472Z\",\"ip\":\"127.0.0.1\"}],\"listeningPorts\":[{\"port\":17473,\"processPath\":\"\\/opt\\/Service\\/ServiceUser\\/ServiceUser\",\"command\":\"\\/opt\\/Service\\/ServiceUser\\/ServiceUser\",\"modified\":\"2023-09-05T09:20:42.893Z\"}]},{\"name\":\"filebeat\",\"startupProcess\":{\"user\":\"root\",\"path\":\"\\/usr\\/share\\/filebeat\\/bin\\/filebeat\",\"command\":\"\\/usr\\/share\\/filebeat\\/bin\\/filebeat --environment systemd -c \\/etc\\/filebeat\\/filebeat.yml --path.home \\/us\",\"ppath\":\"\\/usr\\/lib\\/systemd\\/systemd\",\"md5\":\"mnj87\",\"time\":\"2023-09-05T09:21:13.92Z\", \"interactive\": true, \"modified\": true},\"processes\":[{\"user\":\"root\",\"path\":\"\\/usr\\/share\\/filebeat\\/bin\\/filebeat\",\"command\":\"\\/usr\\/share\\/filebeat\\/bin\\/filebeat --environment systemd -c \\/etc\\/filebeat\\/filebeat.yml --path.home \\/us\",\"ppath\":\"\\/usr\\/lib\\/systemd\\/systemd\",\"md5\":\"kjj\",\"time\":\"2023-09-05T09:21:13.92Z\", \"modified\": true}],\"listeningPorts\":[{\"port\":9001,\"processPath\":\"\\/usr\\/share\\/filebeat\\/bin\\/filebeat\",\"command\":\"\\/usr\\/share\\/filebeat\\/bin\\/filebeat\",\"modified\":\"2023-09-05T09:21:16.37Z\"}]}],\"sshEvents\":[{\"loginTime\":1694016061,\"ip\":\"0.0.0.0\",\"user\":\"root\",\"path\":\"\\/usr\\/libexec\\/docker\\/cli-plugins\\/docker-app\",\"command\":\"\\/usr\\/libexec\\/docker\\/cli-plugins\\/docker-app docker-cli-plugin-metadata - High rate of events, throttling started\",\"ppath\":\"\\/usr\\/bin\\/docker\",\"md5\":\"dhdj\",\"time\":\"2023-09-06T16:05:43.768Z\",\"interactive\":false, \"modified\": true}],\"geoip\":{\"countries\":[{\"ip\":\"0.0.0.0\",\"code\":\"US\",\"modified\":\"2023-09-06T16:03:30.578Z\"},{\"ip\":\"81.2.69.142\",\"code\":\"CA\",\"modified\":\"2023-09-06T16:03:30.229Z\"}],\"modified\":\"2023-09-06T16:03:30.703Z\"},\"labels\":[\"osDistro:redhat\",\"osVersion:7\"],\"accountID\":\"Non-onboarded cloud accounts\",\"collections\":[\"All\"]}", + "type": [ + "info" + ] + }, + "host": { + "hostname": "localhost" + }, + "labels": [ + "osDistro:redhat", + "osVersion:7" + ], + "prisma_cloud": { + "host_profile": { + "_id": "localhost", + "account_id": "Non-onboarded cloud accounts", + "apps": [ + { + "listening_ports": [ + { + "command": "/opt/Service/ServiceUser/ServiceUser", + "modified": "2023-09-05T09:20:42.893Z", + "port": 17473, + "process_path": "/opt/Service/ServiceUser/ServiceUser" + } + ], + "name": "ServiceUser", + "outgoing_ports": [ + { + "command": "/opt/Service/ServiceUser/python38/bin/pybin -E ../../Tools/tantls/py/sslserveraudit/auditpackage.py ", + "ip": "127.0.0.1", + "modified": "2023-09-05T09:48:28.472Z", + "port": 25, + "process_path": "/opt/Service/ServiceUser/python38/bin/pybin" + } + ], + "processes": [ + { + "command": "/opt/Service/ServiceUser/ServiceUser -d", + "interactive": true, + "md5": "i98ujh", + "path": "/opt/Service/ServiceUser/ServiceUser", + "ppath": "/usr/lib/systemd/systemd", + "time": "2023-09-05T09:19:54.613Z", + "user": "root" + } + ], + "startup_process": { + "command": "/opt/Service/ServiceUser/ServiceUser -d", + "md5": "aaa", + "path": "/opt/Service/ServiceUser/ServiceUser", + "ppath": "/usr/lib/systemd/systemd", + "time": "2023-09-05T09:19:54.613Z", + "user": "root" + } + }, + { + "listening_ports": [ + { + "command": "/usr/share/filebeat/bin/filebeat", + "modified": "2023-09-05T09:21:16.370Z", + "port": 9001, + "process_path": "/usr/share/filebeat/bin/filebeat" + } + ], + "name": "filebeat", + "processes": [ + { + "command": "/usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /us", + "md5": "kjj", + "modified": true, + "path": "/usr/share/filebeat/bin/filebeat", + "ppath": "/usr/lib/systemd/systemd", + "time": "2023-09-05T09:21:13.920Z", + "user": "root" + } + ], + "startup_process": { + "command": "/usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /us", + "interactive": true, + "md5": "mnj87", + "modified": true, + "path": "/usr/share/filebeat/bin/filebeat", + "ppath": "/usr/lib/systemd/systemd", + "time": "2023-09-05T09:21:13.920Z", + "user": "root" + } + } + ], + "collections": [ + "All" + ], + "created": "2023-09-05T09:19:52.991Z", + "geoip": { + "countries": [ + { + "code": "US", + "ip": "0.0.0.0", + "modified": "2023-09-06T16:03:30.578Z" + }, + { + "code": "CA", + "ip": "81.2.69.142", + "modified": "2023-09-06T16:03:30.229Z" + } + ], + "modified": "2023-09-06T16:03:30.703Z" + }, + "hash": "9", + "labels": [ + "osDistro:redhat", + "osVersion:7" + ], + "ssh_events": [ + { + "command": "/usr/libexec/docker/cli-plugins/docker-app docker-cli-plugin-metadata - High rate of events, throttling started", + "interactive": false, + "ip": "0.0.0.0", + "login_time": "2023-09-06T16:01:01.000Z", + "md5": "dhdj", + "modified": true, + "path": "/usr/libexec/docker/cli-plugins/docker-app", + "ppath": "/usr/bin/docker", + "time": "2023-09-06T16:05:43.768Z", + "user": "root" + } + ], + "time": "0001-01-01T00:00:00.000Z" + } + }, + "process": { + "command_line": [ + "/opt/Service/ServiceUser/ServiceUser -d", + "/usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /us", + "/usr/libexec/docker/cli-plugins/docker-app docker-cli-plugin-metadata - High rate of events, throttling started" + ], + "hash": { + "md5": [ + "aaa", + "mnj87", + "i98ujh", + "kjj", + "dhdj" + ] + }, + "interactive": [ + "true", + "false" + ] + }, + "related": { + "hash": [ + "9", + "aaa", + "mnj87", + "i98ujh", + "kjj", + "dhdj" + ], + "hosts": [ + "localhost" + ], + "ip": [ + "0.0.0.0", + "81.2.69.142", + "127.0.0.1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + } + ] +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-input-config.yml b/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-input-config.yml new file mode 100644 index 00000000000..a2af63f2fcb --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-input-config.yml @@ -0,0 +1,12 @@ +input: cel +service: prisma_cloud +vars: + username: xxxx + password: xxxx +data_stream: + vars: + url: http://{{Hostname}}:{{Port}} + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-tcp-config.yml b/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-tcp-config.yml new file mode 100644 index 00000000000..d7620579d12 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-tcp-config.yml @@ -0,0 +1,11 @@ +service: prisma_cloud-host_profile-tcp +service_notify_signal: SIGHUP +input: tcp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9510 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-udp-config.yml b/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-udp-config.yml new file mode 100644 index 00000000000..cb3e60e9ab2 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/_dev/test/system/test-udp-config.yml @@ -0,0 +1,11 @@ +service: prisma_cloud-host_profile-udp +service_notify_signal: SIGHUP +input: udp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9511 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/host_profile/agent/stream/input.yml.hbs b/packages/prisma_cloud/data_stream/host_profile/agent/stream/input.yml.hbs new file mode 100644 index 00000000000..ab9a19f0d19 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/agent/stream/input.yml.hbs @@ -0,0 +1,100 @@ +config_version: 2 +interval: {{interval}} +{{#if enable_request_tracer}} +resource.tracer.filename: "../../logs/httpjson/http-request-trace-*.ndjson" +{{/if}} +{{#if proxy_url}} +resource.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +resource.url: {{url}} +state: + user: {{username}} + password: {{password}} + batch_size: {{batch_size}} + offset: {{offset}} + want_more: false +redact: + fields: + - password +program: | + ( + state.with(has(state.want_more) && !(state.want_more) + ? + post_request( + state.url + "/authenticate", + "application/json", + {"username":state.user,"password":state.password}.encode_json() + ).do_request().as(resp, bytes(resp.Body).decode_json().as(body, { + "access_token": body.token, + })) + : + {} + ).as(state, + request("GET", + ( + has(state.cursor) && has(state.cursor.new_offset) && state.cursor.new_offset != null + ? + state.url + "/profiles/host?limit=" + string(state.batch_size) + "&offset=" + string(state.cursor.new_offset) + : + state.url + "/profiles/host?limit=" + string(state.batch_size) + "&offset=" + string(state.offset) + )).with({ + "Header":{ + "Authorization": ["Bearer " + state.access_token], + } + }).do_request().as(resp, bytes(resp.Body).decode_json().as(inner_body, { + "events": (inner_body != null ? + inner_body.map(e, { + "message": e.encode_json(), + }) + : + [] + ), + "url": state.url, + "want_more": inner_body.size() > 0, + "user": state.user, + "password": state.password, + "batch_size": string(state.batch_size), + "access_token": state.access_token, + "cursor": + { + "new_offset": + ( + has(state.cursor) && has(state.cursor.new_offset) && state.cursor.new_offset != null + ? + ( + inner_body != null && inner_body.size() > 0 + ? + string(int(state.cursor.new_offset) + int(inner_body.size())) + : + state.cursor.new_offset + ) + : + string(int(state.offset) + int(inner_body.size())) + ) + }, + })) + ) + ) +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/host_profile/agent/stream/tcp.yml.hbs b/packages/prisma_cloud/data_stream/host_profile/agent/stream/tcp.yml.hbs new file mode 100644 index 00000000000..c29dbc0fc55 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/agent/stream/tcp.yml.hbs @@ -0,0 +1,24 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if tcp_options}} +{{tcp_options}} +{{/if}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/host_profile/agent/stream/udp.yml.hbs b/packages/prisma_cloud/data_stream/host_profile/agent/stream/udp.yml.hbs new file mode 100644 index 00000000000..3e79cc97e77 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/agent/stream/udp.yml.hbs @@ -0,0 +1,21 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if udp_options}} +{{udp_options}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/host_profile/elasticsearch/ingest_pipeline/default.yml b/packages/prisma_cloud/data_stream/host_profile/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..713327c2f85 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,759 @@ +--- +description: Pipeline for processing host_profile logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 8.10.0 + - set: + field: event.kind + tag: set_event_kind + value: asset + - append: + field: event.category + tag: append_event_category + value: host + - append: + field: event.type + tag: append_event_type + value: info + - rename: + field: message + tag: rename_message + target_field: event.original + ignore_missing: true + - drop: + if: ctx.event?.original != null && ctx.event.original.isEmpty() + - json: + field: event.original + tag: json_message + target_field: json + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.accountID + tag: rename_prisma_cloud_host_profile_accountID + target_field: prisma_cloud.host_profile.account_id + ignore_missing: true + - set: + field: cloud.account.id + tag: set_prisma_cloud_host_profile_account_id_to_cloud_account_id + copy_from: prisma_cloud.host_profile.account_id + ignore_empty_value: true + - rename: + field: json._id + tag: rename_prisma_cloud_host_profile_id + target_field: prisma_cloud.host_profile._id + ignore_missing: true + - set: + field: host.hostname + tag: set_prisma_cloud_host_profile_id_to_hostname + copy_from: prisma_cloud.host_profile._id + ignore_empty_value: true + - append: + field: related.hosts + tag: append_prisma_cloud_host_profile_id_to_related_hosts + value: '{{{prisma_cloud.host_profile._id}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.host_profile?._id != null + - rename: + field: json.collections + tag: rename_prisma_cloud_host_profile_collections + target_field: prisma_cloud.host_profile.collections + ignore_missing: true + - date: + field: json.created + tag: date_prisma_cloud_host_profile_created + target_field: prisma_cloud.host_profile.created + formats: + - ISO8601 + if: ctx.json?.created != null && ctx.json.created != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.created + tag: set_prisma_cloud_host_profile_created + copy_from: prisma_cloud.host_profile.created + ignore_empty_value: true + - rename: + field: json.labels + tag: rename_prisma_cloud_host_profile_labels + target_field: prisma_cloud.host_profile.labels + ignore_missing: true + - set: + field: labels + tag: set_prisma_cloud_host_profile_labels + copy_from: prisma_cloud.host_profile.labels + ignore_empty_value: true + - rename: + field: json.geoip.countries + tag: rename_prisma_cloud_host_profile_geoip_countries + target_field: prisma_cloud.host_profile.geoip.countries + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.geoip.countries + if: ctx.prisma_cloud?.host_profile?.geoip?.countries instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + tag: convert_prisma_cloud_host_profile_ip_to_ip + type: ip + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.ip + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.geoip.countries + if: ctx.prisma_cloud?.host_profile?.geoip?.countries instanceof List + ignore_failure: true + processor: + append: + field: related.ip + tag: append_prisma_cloud_host_profile_ip_into_related_ip_1 + value: '{{{_ingest._value.ip}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.geoip.countries + if: ctx.prisma_cloud?.host_profile?.geoip?.countries instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.modified + target_field: _ingest._value.modified + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - date: + field: json.geoip.modified + tag: date_prisma_cloud_host_profile_modified + target_field: prisma_cloud.host_profile.geoip.modified + formats: + - ISO8601 + if: ctx.json?.geoip?.modified != null && ctx.json.geoip.modified != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.hash + tag: convert_prisma_cloud_host_profile_hash_keyword + target_field: prisma_cloud.host_profile.hash + type: string + ignore_missing: true + if: ctx.json?.hash != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - append: + field: related.hash + tag: append_json_hash_into_related_hash + value: '{{{prisma_cloud.host_profile.hash}}}' + allow_duplicates: false + - rename: + field: json.apps + tag: rename_prisma_cloud_host_profile_apps + target_field: prisma_cloud.host_profile.apps + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.startupProcess + tag: rename_prisma_cloud_host_profile_apps_startupProcess + target_field: _ingest._value.startup_process + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + append: + field: process.command_line + tag: append_prisma_cloud_host_profile_apps_startupProcess_command_line + value: '{{{_ingest._value.startup_process.command}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.startup_process.interactive + tag: convert_prisma_cloud_host_profile_apps_startupProcess_interactive_to_interactive + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.startup_process.interactive + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + append: + field: process.interactive + tag: append_prisma_cloud_host_profile_apps_startupProcess_interactive + value: '{{{_ingest._value.startup_process.interactive}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + append: + field: process.hash.md5 + tag: append_prisma_cloud_host_profile_apps_startupProcess_md5 + value: '{{{_ingest._value.startup_process.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + append: + field: related.hash + tag: append_prisma_cloud_host_profile_apps_startupProcess_md5_into_related_hash + value: '{{{_ingest._value.startup_process.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.startup_process.modified + tag: convert_prisma_cloud_host_profile_apps_startupProcess_modified_to_modified + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.startup_process.modified + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.startup_process.time + target_field: _ingest._value.startup_process.time + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.startup_process.time + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + tag: remove_ecs_duplicate_fields_apps_startup_process_fields + field: + - _ingest._value.startup_process.command + - _ingest._value.startup_process.interactive + - _ingest._value.startup_process.md5 + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.outgoingPorts + tag: rename_prisma_cloud_host_profile_apps_outgoingPorts + target_field: _ingest._value.outgoing_ports + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.outgoing_ports + ignore_failure: true + processor: + rename: + field: _ingest._value.processPath + tag: rename_prisma_cloud_host_profile_apps_outgoing_ports_processPath + target_field: _ingest._value.process_path + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.outgoing_ports + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + tag: convert_prisma_cloud_host_profile_apps_outgoing_ports_ip_to_ip + type: ip + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.ip + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.outgoing_ports + ignore_failure: true + processor: + append: + field: related.ip + tag: append_prisma_cloud_host_profile_outgoing_ports_ip_into_related_ip_1 + value: '{{{_ingest._value.ip}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.outgoing_ports + ignore_failure: true + processor: + convert: + field: _ingest._value.port + tag: convert_prisma_cloud_host_profile_outgoing_ports_port + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.port + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.outgoing_ports + ignore_failure: true + processor: + date: + field: _ingest._value.modified + target_field: _ingest._value.modified + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.listeningPorts + tag: rename_ingest_value_listeningPorts + target_field: _ingest._value.listening_ports + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.listening_ports + ignore_failure: true + processor: + rename: + field: _ingest._value.processPath + tag: rename_prisma_cloud_host_profile_listening_ports_processPath + target_field: _ingest._value.process_path + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.listening_ports + ignore_failure: true + processor: + date: + field: _ingest._value.modified + target_field: _ingest._value.modified + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.listening_ports + ignore_failure: true + processor: + convert: + field: _ingest._value.port + tag: convert_prisma_cloud_host_profile_listening_ports_port_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.port + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + convert: + field: _ingest._value.interactive + tag: convert_prisma_cloud_host_profile_listening_ports_interactive + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.interactive + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + append: + field: process.interactive + value: '{{{_ingest._value.interactive}}}' + tag: append_prisma_cloud_host_profile_apps_processes_interactive_into_process_interactive + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + date: + field: _ingest._value.time + target_field: _ingest._value.time + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.time + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + convert: + field: _ingest._value.modified + tag: convert_prisma_cloud_host_profile_apps_processes_modified_to_modified + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + append: + field: process.hash.md5 + tag: append_prisma_cloud_host_profile_app_processes_md5_into_process_hash_md5 + value: '{{{_ingest._value.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + append: + field: related.hash + tag: append_prisma_cloud_host_profile_app_processes_md5_into_related_hash + value: '{{{_ingest._value.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + append: + field: process.command_line + tag: append_prisma_cloud_host_profile_app_processes_command_into_process_command_line + value: '{{{_ingest._value.command}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.apps + if: ctx.prisma_cloud?.host_profile?.apps instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + foreach: + field: _ingest._value.processes + ignore_failure: true + processor: + remove: + tag: remove_ecs_duplicate_fields_apps_startup_process_fields + field: + - _ingest._value.command + - _ingest._value.interactive + - _ingest._value.md5 + ignore_missing: true + - rename: + field: json.sshEvents + tag: rename_prisma_cloud_host_profile_sshEvents + target_field: prisma_cloud.host_profile.ssh_events + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.interactive + tag: convert_prisma_cloud_host_profile_ssh_events_interactive + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.interactive + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + append: + field: process.interactive + value: '{{{_ingest._value.interactive}}}' + tag: append_prisma_cloud_host_profile_ssh_events_interactive_into_process_interactive + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.modified + tag: convert_prisma_cloud_host_profile_ssh_events_modified_to_modified + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.modified + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + append: + field: process.hash.md5 + tag: append_prisma_cloud_host_profile_ssh_events_md5_into_process_hash_md5_1 + value: '{{{_ingest._value.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + append: + field: related.hash + tag: append_prisma_cloud_host_profile_ssh_events_md5_into_related_hash + value: '{{{_ingest._value.md5}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + append: + field: process.command_line + tag: append_prisma_cloud_host_profile_ssh_events_command_into_process_cmd_line + value: '{{{_ingest._value.command}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + tag: remove_ecs_duplicate_ssh_events_fields + field: + - _ingest._value.command + - _ingest._value.interactive + - _ingest._value.md5 + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.loginTime + target_field: _ingest._value.login_time + formats: + - UNIX + on_failure: + - remove: + field: _ingest._value.loginTime + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value.loginTime + ignore_missing: true + - foreach: + field: prisma_cloud.host_profile.ssh_events + if: ctx.prisma_cloud?.host_profile?.ssh_events instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.time + target_field: _ingest._value.time + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.time + ignore_missing: true + - date: + field: json.time + tag: 'date_rename_time_to_custom_name' + target_field: prisma_cloud.host_profile.time + formats: + - ISO8601 + if: ctx.json?.time != null && ctx.json.time != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - remove: + field: + - json + ignore_missing: true + - remove: + field: + - event.original + ignore_missing: true + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + - remove: + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + ignore_missing: true + tag: remove_preserve_duplicate_custom_fields + field: + - prisma_cloud.host_profile.account_id + - prisma_cloud.host_profile._id + - prisma_cloud.host_profile.created + - prisma_cloud.host_profile.labels + - script: + lang: painless + description: Drops null/empty values recursively. + source: |- + boolean drop(Object object) { + if (object == null || object == '') { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(v -> drop(v)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(v -> drop(v)); + return (((List) object).length == 0); + } + return false; + } + drop(ctx); + - set: + field: event.kind + value: pipeline_error + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + value: pipeline_error + + diff --git a/packages/prisma_cloud/data_stream/host_profile/fields/base-fields.yml b/packages/prisma_cloud/data_stream/host_profile/fields/base-fields.yml new file mode 100644 index 00000000000..7b31320f5ec --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: prisma_cloud +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: prisma_cloud.host_profile +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/prisma_cloud/data_stream/host_profile/fields/beats.yml b/packages/prisma_cloud/data_stream/host_profile/fields/beats.yml new file mode 100644 index 00000000000..b3701b581cf --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/fields/beats.yml @@ -0,0 +1,9 @@ +- name: input.type + type: keyword + description: Type of filebeat input. +- name: log.offset + type: long + description: Log offset. +- name: tags + type: keyword + description: User defined tags. diff --git a/packages/prisma_cloud/data_stream/host_profile/fields/fields.yml b/packages/prisma_cloud/data_stream/host_profile/fields/fields.yml new file mode 100644 index 00000000000..5e35a462467 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/fields/fields.yml @@ -0,0 +1,180 @@ +- name: prisma_cloud + type: group + fields: + - name: host_profile + type: group + fields: + - name: _id + type: keyword + description: ID is the profile ID (hostname). + - name: account_id + type: keyword + description: AccountID is the cloud account ID associated with the profile. + - name: apps + type: group + fields: + - name: listening_ports + type: group + fields: + - name: command + type: keyword + description: Command represents the command that triggered the connection. + - name: modified + type: date + description: Modified is a timestamp of when the event occurred. + - name: port + type: long + description: Port is the port number. + - name: process_path + type: keyword + description: ProcessPath represents the path to the process that uses the port. + - name: name + type: keyword + description: Name is the app name. + - name: outgoing_ports + type: group + fields: + - name: command + type: keyword + description: Command represents the command that triggered the connection. + - name: country + type: keyword + description: Country is the country ISO code for the given IP address. + - name: ip + type: ip + description: IP is the IP address captured over this port. + - name: modified + type: date + description: Modified is a timestamp of when the event occurred. + - name: port + type: long + description: Port is the port number. + - name: process_path + type: keyword + description: ProcessPath represents the path to the process that uses the port. + - name: processes + type: group + fields: + - name: command + type: keyword + description: Command represents the command that triggered the connection. + - name: interactive + type: boolean + description: Interactive indicates whether the process belongs to an interactive session. + - name: md5 + type: keyword + description: MD5 is the process binary MD5 sum. + - name: modified + type: boolean + description: Modified indicates the process binary was modified after the container has started. + - name: path + type: keyword + description: Path is the process binary path. + - name: ppath + type: keyword + description: PPath is the parent process path. + - name: time + type: date + description: Time is the time in which the process was added. If the process was modified, Time is the modification time. + - name: user + type: keyword + description: User represents the username that started the process. + - name: startup_process + type: group + fields: + - name: command + type: keyword + description: Command represents the command that triggered the connection. + - name: interactive + type: boolean + description: Interactive indicates whether the process belongs to an interactive session. + - name: md5 + type: keyword + description: MD5 is the process binary MD5 sum. + - name: modified + type: boolean + description: Modified is a timestamp of when the event occurred. + - name: path + type: keyword + description: Path is the process binary path. + - name: ppath + type: keyword + description: PPath is the parent process path. + - name: time + type: date + description: Time is the time in which the process was added. If the process was modified, Time is the modification time. + - name: user + type: keyword + description: User represents the username that started the process. + - name: collections + type: keyword + description: Collections is a list of collections to which this profile applies. + - name: created + type: date + description: Created is the profile creation time. + - name: geoip + type: group + fields: + - name: countries + type: group + fields: + - name: code + type: keyword + description: Code is the country iso code. + - name: ip + type: ip + description: Ip is the Ip address. + - name: modified + type: date + description: Modified is the last modified time of this entry. + - name: modified + type: date + description: Modified is the last modified time of the cache. + - name: hash + type: keyword + description: ProfileHash represents the profile hash It is allowed to contain up to uint32 numbers, and represented by int64 since mongodb does not support unsigned data types. + - name: labels + type: keyword + description: Labels are the labels associated with the profile. + - name: ssh_events + type: group + fields: + - name: command + type: keyword + description: Command represents the command that triggered the connection. + - name: country + type: keyword + description: Country represents the SSH client's origin country. + - name: interactive + type: boolean + description: Interactive indicates whether the process belongs to an interactive session. + - name: ip + type: keyword + description: IP address represents the connection client IP address. + - name: login_time + type: date + description: LoginTime represents the SSH login time. + - name: md5 + type: keyword + description: MD5 is the process binary MD5 sum. + - name: modified + type: boolean + description: Modified indicates the process binary was modified after the container has started. + - name: path + type: keyword + description: Path is the process binary path. + - name: ppath + type: keyword + description: PPath is the parent process path. + - name: time + type: date + description: Time is the time in which the process was added. If the process was modified, Time is the modification time. + - name: user + type: keyword + description: User represents the username that started the process. + - name: time + type: date + description: Time is the last time when this profile was modified. +- name: log.source.address + type: keyword + description: Source address from which the log event was read / sent from. diff --git a/packages/prisma_cloud/data_stream/host_profile/manifest.yml b/packages/prisma_cloud/data_stream/host_profile/manifest.yml new file mode 100644 index 00000000000..1c603aee30e --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/manifest.yml @@ -0,0 +1,219 @@ +title: Collect Host Profile logs from Prisma Cloud Workload Protection. +type: logs +streams: + - input: cel + title: Host Profile Logs + description: Collect Host Profile logs from Prisma Cloud Workload Protection. + template_path: input.yml.hbs + enabled: false + vars: + - name: url + type: text + title: URL + description: Base URL of the Prisma Cloud Server API. + required: true + show_user: true + - name: interval + type: text + title: Interval + description: Interval between two REST API calls. Supported units for this parameter are h/m/s. + default: 1m + multi: false + required: true + show_user: true + - name: offset + type: integer + title: Offset + description: Offsets the result to a specific report count. Offset starts from 0. + default: 0 + multi: false + required: true + show_user: false + - name: batch_size + type: integer + title: Batch Size + description: Number of reports to retrieve in a page. For PCCE, the maximum limit is 250. For PCEE, the maximum limit is 50. + default: 50 + multi: false + required: true + show_user: false + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Supported time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + required: false + show_user: false + description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-host_profile + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.host_profile fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: tcp + template_path: tcp.yml.hbs + title: Host Profile logs + description: Collect Host Profile logs via TCP input. + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for TCP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The TCP port number to listen on. + multi: false + required: true + show_user: true + default: 9508 + - name: tcp_options + type: yaml + title: Custom TCP Options + multi: false + required: false + show_user: false + default: | + #max_message_size: 20MiB + #max_connections: 1 + #framing: delimiter + #line_delimiter: "\n" + description: Specify custom configuration options for the TCP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-host_profile + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.host_profile fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + template_path: udp.yml.hbs + title: Host Profile logs + description: Collect Host Profile logs via UDP input. + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for UDP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The UDP port number to listen on. + multi: false + required: true + show_user: true + default: 9508 + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + #max_message_size: 10KiB + #timeout: 300s + description: Specify custom configuration options for the UDP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-host_profile + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.host_profile fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/prisma_cloud/data_stream/host_profile/sample_event.json b/packages/prisma_cloud/data_stream/host_profile/sample_event.json new file mode 100644 index 00000000000..2ce60988d64 --- /dev/null +++ b/packages/prisma_cloud/data_stream/host_profile/sample_event.json @@ -0,0 +1,68 @@ +{ + "@timestamp": "2023-10-18T12:15:22.607Z", + "agent": { + "ephemeral_id": "27dd294d-e02a-4b56-a204-034c7853e226", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "data_stream": { + "dataset": "prisma_cloud.host_profile", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "host" + ], + "created": "2023-08-11T06:53:48.855Z", + "dataset": "prisma_cloud.host_profile", + "ingested": "2023-10-18T12:15:23Z", + "kind": "asset", + "original": "{\"_id\":\"DESKTOP-6PQXXMS\",\"collections\":[\"All\"],\"created\":\"2023-08-11T06:53:48.855Z\",\"hash\":1,\"time\":\"0001-01-01T00:00:00Z\"}", + "type": [ + "info" + ] + }, + "host": { + "hostname": "DESKTOP-6PQXXMS" + }, + "input": { + "type": "cel" + }, + "prisma_cloud": { + "host_profile": { + "_id": "DESKTOP-6PQXXMS", + "collections": [ + "All" + ], + "created": "2023-08-11T06:53:48.855Z", + "hash": "1", + "time": "0001-01-01T00:00:00.000Z" + } + }, + "related": { + "hash": [ + "1" + ], + "hosts": [ + "DESKTOP-6PQXXMS" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-host_profile" + ] +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-common-config.yml b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-incident-audit.log b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-incident-audit.log new file mode 100644 index 00000000000..016ac4ca028 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-incident-audit.log @@ -0,0 +1 @@ +{"_id":"string","accountID":"123abc","acknowledged":"","app":"string","appID":"string","audits":[{"_id":"string","accountID":"abdcsfData","app":"string","appID":"string","attackTechniques":[["exploitationForPrivilegeEscalation"]],"attackType":"cloudMetadataProbing","cluster":"string","collections":["string"],"command":"string","container":true,"containerId":"string","containerName":"string","count":0,"country":"string","domain":"string","effect":["block","prevent"],"err":"string","filepath":"string","fqdn":"audits-fqdn-hostname","function":"string","functionID":"string","hostname":"string","imageId":"string","imageName":"string","interactive":true,"ip":"0.0.0.0","label":"string","labels":{},"md5":"string","msg":"string","namespace":"string","os":"string","pid":0,"port":0,"processPath":"string","profileId":"string","provider":"alibaba","rawEvent":"string","region":"string","requestID":"string","resourceID":"string","ruleName":"string","runtime":["python3.6"],"severity":["low","medium","high"],"time":"2023-09-19T07:15:31.899Z","type":["processes"],"user":"string","version":"string","vmID":"string","wildFireReportURL":"string"}],"category":"malware","cluster":"string","collections":["string"],"containerID":"string","containerName":"string","customRuleName":"string","fqdn":"string","function":"string","functionID":"string","hostname":"string","imageID":"string","imageName":"string","labels":{},"namespace":"string","profileID":"string","provider":"oci","region":"string","resourceID":"string","runtime":"string","serialNum":0,"shouldCollect":true,"time":"2023-09-19T07:15:31.899Z","type":"host","vmID":"string","windows":true} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-incident-audit.log-expected.json b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-incident-audit.log-expected.json new file mode 100644 index 00000000000..88b86205d0c --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/pipeline/test-incident-audit.log-expected.json @@ -0,0 +1,207 @@ +{ + "expected": [ + { + "@timestamp": "2023-09-19T07:15:31.899Z", + "cloud": { + "account": { + "id": [ + "123abc", + "abdcsfData" + ] + }, + "provider": [ + "alibaba", + "oci" + ], + "region": "string" + }, + "container": { + "id": "string", + "image": { + "name": [ + "string" + ] + }, + "name": [ + "string" + ] + }, + "ecs": { + "version": "8.10.0" + }, + "event": { + "category": [ + "malware" + ], + "id": "string", + "kind": "event", + "original": "{\"_id\":\"string\",\"accountID\":\"123abc\",\"acknowledged\":\"\",\"app\":\"string\",\"appID\":\"string\",\"audits\":[{\"_id\":\"string\",\"accountID\":\"abdcsfData\",\"app\":\"string\",\"appID\":\"string\",\"attackTechniques\":[[\"exploitationForPrivilegeEscalation\"]],\"attackType\":\"cloudMetadataProbing\",\"cluster\":\"string\",\"collections\":[\"string\"],\"command\":\"string\",\"container\":true,\"containerId\":\"string\",\"containerName\":\"string\",\"count\":0,\"country\":\"string\",\"domain\":\"string\",\"effect\":[\"block\",\"prevent\"],\"err\":\"string\",\"filepath\":\"string\",\"fqdn\":\"audits-fqdn-hostname\",\"function\":\"string\",\"functionID\":\"string\",\"hostname\":\"string\",\"imageId\":\"string\",\"imageName\":\"string\",\"interactive\":true,\"ip\":\"0.0.0.0\",\"label\":\"string\",\"labels\":{},\"md5\":\"string\",\"msg\":\"string\",\"namespace\":\"string\",\"os\":\"string\",\"pid\":0,\"port\":0,\"processPath\":\"string\",\"profileId\":\"string\",\"provider\":\"alibaba\",\"rawEvent\":\"string\",\"region\":\"string\",\"requestID\":\"string\",\"resourceID\":\"string\",\"ruleName\":\"string\",\"runtime\":[\"python3.6\"],\"severity\":[\"low\",\"medium\",\"high\"],\"time\":\"2023-09-19T07:15:31.899Z\",\"type\":[\"processes\"],\"user\":\"string\",\"version\":\"string\",\"vmID\":\"string\",\"wildFireReportURL\":\"string\"}],\"category\":\"malware\",\"cluster\":\"string\",\"collections\":[\"string\"],\"containerID\":\"string\",\"containerName\":\"string\",\"customRuleName\":\"string\",\"fqdn\":\"string\",\"function\":\"string\",\"functionID\":\"string\",\"hostname\":\"string\",\"imageID\":\"string\",\"imageName\":\"string\",\"labels\":{},\"namespace\":\"string\",\"profileID\":\"string\",\"provider\":\"oci\",\"region\":\"string\",\"resourceID\":\"string\",\"runtime\":\"string\",\"serialNum\":0,\"shouldCollect\":true,\"time\":\"2023-09-19T07:15:31.899Z\",\"type\":\"host\",\"vmID\":\"string\",\"windows\":true}", + "type": [ + "info" + ] + }, + "host": { + "domain": [ + "audits-fqdn-hostname", + "string" + ], + "hostname": "string" + }, + "prisma_cloud": { + "incident_audit": { + "_id": "string", + "account_id": "123abc", + "app": { + "id": "string", + "value": "string" + }, + "category": "malware", + "cluster": "string", + "collections": [ + "string" + ], + "container": { + "id": "string", + "name": "string" + }, + "custom_rule_name": "string", + "data": [ + { + "_id": "string", + "account_id": "abdcsfData", + "app": { + "id": "string", + "value": "string" + }, + "attack": { + "techniques": [ + [ + "exploitationForPrivilegeEscalation" + ] + ], + "type": "cloudMetadataProbing" + }, + "cluster": "string", + "collections": [ + "string" + ], + "command": "string", + "container": { + "id": "string", + "name": "string", + "value": true + }, + "count": 0, + "country": "string", + "domain": "string", + "effect": [ + "block", + "prevent" + ], + "err": "string", + "filepath": "string", + "fqdn": "audits-fqdn-hostname", + "function": { + "id": "string", + "value": "string" + }, + "hostname": "string", + "image": { + "id": "string", + "name": "string" + }, + "interactive": true, + "ip": "0.0.0.0", + "label": "string", + "md5": "string", + "msg": "string", + "namespace": "string", + "os": "string", + "pid": 0, + "port": 0, + "process_path": "string", + "profile_id": "string", + "provider": "alibaba", + "raw_event": "string", + "region": "string", + "request_id": "string", + "resource_id": "string", + "rule_name": "string", + "runtime": [ + "python3.6" + ], + "severity": [ + "low", + "medium", + "high" + ], + "time": "2023-09-19T07:15:31.899Z", + "type": [ + "processes" + ], + "user": "string", + "version": "string", + "vm_id": "string", + "wild_fire_report_url": "string" + } + ], + "fqdn": "string", + "function": { + "id": "string", + "value": "string" + }, + "hostname": "string", + "image": { + "id": "string", + "name": "string" + }, + "namespace": "string", + "profile_id": "string", + "provider": "oci", + "region": "string", + "resource_id": "string", + "runtime": "string", + "serial_num": 0, + "should_collect": true, + "time": "2023-09-19T07:15:31.899Z", + "type": "host", + "vm_id": "string", + "windows": true + } + }, + "related": { + "hosts": [ + "audits-fqdn-hostname", + "string" + ], + "ip": [ + "0.0.0.0" + ], + "user": [ + "string" + ] + }, + "rule": { + "name": [ + "string" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "threat": { + "technique": { + "name": [ + "exploitationForPrivilegeEscalation" + ], + "subtechnique": { + "name": [ + "cloudMetadataProbing" + ] + } + } + } + } + ] +} \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/incident_audit/_dev/test/system/test-tcp-config.yml b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/system/test-tcp-config.yml new file mode 100644 index 00000000000..0781b46b298 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/system/test-tcp-config.yml @@ -0,0 +1,11 @@ +service: prisma_cloud-incident_audit-tcp +service_notify_signal: SIGHUP +input: tcp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9512 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/incident_audit/_dev/test/system/test-udp-config.yml b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/system/test-udp-config.yml new file mode 100644 index 00000000000..dda062aaf81 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/_dev/test/system/test-udp-config.yml @@ -0,0 +1,11 @@ +service: prisma_cloud-incident_audit-udp +service_notify_signal: SIGHUP +input: udp +data_stream: + vars: + listen_address: 0.0.0.0 + listen_port: 9513 + preserve_original_event: true + preserve_duplicate_custom_fields: true +assert: + hit_count: 1 diff --git a/packages/prisma_cloud/data_stream/incident_audit/agent/stream/tcp.yml.hbs b/packages/prisma_cloud/data_stream/incident_audit/agent/stream/tcp.yml.hbs new file mode 100644 index 00000000000..c29dbc0fc55 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/agent/stream/tcp.yml.hbs @@ -0,0 +1,24 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if tcp_options}} +{{tcp_options}} +{{/if}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/incident_audit/agent/stream/udp.yml.hbs b/packages/prisma_cloud/data_stream/incident_audit/agent/stream/udp.yml.hbs new file mode 100644 index 00000000000..3e79cc97e77 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/agent/stream/udp.yml.hbs @@ -0,0 +1,21 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if udp_options}} +{{udp_options}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/prisma_cloud/data_stream/incident_audit/elasticsearch/ingest_pipeline/default.yml b/packages/prisma_cloud/data_stream/incident_audit/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..3388bac0d59 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,809 @@ +--- +description: Pipeline for processing incident_audit logs. +processors: + - set: + field: ecs.version + tag: set_ecs_version + value: 8.10.0 + - set: + field: event.kind + tag: set_event_kind + value: event + - rename: + field: message + tag: rename_message + target_field: event.original + ignore_missing: true + - json: + field: event.original + tag: json_message + target_field: json + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json._id + tag: rename_prisma_cloud_incident_audit_id + target_field: prisma_cloud.incident_audit._id + ignore_missing: true + - set: + field: event.id + tag: set_prisma_cloud_incident_audit_id + copy_from: prisma_cloud.incident_audit._id + ignore_empty_value: true + - rename: + field: json.accountID + tag: rename_prisma_cloud_incident_audit_accountID + target_field: prisma_cloud.incident_audit.account_id + ignore_missing: true + - append: + field: cloud.account.id + tag: append_prisma_cloud_incident_audit_account_id + value: '{{{prisma_cloud.incident_audit.account_id}}}' + allow_duplicates: false + - convert: + field: json.acknowledged + tag: convert_prisma_cloud_incident_audit_acknowledged + target_field: prisma_cloud.incident_audit.acknowledged + type: boolean + ignore_missing: true + if: ctx.json?.acknowledged != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.appID + tag: rename_prisma_cloud_incident_audit_appID + target_field: prisma_cloud.incident_audit.app.id + ignore_missing: true + - rename: + field: json.app + tag: rename_prisma_cloud_incident_audit_app + target_field: prisma_cloud.incident_audit.app.value + ignore_missing: true + - rename: + field: json.audits + tag: rename_prisma_cloud_incident_audit_audits + target_field: prisma_cloud.incident_audit.data + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.app + tag: rename_prisma_cloud_incident_audit_data_app + target_field: _ingest._value.app.value + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.appID + tag: rename_prisma_cloud_incident_audit_data_appID + target_field: _ingest._value.app.id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.accountID + tag: rename_prisma_cloud_incident_audit_data_accountID + target_field: _ingest._value.account_id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: cloud.account.id + tag: append_prisma_cloud_incident_audit_data_accountID_into_cloud_account_id + value: '{{{_ingest._value.account_id}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.attackTechniques + tag: rename_prisma_cloud_incident_audit_data_attackTechniques + target_field: _ingest._value.attack.techniques + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.attack.techniques + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + append: + field: threat.technique.name + tag: append_prisma_cloud_incident_audit_data_attack_techniques + value: '{{{_ingest._value}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.attackType + tag: rename_prisma_cloud_incident_audit_data_attackType + target_field: _ingest._value.attack.type + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: threat.technique.subtechnique.name + tag: append_prisma_cloud_incident_audit_data_attack_type + value: '{{{_ingest._value.attack.type}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.container + tag: rename_prisma_cloud_incident_audit_data_container + target_field: _ingest._value.container.value + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.container.value + tag: convert_prisma_cloud_incident_audit_data_container_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.container.value + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.containerId + tag: rename_prisma_cloud_incident_audit_data_containerId + target_field: _ingest._value.container.id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.containerName + tag: rename_prisma_cloud_incident_audit_data_containerName + target_field: _ingest._value.container.name + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + tag: foreach_append_container_name + processor: + append: + field: container.name + tag: append_prisma_cloud_incident_audit_data_containerName_into_container_name_1 + value: '{{{_ingest._value.container.name}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.count + tag: convert_prisma_cloud_incident_audit_data_count_to_long_1 + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.count + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + tag: foreach_append_data_fqdn + processor: + append: + field: host.domain + tag: append_prisma_cloud_incident_audit_fqdn_to_host_domain + value: '{{{_ingest._value.fqdn}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + tag: foreach_append_data_fqdn + processor: + append: + field: related.hosts + tag: append_prisma_cloud_incident_audit_fqdn_to_related_hosts + value: '{{{_ingest._value.fqdn}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.function + tag: rename_prisma_cloud_incident_audit_data_function + target_field: _ingest._value.function.value + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.functionID + tag: rename_prisma_cloud_incident_audit_data_functionID + target_field: _ingest._value.function.id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: related.hosts + tag: append_prisma_cloud_incident_audit_hostname_into_related_hosts_1 + value: '{{{_ingest._value.hostname}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.imageId + tag: rename_prisma_cloud_incident_audit_data_imageId + target_field: _ingest._value.image.id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.imageName + tag: rename_prisma_cloud_incident_audit_data_imageName + target_field: _ingest._value.image.name + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: container.image.name + tag: append_prisma_cloud_incident_audit_data_imageName_into_container_image_name_1 + value: '{{{_ingest._value.image.name}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.interactive + tag: convert_prisma_cloud_incident_audit_data_interactive_to_boolean + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.interactive + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + tag: convert_prisma_cloud_incident_audit_data_ip_to_ip + type: ip + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.ip + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: related.ip + tag: append_prisma_cloud_incident_audit_data_ip_into_related_ip_1 + value: '{{{_ingest._value.ip}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.pid + tag: convert_prisma_cloud_incident_audit_data_pid_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.pid + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.port + tag: convert_prisma_cloud_incident_audit_data_port_to_long + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.port + ignore_missing: true + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.processPath + tag: rename_prisma_cloud_incident_audit_data_processPath + target_field: _ingest._value.process_path + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.profileId + tag: rename_prisma_cloud_incident_audit_data_profileId + target_field: _ingest._value.profile_id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: cloud.provider + tag: append_prisma_cloud_incident_audit_data_provider_into_cloud_provider + value: '{{{_ingest._value.provider}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.rawEvent + tag: rename_prisma_cloud_incident_audit_data_rawEvent + target_field: _ingest._value.raw_event + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.requestID + tag: rename_prisma_cloud_incident_audit_data_requestID + target_field: _ingest._value.request_id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.resourceID + tag: rename_prisma_cloud_incident_audit_data_audit_resourceID + target_field: _ingest._value.resource_id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.ruleName + tag: rename_prisma_cloud_incident_audit_data_ruleName + target_field: _ingest._value.rule_name + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: rule.name + tag: append_prisma_cloud_incident_audit_data_ruleName_into_rule_name_1 + value: '{{{_ingest._value.rule_name}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.time + target_field: _ingest._value.time + formats: + - ISO8601 + on_failure: + - remove: + field: _ingest._value.time + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + append: + field: related.user + tag: append_prisma_cloud_incident_audit_data_user_into_related_user_1 + value: '{{{_ingest._value.user}}}' + allow_duplicates: false + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.vmID + tag: rename_prisma_cloud_incident_audit_data_vmId + target_field: _ingest._value.vm_id + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.wildFireReportURL + tag: rename_prisma_cloud_incident_audit_data_wildFireReportURL + target_field: _ingest._value.wild_fire_report_url + ignore_missing: true + - foreach: + field: prisma_cloud.incident_audit.data + if: ctx.prisma_cloud?.incident_audit?.data instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.attack.type + - _ingest._value.attack.techniques + - _ingest._value.container.name + - _ingest._value.image.name + - _ingest._value.rule_name + - _ingest._value.provider + - _ingest._value.fqdn + - _ingest._value.account_id + ignore_missing: true + tag: remove_apps_data_fields_ecs_duplicate + - append: + field: event.category + tag: append_event_category + value: malware + if: ctx.json?.category != null && ctx.json.category.toLowerCase().contains('malware') + - append: + field: event.type + tag: append_event_type + value: info + if: ctx.event?.category != null + - rename: + field: json.category + tag: rename_prisma_cloud_incident_audit_category + target_field: prisma_cloud.incident_audit.category + ignore_missing: true + - rename: + field: json.cluster + tag: rename_prisma_cloud_incident_audit_cluster + target_field: prisma_cloud.incident_audit.cluster + ignore_missing: true + - rename: + field: json.collections + tag: rename_prisma_cloud_incident_audit_collections + target_field: prisma_cloud.incident_audit.collections + ignore_missing: true + - rename: + field: json.containerID + tag: rename_prisma_cloud_incident_audit_containerID + target_field: prisma_cloud.incident_audit.container.id + ignore_missing: true + - set: + field: container.id + tag: set_prisma_cloud_incident_audit_container_id + copy_from: prisma_cloud.incident_audit.container.id + ignore_empty_value: true + - rename: + field: json.containerName + tag: rename_prisma_cloud_incident_audit_containerName + target_field: prisma_cloud.incident_audit.container.name + ignore_missing: true + - append: + field: container.name + tag: append_prisma_cloud_incident_audit_container_name + value: '{{{prisma_cloud.incident_audit.container.name}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.incident_audit?.container?.name != null + - rename: + field: json.customRuleName + tag: rename_prisma_cloud_incident_audit_customRuleName + target_field: prisma_cloud.incident_audit.custom_rule_name + ignore_missing: true + - append: + field: rule.name + tag: append_prisma_cloud_incident_audit_rule_name + value: '{{{prisma_cloud.incident_audit.custom_rule_name}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.incident_audit?.custom_rule_name != null + - rename: + field: json.fqdn + tag: rename_prisma_cloud_incident_audit_fqdn + target_field: prisma_cloud.incident_audit.fqdn + ignore_missing: true + - append: + field: host.domain + tag: append_prisma_cloud_incident_audit_fqdn + value: '{{{prisma_cloud.incident_audit.fqdn}}}' + allow_duplicates: false + - append: + field: related.hosts + tag: append_prisma_cloud_incident_audit_fqdn_to_related_hosts + value: '{{{prisma_cloud.incident_audit.fqdn}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.incident_audit?.fqdn != null + - rename: + field: json.function + tag: rename_prisma_cloud_incident_audit_function + target_field: prisma_cloud.incident_audit.function.value + ignore_missing: true + - rename: + field: json.functionID + tag: rename_prisma_cloud_incident_audit_functionID + target_field: prisma_cloud.incident_audit.function.id + ignore_missing: true + - rename: + field: json.hostname + tag: rename_prisma_cloud_incident_audit_hostname + target_field: prisma_cloud.incident_audit.hostname + ignore_missing: true + - set: + field: host.hostname + tag: set_prisma_cloud_incident_audit_hostname + copy_from: prisma_cloud.incident_audit.hostname + ignore_empty_value: true + - append: + field: related.hosts + tag: append_prisma_cloud_incident_audit_hostname + value: '{{{prisma_cloud.incident_audit.hostname}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.incident_audit?.hostname != null + - rename: + field: json.imageID + tag: rename_prisma_cloud_incident_audit_imageID + target_field: prisma_cloud.incident_audit.image.id + ignore_missing: true + - rename: + field: json.imageName + tag: rename_prisma_cloud_incident_audit_imageName + target_field: prisma_cloud.incident_audit.image.name + ignore_missing: true + - append: + field: container.image.name + tag: append_prisma_cloud_incident_audit_image_name + value: '{{{prisma_cloud.incident_audit.image.name}}}' + allow_duplicates: false + if: ctx.prisma_cloud?.incident_audit?.image?.name != null + - rename: + field: json.labels + tag: rename_prisma_cloud_incident_audit_labels + target_field: prisma_cloud.incident_audit.labels + ignore_missing: true + - rename: + field: json.namespace + tag: rename_prisma_cloud_incident_audit_namespace + target_field: prisma_cloud.incident_audit.namespace + ignore_missing: true + - rename: + field: json.profileID + tag: rename_prisma_cloud_incident_audit_profileID + target_field: prisma_cloud.incident_audit.profile_id + ignore_missing: true + - rename: + field: json.provider + tag: rename_prisma_cloud_incident_audit_provider + target_field: prisma_cloud.incident_audit.provider + ignore_missing: true + - append: + field: cloud.provider + tag: append_prisma_cloud_incident_audit_provider_into_cloud_provider + value: '{{{prisma_cloud.incident_audit.provider}}}' + allow_duplicates: false + - rename: + field: json.region + tag: rename_prisma_cloud_incident_audit_region + target_field: prisma_cloud.incident_audit.region + ignore_missing: true + - set: + field: cloud.region + tag: set_prisma_cloud_incident_audit_region + copy_from: prisma_cloud.incident_audit.region + ignore_empty_value: true + - rename: + field: json.resourceID + tag: rename_prisma_cloud_incident_audit_resourceID + target_field: prisma_cloud.incident_audit.resource_id + ignore_missing: true + - rename: + field: json.runtime + tag: rename_prisma_cloud_incident_audit_runtime + target_field: prisma_cloud.incident_audit.runtime + ignore_missing: true + - convert: + field: json.serialNum + tag: convert_prisma_cloud_incident_audit_serialNum + target_field: prisma_cloud.incident_audit.serial_num + type: long + ignore_missing: true + if: ctx.json?.serialNum != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - convert: + field: json.shouldCollect + tag: convert_prisma_cloud_incident_audit_shouldCollect + target_field: prisma_cloud.incident_audit.should_collect + type: boolean + ignore_missing: true + if: ctx.json?.shouldCollect != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - rename: + field: json.type + tag: rename_prisma_cloud_incident_audit_type + target_field: prisma_cloud.incident_audit.type + ignore_missing: true + - rename: + field: json.vmID + tag: rename_prisma_cloud_incident_audit_vmID + target_field: prisma_cloud.incident_audit.vm_id + ignore_missing: true + - convert: + field: json.windows + tag: convert_prisma_cloud_incident_audit_windows + target_field: prisma_cloud.incident_audit.windows + type: boolean + ignore_missing: true + if: ctx.json?.windows != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.time + tag: date_prisma_cloud_incident_audit_time + formats: + - ISO8601 + if: ctx.json?.time != null && ctx.json.time != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.time + tag: 'date_rename_time_to_custom_name' + target_field: prisma_cloud.incident_audit.time + formats: + - ISO8601 + if: ctx.json?.time != null && ctx.json.time != '' + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - remove: + field: + - json + ignore_missing: true + - remove: + field: + - event.original + ignore_missing: true + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + - remove: + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + ignore_missing: true + tag: remove_preserve_duplicate_custom_fields + field: + - prisma_cloud.incident_audit._id + - prisma_cloud.incident_audit.account_id + - prisma_cloud.incident_audit.container.id + - prisma_cloud.incident_audit.container.name + - prisma_cloud.incident_audit.custom_rule_name + - prisma_cloud.incident_audit.fqdn + - prisma_cloud.incident_audit.hostname + - prisma_cloud.incident_audit.image.name + - prisma_cloud.incident_audit.provider + - prisma_cloud.incident_audit.region + - prisma_cloud.incident_audit.time + - prisma_cloud.incident_audit.category + - script: + lang: painless + description: Drops null/empty values recursively. + source: |- + boolean drop(Object object) { + if (object == null || object == '') { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(v -> drop(v)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(v -> drop(v)); + return (((List) object).length == 0); + } + return false; + } + drop(ctx); + - set: + field: event.kind + value: pipeline_error + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag fail-{{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - set: + field: event.kind + value: pipeline_error \ No newline at end of file diff --git a/packages/prisma_cloud/data_stream/incident_audit/fields/base-fields.yml b/packages/prisma_cloud/data_stream/incident_audit/fields/base-fields.yml new file mode 100644 index 00000000000..caed960552d --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: prisma_cloud +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: prisma_cloud.incident_audit +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/prisma_cloud/data_stream/incident_audit/fields/beats.yml b/packages/prisma_cloud/data_stream/incident_audit/fields/beats.yml new file mode 100644 index 00000000000..b3701b581cf --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/fields/beats.yml @@ -0,0 +1,9 @@ +- name: input.type + type: keyword + description: Type of filebeat input. +- name: log.offset + type: long + description: Log offset. +- name: tags + type: keyword + description: User defined tags. diff --git a/packages/prisma_cloud/data_stream/incident_audit/fields/fields.yml b/packages/prisma_cloud/data_stream/incident_audit/fields/fields.yml new file mode 100644 index 00000000000..7ff9b9ef3b3 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/fields/fields.yml @@ -0,0 +1,276 @@ +- name: prisma_cloud + type: group + fields: + - name: incident_audit + type: group + fields: + - name: _id + type: keyword + description: Internal ID of the incident. + - name: account_id + type: keyword + description: Cloud account ID. + - name: acknowledged + type: boolean + description: Indicates if the incident has been acknowledged (true) or not (false). + - name: app + type: group + fields: + - name: id + type: keyword + description: Application Id. + - name: value + type: keyword + description: Application that caused the incident. + - name: category + type: keyword + - name: cluster + type: keyword + description: Cluster on which the incident was found. + - name: collections + type: keyword + description: Collections to which this incident applies. + - name: container + type: group + fields: + - name: id + type: keyword + description: ID of the container that triggered the incident. + - name: name + type: keyword + description: Container name. + - name: custom_rule_name + type: keyword + description: Name of the custom runtime rule that triggered the incident. + - name: data + type: group + fields: + - name: _id + type: keyword + description: Internal ID of the incident. + - name: account_id + type: keyword + description: ID of the cloud account where the audit was generated. + - name: app + type: group + fields: + - name: id + type: keyword + description: Application id. + - name: value + type: keyword + description: Name of the service which violated the host policy. + - name: attack + type: group + fields: + - name: techniques + type: keyword + description: Given list of techniques in documentation. + - name: type + type: keyword + description: Given list in documentation.RuntimeAttackType is the sub-category of the attack (e.g., malware process, process not in model, etc...). + - name: cluster + type: keyword + description: Cluster name. + - name: collections + type: keyword + description: Collections to which this audit applies. + - name: command + type: keyword + description: ScrubbedCommand is the command executed by the process with scrubbed PII. + - name: container + type: group + fields: + - name: id + type: keyword + description: ID of the container that violates the rule. + - name: name + type: keyword + description: Container name. + - name: value + type: boolean + description: Indicates if this is a container audit (true) or host audit (false). + - name: count + type: long + description: Attack type audits count. + - name: country + type: keyword + description: Outbound country for outgoing network audits. + - name: domain + type: keyword + description: Domain is the requested domain. + - name: effect + type: keyword + description: 'Possible values: [block,prevent,alert,disable]RuleEffect is the effect that will be used in the runtime rule.' + - name: err + type: keyword + description: Unknown error in the audit process. + - name: filepath + type: keyword + description: Filepath is the path of the modified file. + - name: fqdn + type: keyword + description: Current full domain name used in audit alerts. + - name: function + type: group + fields: + - name: id + type: keyword + description: Id of function invoked. + - name: value + type: keyword + description: Name of the serverless function that caused the audit. + - name: hostname + type: keyword + description: current hostname. + - name: image + type: group + fields: + - name: id + type: keyword + description: Container image Id. + - name: name + type: keyword + description: Container image name. + - name: interactive + type: boolean + description: Indicates if the audit was triggered from a process that was spawned in interactive mode (e.g., docker exec ...) (true) or not (false). + - name: ip + type: ip + description: IP is the connection destination IP address. + - name: label + type: keyword + description: Container deployment label. + - name: labels + type: flattened + - name: md5 + type: keyword + description: MD5 is the MD5 of the modified file (only for executables). + - name: msg + type: keyword + description: Blocking message text. + - name: namespace + type: keyword + description: K8s deployment namespace. + - name: os + type: keyword + description: Operating system distribution. + - name: pid + type: long + description: ID of the process that caused the audit event. + - name: port + type: long + description: Port is the connection destination port. + - name: process_path + type: keyword + description: Path of the process that caused the audit event. + - name: profile_id + type: keyword + description: Profile ID of the audit. + - name: provider + type: keyword + description: "Possible values: [aws,azure,gcp,alibaba,oci,others]. CloudProvider specifies the cloud provider name." + - name: raw_event + type: keyword + description: Unparsed function handler event input. + - name: region + type: keyword + description: Region of the resource where the audit was generated. + - name: request_id + type: keyword + description: ID of the lambda function invocation request. + - name: resource_id + type: keyword + description: Unique ID of the resource where the audit was generated. + - name: rule_name + type: keyword + description: Name of the rule that was applied, if blocked. + - name: runtime + type: keyword + description: '[python,python3.6,python3.7,python3.8,python3.9,nodejs12.x,nodejs14.x,dotnetcore2.1,dotnetcore3.1,dotnet6,java8,java11,ruby2.7].' + - name: severity + type: keyword + description: Possible value [high, low, medium]. + - name: time + type: date + description: Time of the audit event (in UTC time). + - name: type + type: keyword + description: "Possible values: [processes,network,kubernetes,filesystem]\r\nRuntimeType represents the runtime protection type." + - name: user + type: keyword + description: Service user. + - name: version + type: keyword + description: Defender version. + - name: vm_id + type: keyword + description: Azure unique VM ID where the audit was generated. + - name: wild_fire_report_url + type: keyword + description: WildFireReportURL is a URL link of the report generated by wildFire. + - name: fqdn + type: keyword + description: Current hostname's full domain name. + - name: function + type: group + fields: + - name: id + type: keyword + description: ID of the function that triggered the incident. + - name: value + type: keyword + description: Name of the serverless function. + - name: hostname + type: keyword + description: Current hostname. + - name: image + type: group + fields: + - name: id + type: keyword + description: Container image id. + - name: name + type: keyword + description: Container image name. + - name: labels + type: flattened + - name: namespace + type: keyword + description: k8s deployment namespace. + - name: profile_id + type: keyword + description: Runtime profile ID. + - name: provider + type: keyword + description: 'Possible values: [aws,azure,gcp,alibaba,oci,others].' + - name: region + type: keyword + description: Region of the resource on which the incident was found. + - name: resource_id + type: keyword + description: Unique ID of the resource on which the incident was found. + - name: runtime + type: keyword + description: Runtime of the serverless function. + - name: serial_num + type: long + description: Serial number of incident. + - name: should_collect + type: boolean + description: Indicates if this incident should be collected (true) or not (false). + - name: time + type: date + description: Time of the incident (in UTC time). + - name: type + type: keyword + description: 'Possible values: [host,container,function,appEmbedded,fargate].' + - name: vm_id + type: keyword + description: Azure unique VM ID on which the incident was found. + - name: windows + type: boolean + description: Windows indicates if defender OS type is Windows. +- name: log.source.address + type: keyword + description: Source address from which the log event was read / sent from. diff --git a/packages/prisma_cloud/data_stream/incident_audit/manifest.yml b/packages/prisma_cloud/data_stream/incident_audit/manifest.yml new file mode 100644 index 00000000000..38663f07310 --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/manifest.yml @@ -0,0 +1,135 @@ +title: Collect Incident Audit logs from Prisma Cloud Workload Protection. +type: logs +streams: + - input: tcp + template_path: tcp.yml.hbs + title: Incident Audit logs + description: Collect Incident Audit logs via TCP input. + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for TCP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The TCP port number to listen on. + multi: false + required: true + show_user: true + default: 9508 + - name: tcp_options + type: yaml + title: Custom TCP Options + multi: false + required: false + show_user: false + default: | + #max_message_size: 20MiB + #max_connections: 1 + #framing: delimiter + #line_delimiter: "\n" + description: Specify custom configuration options for the TCP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-incident_audit + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.incident_audit fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + template_path: udp.yml.hbs + title: Incident Audit logs + description: Collect Incident Audit logs via UDP input. + enabled: false + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for UDP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: listen_port + type: integer + title: Listen Port + description: The UDP port number to listen on. + multi: false + required: true + show_user: true + default: 9508 + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + #max_message_size: 10KiB + #timeout: 300s + description: Specify custom configuration options for the UDP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - prisma_cloud-incident_audit + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve prisma_cloud.incident_audit fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/prisma_cloud/data_stream/incident_audit/sample_event.json b/packages/prisma_cloud/data_stream/incident_audit/sample_event.json new file mode 100644 index 00000000000..a589f1a996a --- /dev/null +++ b/packages/prisma_cloud/data_stream/incident_audit/sample_event.json @@ -0,0 +1,218 @@ +{ + "@timestamp": "2023-08-30T08:42:17.834Z", + "agent": { + "ephemeral_id": "36c96ae0-38a3-4668-a74c-b41dae4b633f", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "cloud": { + "account": { + "id": [ + "accounttest" + ] + }, + "provider": [ + "aws" + ], + "region": "testregion" + }, + "container": { + "id": "testcontainer", + "image": { + "name": [ + "testimgname" + ] + }, + "name": [ + "testcontainername" + ] + }, + "data_stream": { + "dataset": "prisma_cloud.incident_audit", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "dataset": "prisma_cloud.incident_audit", + "id": "thgry1736", + "ingested": "2023-10-18T12:17:39Z", + "kind": "event", + "original": "{\"_id\":\"thgry1736\",\"accountID\":\"accounttest\",\"acknowledged\":true,\"app\":\"test\",\"appID\":\"test123\",\"audits\":[{\"_id\":\"id1234\",\"accountID\":\"accounttest\",\"app\":\"test\",\"appID\":\"test123\",\"attackTechniques\":[\"exploitationForPrivilegeEscalation\"],\"attackType\":[\"cloudMetadataProbing\"],\"cluster\":\"clustertest\",\"collections\":[\"collectiontest\"],\"command\":\"commandtest\",\"container\":true,\"containerId\":\"testcontainerid\",\"containerName\":\"testcontainername\",\"count\":0,\"country\":\"in\",\"domain\":\"testdomain\",\"effect\":\"block\",\"err\":\"testerr\",\"filepath\":\"testfilepath\",\"fqdn\":\"testfqdn\",\"function\":\"testfun\",\"functionID\":\"testfunid\",\"hostname\":\"testhostname\",\"imageId\":\"testimgid\",\"imageName\":\"testimgname\",\"interactive\":true,\"ip\":\"81.2.69.142\",\"label\":\"testlabel\",\"labels\":{\"sjhia\": \"ifo\"},\"md5\":\"testmd5\",\"msg\":\"testmsg\",\"namespace\":\"testnamespace\",\"os\":\"testos\",\"pid\":0,\"port\":0,\"processPath\":\"testprocesspath\",\"profileId\":\"testprofileid\",\"provider\":\"aws\",\"rawEvent\":\"testrawevent\",\"region\":\"testregion\",\"requestID\":\"testrequestid\",\"resourceID\":\"testresourceid\",\"ruleName\":\"testrulename\",\"runtime\":\"python\",\"severity\":\"low\",\"time\":\"2023-08-30T08:42:17.834Z\",\"type\":\"processes\",\"user\":\"testuser\",\"version\":\"testversion\",\"vmID\":\"testvmid\",\"wildFireReportURL\":\"testwildfirereporturl\"}],\"category\":\"portScanning\",\"cluster\":\"testcluster\",\"collections\":[\"testcollections\"],\"containerID\":\"testcontainer\",\"containerName\":\"testcontainername\",\"customRuleName\":\"testcustomrulename\",\"fqdn\":\"testfqdn\",\"function\":\"testfunction\",\"functionID\":\"testfunctionid\",\"hostname\":\"testhostname\",\"imageID\":\"testimgid\",\"imageName\":\"testimgname\",\"labels\":{\"aaa\":\"bbb\"},\"namespace\":\"testnamespace\",\"profileID\":\"testprofileid\",\"provider\":\"aws\",\"region\":\"testregion\",\"resourceID\":\"testresourceid\",\"runtime\":\"testruntime\",\"serialNum\":0,\"shouldCollect\":true,\"time\":\"2023-08-30T08:42:17.834Z\",\"type\":\"host\",\"vmID\":\"testvmid\",\"windows\":true}" + }, + "host": { + "domain": [ + "testfqdn" + ], + "hostname": "testhostname" + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.48.7:57454" + } + }, + "prisma_cloud": { + "incident_audit": { + "_id": "thgry1736", + "account_id": "accounttest", + "acknowledged": true, + "app": { + "id": "test123", + "value": "test" + }, + "category": "portScanning", + "cluster": "testcluster", + "collections": [ + "testcollections" + ], + "container": { + "id": "testcontainer", + "name": "testcontainername" + }, + "custom_rule_name": "testcustomrulename", + "data": [ + { + "_id": "id1234", + "account_id": "accounttest", + "app": { + "id": "test123", + "value": "test" + }, + "attack": { + "techniques": [ + "exploitationForPrivilegeEscalation" + ], + "type": [ + "cloudMetadataProbing" + ] + }, + "cluster": "clustertest", + "collections": [ + "collectiontest" + ], + "command": "commandtest", + "container": { + "id": "testcontainerid", + "name": "testcontainername", + "value": true + }, + "count": 0, + "country": "in", + "domain": "testdomain", + "effect": "block", + "err": "testerr", + "filepath": "testfilepath", + "fqdn": "testfqdn", + "function": { + "id": "testfunid", + "value": "testfun" + }, + "hostname": "testhostname", + "image": { + "id": "testimgid", + "name": "testimgname" + }, + "interactive": true, + "ip": "81.2.69.142", + "label": "testlabel", + "labels": { + "sjhia": "ifo" + }, + "md5": "testmd5", + "msg": "testmsg", + "namespace": "testnamespace", + "os": "testos", + "pid": 0, + "port": 0, + "process_path": "testprocesspath", + "profile_id": "testprofileid", + "provider": "aws", + "raw_event": "testrawevent", + "region": "testregion", + "request_id": "testrequestid", + "resource_id": "testresourceid", + "rule_name": "testrulename", + "runtime": "python", + "severity": "low", + "time": "2023-08-30T08:42:17.834Z", + "type": "processes", + "user": "testuser", + "version": "testversion", + "vm_id": "testvmid", + "wild_fire_report_url": "testwildfirereporturl" + } + ], + "fqdn": "testfqdn", + "function": { + "id": "testfunctionid", + "value": "testfunction" + }, + "hostname": "testhostname", + "image": { + "id": "testimgid", + "name": "testimgname" + }, + "labels": { + "aaa": "bbb" + }, + "namespace": "testnamespace", + "profile_id": "testprofileid", + "provider": "aws", + "region": "testregion", + "resource_id": "testresourceid", + "runtime": "testruntime", + "serial_num": 0, + "should_collect": true, + "time": "2023-08-30T08:42:17.834Z", + "type": "host", + "vm_id": "testvmid", + "windows": true + } + }, + "related": { + "hosts": [ + "testfqdn", + "testhostname" + ], + "ip": [ + "81.2.69.142" + ], + "user": [ + "testuser" + ] + }, + "rule": { + "name": [ + "testrulename", + "testcustomrulename" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-incident_audit" + ], + "threat": { + "technique": { + "subtechnique": { + "name": [ + "{0=cloudMetadataProbing}" + ] + } + } + } +} \ No newline at end of file diff --git a/packages/prisma_cloud/docs/README.md b/packages/prisma_cloud/docs/README.md new file mode 100644 index 00000000000..e5ef0326e90 --- /dev/null +++ b/packages/prisma_cloud/docs/README.md @@ -0,0 +1,2040 @@ +# Prisma Cloud + +This [Prisma Cloud](https://docs.paloaltonetworks.com/prisma/prisma-cloud/prisma-cloud-admin-compute/welcome) is a cloud infrastructure security solution and a Security Operations Center (SOC) enablement tool that enables you to address risks and secure your workloads in a heterogeneous environment (hybrid and multi cloud) from a single console. It provides complete visibility and control over risks within your public cloud infrastructure—Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Oracle Cloud Infrastructure (OCI), Alibaba Cloud— and enables you to manage vulnerabilities, detect anomalies, ensure compliance, and provide runtime defense in heterogeneous environments, such as Windows, Linux, Kubernetes, Red Hat OpenShift, AWS Lambda, Azure Functions, and GCP Cloud Functions. + +## Prisma Cloud Security Posture Management (CSPM) + +Single pane of glass for both CSPM (Cloud Security Posture Management) & CWPP (Cloud Workload Protection Platform). Compute (formerly Twistlock, a CWPP solution) is delivered as part of the larger Prisma Cloud system. Palo Alto Networks runs, manages, and updates Compute Console for you. You deploy and manage Defenders in your environment. You access the Compute Console from a tab within the Prisma Cloud user interface. + +CSPM uses REST API mode to collect data. Elastic Agent fetches data via API endpoints. + +## Prisma Cloud Workload Protection (CWP) + +Self-hosted, stand-alone, self-operated version of Compute (formerly Twistlock). Download the entire software suite, and run it in any environment. You deploy and manage both Console and Defenders. + +CWP can be used in two different modes to collect data: +- REST API mode. +- Syslog mode: This includes TCP and UDP. + +## Compatibility + +This module has been tested against the latest CSPM version **v2** and CWP version **v30.03**. + +## Data streams + +The Prisma Cloud integration collects data for the following five events: + +| Event Type | +|-------------------------------| +| Alert | +| Audit | +| Host | +| Host Profile | +| Incident Audit | + +**NOTE**: + +1. Alert and Audit data-streams are part of [CSPM](https://pan.dev/prisma-cloud/api/cspm/) module, whereas Host, Host Profile and Incident Audit are part of [CWP](https://pan.dev/prisma-cloud/api/cwpp/) module. +2. Currently, we are unable to collect logs of Incident Audit datastream via defined API. Hence, we have not added the configuration of Incident Audit data stream via REST API. + +## Requirements + +- Elastic Agent must be installed. +- You can install only one Elastic Agent per host. +- Elastic Agent is required to stream data through the REST API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines. + +### Installing and managing an Elastic Agent: + +You have a few options for installing and managing an Elastic Agent: + +### Install a Fleet-managed Elastic Agent (recommended): + +With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier. + +### Install Elastic Agent in standalone mode (advanced users): + +With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only. + +### Install Elastic Agent in a containerized environment: + +You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes. + +There are some minimum requirements for running Elastic Agent and for more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html). + +The minimum **kibana.version** required is **8.10.1**. + +## Setup + +### To collect data through REST API, follow the below steps: + +### CSPM + +1. Considering you already have a Prisma Cloud account, to obtain an access key ID and secret access key from the Prisma Cloud system administrator, refer this [link](https://docs.paloaltonetworks.com/prisma/prisma-cloud/prisma-cloud-admin/manage-prisma-cloud-administrators/create-access-keys). +2. The base URL of your CSPM API request depends on the region of your Prisma Cloud tenant and is similar to your Prisma Cloud administrative console URL. Obtain your URL from this [link](https://pan.dev/prisma-cloud/api/cspm/api-urls/). + +### CWP + +1. Assuming you've already generated your access key ID and secret access key from the Prisma Cloud Console; if not, see the section above. +2. The base URL of your CWP API request depends on the console path and the API version of your Prisma Cloud Compute console. +3. To find your API version, log in to your Prisma Cloud Compute console, click the bell icon in the top right of the page, your API version is displayed. +4. To get your console path, navigate to Compute > Manage > System > Downloads. you can find your console path listed under Path to Console. +5. Now you can create your base URL in this format: `https:///api/v`. + +**NOTE**: You can specify a date and time for the access key validity. If you do not select key expiry, the key is set to never expire; if you select it, but do not specify a date, the key expires in a month. + +### Enabling the integration in Elastic: + +1. In Kibana go to Management > Integrations +2. In "Search for integrations" search bar, type Palo Alto Prisma Cloud. +3. Click on the "Palo Alto Prisma Cloud" integration from the search results. +4. Click on the Add Palo Alto Prisma Cloud Integration button to add the integration. +5. While adding the integration, if you want to collect Alert and Audit data via REST API, then you have to put the following details: + - username + - password + - url + - interval + - time amount + - time unit + - batch size + + or if you want to collect Host, Host Profile and Incident Audit data via REST API, then you have to put the following details: + - username + - password + - url + - interval + - offset + - batch size + + or if you want to collect Host, Host Profile and Incident Audit data via TCP/UDP, then you have to put the following details: + - listen address + - listen port + +**NOTE**: Your Access key ID is your username and Secret Access key is your password. + +## Logs Reference + +### Alert + +This is the `Alert` dataset. + +#### Example + +An example event for `alert` looks as following: + +```json +{ + "@timestamp": "2023-09-06T12:30:41.966Z", + "agent": { + "ephemeral_id": "7aae6130-635a-422f-ac2e-e40324e86921", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "cloud": { + "account": { + "id": "710002259376" + }, + "provider": "aws", + "service": { + "name": "Amazon EC2" + } + }, + "data_stream": { + "dataset": "prisma_cloud.alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "threat" + ], + "dataset": "prisma_cloud.alert", + "end": "2023-09-06T12:30:41.966Z", + "id": "N-3910", + "ingested": "2023-10-18T12:08:31Z", + "kind": "alert", + "original": "{\"alertAdditionalInfo\":{\"scannerVersion\":\"CS_2.0\"},\"alertAttribution\":{\"attributionEventList\":[{\"event\":\"first_event\",\"event_ts\":1694003441966,\"username\":\"alex123\"}],\"resourceCreatedBy\":\"string\",\"resourceCreatedOn\":0},\"alertRules\":[],\"alertTime\":1694003441966,\"firstSeen\":1694003441966,\"history\":[{\"modifiedBy\":\"alex123\",\"modifiedOn\":\"1694003441966\",\"reason\":\"Reason1\",\"status\":\"OPEN\"}],\"id\":\"N-3910\",\"investigateOptions\":{\"alertId\":\"N-3910\"},\"lastSeen\":1694003441966,\"lastUpdated\":1694003441966,\"metadata\":null,\"policy\":{\"complianceMetadata\":[{\"complianceId\":\"qwer345bv\",\"customAssigned\":true,\"policyId\":\"werf435tr\",\"requirementDescription\":\"Description of policy compliance.\",\"requirementId\":\"req-123-xyz\",\"requirementName\":\"rigidity\",\"sectionDescription\":\"Description of section.\",\"sectionId\":\"sect-453-abc\",\"sectionLabel\":\"label-1\",\"standardDescription\":\"Description of standard.\",\"standardId\":\"stand-543-pqr\",\"standardName\":\"Class 1\"}],\"deleted\":false,\"description\":\"This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.\",\"findingTypes\":[],\"labels\":[\"Prisma_Cloud\",\"Attack Path Rule\"],\"lastModifiedBy\":\"template@redlock.io\",\"lastModifiedOn\":1687474999057,\"name\":\"AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0/0)\",\"policyId\":\"ad23603d-754e-4499-8988-b8017xxxx98\",\"policyType\":\"network\",\"recommendation\":\"The following steps are recommended to restrict unrestricted access from the Internet:\\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\\n2. Identify the network component on which restrictive rules can be implemented.\\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\\n a) The overly permissive Security Group rules can be made more restrictive.\\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.\",\"remediable\":false,\"remediation\":{\"actions\":[{\"operation\":\"buy\",\"payload\":\"erefwsdf\"}],\"cliScriptTemplate\":\"temp1\",\"description\":\"Description of CLI Script Template.\"},\"severity\":\"high\",\"systemDefault\":true},\"policyId\":\"ad23603d-754e-4499-8988-b801xxx85898\",\"reason\":\"NEW_ALERT\",\"resource\":{\"account\":\"AWS Cloud Account\",\"accountId\":\"710002259376\",\"additionalInfo\":null,\"cloudAccountGroups\":[\"Default Account Group\"],\"cloudServiceName\":\"Amazon EC2\",\"cloudType\":\"aws\",\"data\":null,\"id\":\"i-04578exxxx8100947\",\"name\":\"IS-37133\",\"region\":\"AWS Virginia\",\"regionId\":\"us-east-1\",\"resourceApiName\":\"aws-ec2-describe-instances\",\"resourceConfigJsonAvailable\":false,\"resourceDetailsAvailable\":true,\"resourceTs\":1694003441915,\"resourceType\":\"INSTANCE\",\"rrn\":\"rrn:aws:instance:us-east-1:710000059376:e7ddce5a1ffcb47bxxxxxerf2635a3b4d9da3:i-04578e0008100947\",\"unifiedAssetId\":\"66c543b6261c4d9edxxxxxb42e15f4\",\"url\":\"https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947\"},\"status\":\"open\"}", + "start": "2023-09-06T12:30:41.966Z", + "type": [ + "indicator" + ] + }, + "input": { + "type": "cel" + }, + "prisma_cloud": { + "alert": { + "additional_info": { + "scanner_version": "CS_2.0" + }, + "attribution": { + "event_list": [ + { + "ts": "2023-09-06T12:30:41.966Z", + "username": "alex123", + "value": "first_event" + } + ], + "resource": { + "created_by": "string", + "created_on": "1970-01-01T00:00:00.000Z" + } + }, + "first_seen": "2023-09-06T12:30:41.966Z", + "history": [ + { + "modified_by": "alex123", + "modified_on": "2023-09-06T12:30:41.966Z", + "reason": "Reason1", + "status": "OPEN" + } + ], + "id": "N-3910", + "last": { + "seen": "2023-09-06T12:30:41.966Z", + "updated": "2023-09-06T12:30:41.966Z" + }, + "policy": { + "compliance_metadata": [ + { + "compliance_id": "qwer345bv", + "custom_assigned": true, + "policy_id": "werf435tr", + "requirement": { + "description": "Description of policy compliance.", + "id": "req-123-xyz", + "name": "rigidity" + }, + "section": { + "description": "Description of section.", + "id": "sect-453-abc", + "label": "label-1" + }, + "standard": { + "description": "Description of standard.", + "id": "stand-543-pqr", + "name": "Class 1" + } + } + ], + "deleted": false, + "description": "This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.", + "id": "ad23603d-754e-4499-8988-b8017xxxx98", + "labels": [ + "Prisma_Cloud", + "Attack Path Rule" + ], + "last_modified_by": "template@redlock.io", + "last_modified_on": "2023-06-22T23:03:19.057Z", + "name": "AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0/0)", + "recommendation": "The following steps are recommended to restrict unrestricted access from the Internet:\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\n2. Identify the network component on which restrictive rules can be implemented.\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\n a) The overly permissive Security Group rules can be made more restrictive.\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.", + "remediable": false, + "remediation": { + "actions": [ + { + "operation": "buy", + "payload": "erefwsdf" + } + ], + "cli_script_template": "temp1", + "description": "Description of CLI Script Template." + }, + "severity": "high", + "system_default": true, + "type": "network" + }, + "policy_id": "ad23603d-754e-4499-8988-b801xxx85898", + "reason": "NEW_ALERT", + "resource": { + "account": { + "id": "710002259376", + "value": "AWS Cloud Account" + }, + "api_name": "aws-ec2-describe-instances", + "cloud": { + "account": { + "groups": [ + "Default Account Group" + ] + }, + "service_name": "Amazon EC2", + "type": "aws" + }, + "config_json_available": false, + "details_available": true, + "id": "i-04578exxxx8100947", + "name": "IS-37133", + "region": { + "id": "us-east-1", + "value": "AWS Virginia" + }, + "rrn": "rrn:aws:instance:us-east-1:710000059376:e7ddce5a1ffcb47bxxxxxerf2635a3b4d9da3:i-04578e0008100947", + "ts": "2023-09-06T12:30:41.915Z", + "type": "INSTANCE", + "unified_asset_id": "66c543b6261c4d9edxxxxxb42e15f4", + "url": "https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947" + }, + "status": "open", + "time": "2023-09-06T12:30:41.966Z" + } + }, + "related": { + "user": [ + "alex123" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-alert" + ], + "url": { + "domain": "console.aws.amazon.com", + "fragment": "Instances:instanceId=i-0457xxxxx00947", + "original": "https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947", + "path": "/ec2/v2/home", + "query": "region=us-east-1", + "scheme": "https" + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| event.dataset | Event dataset. | constant_keyword | +| event.module | Event module. | constant_keyword | +| input.type | Type of filebeat input. | keyword | +| log.offset | Log offset. | long | +| prisma_cloud.alert.additional_info.scanner_version | | keyword | +| prisma_cloud.alert.attribution.event_list.ts | | date | +| prisma_cloud.alert.attribution.event_list.username | | keyword | +| prisma_cloud.alert.attribution.event_list.value | | keyword | +| prisma_cloud.alert.attribution.resource.created_by | | keyword | +| prisma_cloud.alert.attribution.resource.created_on | | date | +| prisma_cloud.alert.count | | long | +| prisma_cloud.alert.dismissal.duration | | keyword | +| prisma_cloud.alert.dismissal.note | | keyword | +| prisma_cloud.alert.dismissal.until_ts | | date | +| prisma_cloud.alert.dismissed_by | | keyword | +| prisma_cloud.alert.event_occurred | Timestamp when the event occurred. Set only for Audit Event policies. | date | +| prisma_cloud.alert.first_seen | Timestamp of the first policy violation for the alert resource (i.e. the alert creation timestamp). | date | +| prisma_cloud.alert.history.modified_by | | keyword | +| prisma_cloud.alert.history.modified_on | | date | +| prisma_cloud.alert.history.reason | | keyword | +| prisma_cloud.alert.history.status | | keyword | +| prisma_cloud.alert.id | Alert ID. | keyword | +| prisma_cloud.alert.last.seen | Timestamp when alert status was last updated. | date | +| prisma_cloud.alert.last.updated | Timestamp when alert was last updated. Updates include but are not limited to resource updates, policy updates, alert rule updates, and alert status changes. | date | +| prisma_cloud.alert.metadata.save_search_id | | keyword | +| prisma_cloud.alert.policy.cloud_type | Possible values: [ALL, AWS, AZURE, GCP, ALIBABA_CLOUD, OCI, IBM] Cloud type (Required for config policies). Not case-sensitive. Default is ALL. | keyword | +| prisma_cloud.alert.policy.compliance_metadata.compliance_id | Compliance Section UUID. | keyword | +| prisma_cloud.alert.policy.compliance_metadata.custom_assigned | | boolean | +| prisma_cloud.alert.policy.compliance_metadata.policy_id | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.requirement.description | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.requirement.id | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.requirement.name | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.requirement.view_order | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.section.description | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.section.id | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.section.label | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.section.view_order | | long | +| prisma_cloud.alert.policy.compliance_metadata.standard.description | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.standard.id | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.standard.name | | keyword | +| prisma_cloud.alert.policy.compliance_metadata.system_default | | boolean | +| prisma_cloud.alert.policy.created_by | | keyword | +| prisma_cloud.alert.policy.created_on | | date | +| prisma_cloud.alert.policy.deleted | | boolean | +| prisma_cloud.alert.policy.description | | keyword | +| prisma_cloud.alert.policy.enabled | | boolean | +| prisma_cloud.alert.policy.finding_types | | keyword | +| prisma_cloud.alert.policy.id | | keyword | +| prisma_cloud.alert.policy.labels | | keyword | +| prisma_cloud.alert.policy.last_modified_by | | keyword | +| prisma_cloud.alert.policy.last_modified_on | | date | +| prisma_cloud.alert.policy.name | | keyword | +| prisma_cloud.alert.policy.recommendation | | keyword | +| prisma_cloud.alert.policy.remediable | | boolean | +| prisma_cloud.alert.policy.remediation.actions.operation | | keyword | +| prisma_cloud.alert.policy.remediation.actions.payload | | keyword | +| prisma_cloud.alert.policy.remediation.cli_script_template | | keyword | +| prisma_cloud.alert.policy.remediation.description | | keyword | +| prisma_cloud.alert.policy.rule.api_name | | keyword | +| prisma_cloud.alert.policy.rule.cloud.account | | keyword | +| prisma_cloud.alert.policy.rule.cloud.type | | keyword | +| prisma_cloud.alert.policy.rule.criteria | Saved search ID that defines the rule criteria. | keyword | +| prisma_cloud.alert.policy.rule.data_criteria.classification_result | Data policy. Required for DLP rule criteria. | keyword | +| prisma_cloud.alert.policy.rule.data_criteria.exposure | Possible values [private, public, conditional]. | keyword | +| prisma_cloud.alert.policy.rule.data_criteria.extension | | keyword | +| prisma_cloud.alert.policy.rule.last_modified_on | | date | +| prisma_cloud.alert.policy.rule.name | | keyword | +| prisma_cloud.alert.policy.rule.parameters | | flattened | +| prisma_cloud.alert.policy.rule.resource.id_path | | keyword | +| prisma_cloud.alert.policy.rule.resource.type | | keyword | +| prisma_cloud.alert.policy.rule.type | Possible values [Config, Network, AuditEvent, DLP, IAM, NetworkConfig] Type of rule or RQL query. | keyword | +| prisma_cloud.alert.policy.severity | Possible values [high, medium, low]. | keyword | +| prisma_cloud.alert.policy.system_default | | boolean | +| prisma_cloud.alert.policy.type | Possible values: [config, network, audit_event, anomaly, data, iam, workload_vulnerability, workload_incident, waas_event, attack_path] Policy type. Policy type anomaly is read-only. | keyword | +| prisma_cloud.alert.policy.upi | | keyword | +| prisma_cloud.alert.policy_id | | keyword | +| prisma_cloud.alert.reason | | keyword | +| prisma_cloud.alert.resource.account.id | | keyword | +| prisma_cloud.alert.resource.account.value | | keyword | +| prisma_cloud.alert.resource.additional_info | Additional info. | flattened | +| prisma_cloud.alert.resource.api_name | | keyword | +| prisma_cloud.alert.resource.cloud.account.ancestors | | keyword | +| prisma_cloud.alert.resource.cloud.account.groups | | keyword | +| prisma_cloud.alert.resource.cloud.account.owners | | keyword | +| prisma_cloud.alert.resource.cloud.service_name | | keyword | +| prisma_cloud.alert.resource.cloud.type | | keyword | +| prisma_cloud.alert.resource.config_json_available | | boolean | +| prisma_cloud.alert.resource.data | | flattened | +| prisma_cloud.alert.resource.details_available | | boolean | +| prisma_cloud.alert.resource.id | | keyword | +| prisma_cloud.alert.resource.name | | keyword | +| prisma_cloud.alert.resource.region.id | | keyword | +| prisma_cloud.alert.resource.region.value | | keyword | +| prisma_cloud.alert.resource.rrn | | keyword | +| prisma_cloud.alert.resource.tags | | flattened | +| prisma_cloud.alert.resource.ts | | date | +| prisma_cloud.alert.resource.type | | keyword | +| prisma_cloud.alert.resource.unified_asset_id | | keyword | +| prisma_cloud.alert.resource.url | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.cloud_type | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.compliance.id | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.custom_assigned | | boolean | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.policy.id | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.requirement.description | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.requirement.id | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.requirement.name | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.section.description | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.section.id | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.section.label | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.standard.description | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.standard.id | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.compliance_metadata.standard.name | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.created.by | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.created.on | | date | +| prisma_cloud.alert.risk_detail.policy_scores.deleted | | boolean | +| prisma_cloud.alert.risk_detail.policy_scores.description | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.enabled | | boolean | +| prisma_cloud.alert.risk_detail.policy_scores.finding_types | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.labels | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.last_modified.by | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.last_modified.on | | date | +| prisma_cloud.alert.risk_detail.policy_scores.name | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.overridden | | boolean | +| prisma_cloud.alert.risk_detail.policy_scores.points | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.policy.id | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.policy.subtypes | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.policy.type | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.policy.upi | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.recommendation | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.remediable | | boolean | +| prisma_cloud.alert.risk_detail.policy_scores.remediation.actions.operation | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.remediation.actions.payload | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.remediation.cli_script_template | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.remediation.description | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.remediation.impact | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.restrict_alert_dismissal | | boolean | +| prisma_cloud.alert.risk_detail.policy_scores.risk_score.max | | long | +| prisma_cloud.alert.risk_detail.policy_scores.risk_score.value | | long | +| prisma_cloud.alert.risk_detail.policy_scores.rule.api_name | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.cloud.account | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.cloud.type | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.criteria | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.data_criteria.classification_result | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.data_criteria.exposure | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.data_criteria.extension | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.last_modified_on | | date | +| prisma_cloud.alert.risk_detail.policy_scores.rule.name | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.parameters | | flattened | +| prisma_cloud.alert.risk_detail.policy_scores.rule.resource.id_path | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.resource.type | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.rule.type | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.severity | | keyword | +| prisma_cloud.alert.risk_detail.policy_scores.system_default | | boolean | +| prisma_cloud.alert.risk_detail.rating | | keyword | +| prisma_cloud.alert.risk_detail.risk_score.max | | long | +| prisma_cloud.alert.risk_detail.risk_score.value | | long | +| prisma_cloud.alert.risk_detail.score | | keyword | +| prisma_cloud.alert.save_search_id | | keyword | +| prisma_cloud.alert.status | | keyword | +| prisma_cloud.alert.time | Timestamp when alert was last reopened for resource update, or the same as firstSeen if there are no status changes. | date | +| prisma_cloud.alert.triggered_by | | keyword | +| tags | User defined tags. | keyword | + + +### Audit + +This is the `Audit` dataset. + +#### Example + +An example event for `audit` looks as following: + +```json +{ + "@timestamp": "2023-09-13T08:40:39.068Z", + "agent": { + "ephemeral_id": "7aae6130-635a-422f-ac2e-e40324e86921", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "data_stream": { + "dataset": "prisma_cloud.audit", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "action": "login", + "agent_id_status": "verified", + "category": [ + "authentication" + ], + "dataset": "prisma_cloud.audit", + "ingested": "2023-10-18T12:09:30Z", + "kind": "event", + "original": "{\"action\":\"'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key.\",\"actionType\":\"LOGIN\",\"ipAddress\":\"81.2.69.192\",\"resourceName\":\"john.user@google.com\",\"resourceType\":\"Login\",\"result\":\"Successful\",\"timestamp\":1694594439068,\"user\":\"john.user@google.com\"}", + "outcome": "success", + "type": [ + "info" + ] + }, + "host": { + "ip": [ + "81.2.69.192" + ] + }, + "input": { + "type": "cel" + }, + "prisma_cloud": { + "audit": { + "action": { + "type": "LOGIN", + "value": "'john.user@google.com'(with role 'System Admin':'System Admin') logged in via access key." + }, + "ip_address": "81.2.69.192", + "resource": { + "name": "john.user@google.com", + "type": "Login" + }, + "result": "Successful", + "timestamp": "2023-09-13T08:40:39.068Z", + "user": "john.user@google.com" + } + }, + "related": { + "ip": [ + "81.2.69.192" + ], + "user": [ + "john.user@google.com" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-audit" + ], + "user": { + "email": "john.user@google.com" + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| event.dataset | Event dataset. | constant_keyword | +| event.module | Event module. | constant_keyword | +| input.type | Type of filebeat input. | keyword | +| log.offset | Log offset. | long | +| prisma_cloud.audit.action.type | Action Type. | keyword | +| prisma_cloud.audit.action.value | | keyword | +| prisma_cloud.audit.ip_address | IP Address. | ip | +| prisma_cloud.audit.resource.name | | keyword | +| prisma_cloud.audit.resource.type | | keyword | +| prisma_cloud.audit.result | | keyword | +| prisma_cloud.audit.timestamp | Timestamp. | date | +| prisma_cloud.audit.user | User. | keyword | +| tags | User defined tags. | keyword | + + +### Host + +This is the `Host` dataset. + +#### Example + +An example event for `host` looks as following: + +```json +{ + "@timestamp": "2023-10-18T12:12:26.324Z", + "agent": { + "ephemeral_id": "b495d34c-84f2-4dde-abdf-838c08e654af", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "cloud": { + "account": { + "id": "Non-onboarded cloud accounts" + }, + "instance": { + "id": "string", + "name": "string" + }, + "machine": { + "type": "string" + }, + "provider": [ + "aws" + ], + "region": "string" + }, + "data_stream": { + "dataset": "prisma_cloud.host", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "host" + ], + "dataset": "prisma_cloud.host", + "id": "DESKTOP-6PQXXMS", + "ingested": "2023-10-18T12:12:27Z", + "kind": "event", + "original": "{\"Secrets\":[],\"_id\":\"DESKTOP-6PQXXMS\",\"agentless\":false,\"allCompliance\":{\"compliance\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.949Z\",\"exploit\":[\"exploit-db\"],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}],\"enabled\":\"true\"},\"appEmbedded\":false,\"applications\":[{\"installedFromPackage\":true,\"knownVulnerabilities\":0,\"layerTime\":0,\"name\":\"string\",\"path\":\"string\",\"service\":true,\"version\":\"string\"}],\"binaries\":[{\"altered\":true,\"cveCount\":0,\"deps\":[\"string\"],\"fileMode\":0,\"functionLayer\":\"string\",\"md5\":\"string\",\"missingPkg\":true,\"name\":\"string\",\"path\":\"string\",\"pkgRootDir\":\"string\",\"services\":[\"string\"],\"version\":\"string\"}],\"cloudMetadata\":{\"accountID\":\"Non-onboarded cloud accounts\",\"awsExecutionEnv\":\"string\",\"image\":\"string\",\"labels\":[{\"key\":\"string\",\"sourceName\":\"string\",\"sourceType\":[\"namespace\"],\"timestamp\":\"2023-09-08T04:01:49.949Z\",\"value\":\"string\"}],\"name\":\"string\",\"provider\":[\"aws\"],\"region\":\"string\",\"resourceID\":\"string\",\"resourceURL\":\"string\",\"type\":\"string\",\"vmID\":\"string\",\"vmImageID\":\"string\"},\"clusters\":[\"string\"],\"collections\":[\"All\"],\"complianceDistribution\":{\"critical\":4,\"high\":0,\"low\":0,\"medium\":0,\"total\":4},\"complianceIssuesCount\":4,\"complianceRiskScore\":4000000,\"creationTime\":\"0001-01-01T00:00:00Z\",\"distro\":\"Microsoft Windows [Version 10.0.19045.2006]\",\"err\":\"\",\"externalLabels\":[{\"key\":\"string\",\"sourceName\":\"string\",\"sourceType\":[\"namespace\"],\"timestamp\":\"2023-09-08T04:01:49.949Z\",\"value\":\"string\"}],\"files\":[{\"md5\":\"string\",\"path\":\"string\",\"sha1\":\"string\",\"sha256\":\"string\"}],\"firewallProtection\":{\"enabled\":false,\"outOfBandMode\":\"Observation\",\"ports\":[0],\"supported\":false,\"tlsPorts\":[0],\"unprotectedProcesses\":[{\"port\":0,\"process\":\"string\",\"tls\":true}]},\"firstScanTime\":\"2023-08-11T06:53:57.456Z\",\"history\":[{\"baseLayer\":true,\"created\":0,\"emptyLayer\":true,\"id\":\"string\",\"instruction\":\"string\",\"sizeBytes\":0,\"tags\":[\"string\"],\"vulnerabilities\":[{\"applicableRules\":[\"string\"],\"binaryPkgs\":[\"string\"],\"block\":true,\"cause\":\"string\",\"cri\":true,\"custom\":true,\"cve\":\"string\",\"cvss\":0,\"description\":\"string\",\"discovered\":\"2023-09-08T04:01:49.950Z\",\"exploit\":[\"exploit-db\"],\"exploits\":[{\"kind\":[\"poc\",\"in-the-wild\"],\"link\":\"string\",\"source\":[\"\",\"exploit-db\"]}],\"fixDate\":0,\"fixLink\":\"string\",\"functionLayer\":\"string\",\"gracePeriodDays\":0,\"id\":0,\"layerTime\":0,\"link\":\"string\",\"packageName\":\"string\",\"packageVersion\":\"string\",\"published\":0,\"riskFactors\":{},\"severity\":\"string\",\"status\":\"string\",\"templates\":[[\"PCI\"]],\"text\":\"string\",\"title\":\"string\",\"twistlock\":true,\"type\":[\"container\"],\"vecStr\":\"string\",\"vulnTagInfos\":[{\"color\":\"string\",\"comment\":\"string\",\"name\":\"string\"}],\"wildfireMalware\":{\"md5\":\"string\",\"path\":\"string\",\"verdict\":\"string\"}}]}],\"hostDevices\":[{\"ip\":\"0.0.0.0\",\"name\":\"string\"}],\"hostname\":\"DESKTOP-6PQXXMS\",\"hosts\":{},\"id\":\"string\",\"image\":{\"created\":\"0001-01-01T00:00:00Z\",\"entrypoint\":[\"string\"],\"env\":[\"string\"],\"healthcheck\":true,\"id\":\"string\",\"labels\":{},\"layers\":[\"string\"],\"os\":\"string\",\"repoDigest\":[\"string\"],\"repoTags\":[\"string\"],\"user\":\"string\",\"workingDir\":\"string\"},\"instances\":[{\"host\":\"string\",\"image\":\"string\",\"modified\":\"2023-09-08T04:01:49.951Z\",\"registry\":\"string\",\"repo\":\"string\",\"tag\":\"string\"}],\"isARM64\":false,\"malwareAnalyzedTime\":\"0001-01-01T00:00:00Z\",\"osDistro\":\"windows\",\"osDistroRelease\":\"Windows\",\"osDistroVersion\":\"string\",\"packageCorrelationDone\":true,\"packageManager\":true,\"packages\":[{\"pkgs\":[{\"binaryIdx\":[0],\"binaryPkgs\":[\"string\"],\"cveCount\":0,\"defaultGem\":true,\"files\":[{\"md5\":\"string\",\"path\":\"string\",\"sha1\":\"string\",\"sha256\":\"string\"}],\"functionLayer\":\"string\",\"goPkg\":true,\"jarIdentifier\":\"string\",\"layerTime\":0,\"license\":\"string\",\"name\":\"string\",\"osPackage\":true,\"path\":\"string\",\"version\":\"string\"}],\"pkgsType\":\"nodejs\"}],\"pushTime\":\"0001-01-01T00:00:00Z\",\"redHatNonRPMImage\":false,\"repoDigests\":[],\"repoTag\":null,\"riskFactors\":{},\"scanID\":0,\"scanTime\":\"2023-08-23T11:48:41.803Z\",\"tags\":[{\"digest\":\"string\",\"id\":\"string\",\"registry\":\"string\",\"repo\":\"string\",\"tag\":\"string\"}],\"trustResult\":{\"hostsStatuses\":[{\"host\":\"string\",\"status\":\"trusted\"}]},\"trustStatus\":\"\",\"type\":\"host\",\"vulnerabilitiesCount\":0,\"vulnerabilityDistribution\":{\"critical\":0,\"high\":0,\"low\":0,\"medium\":0,\"total\":0},\"vulnerabilityRiskScore\":0,\"wildFireUsage\":null}", + "start": "0001-01-01T00:00:00.000Z", + "type": [ + "info" + ] + }, + "file": { + "hash": { + "md5": [ + "string" + ], + "sha1": [ + "string" + ], + "sha256": [ + "string" + ] + }, + "path": [ + "string" + ] + }, + "host": { + "hostname": "DESKTOP-6PQXXMS", + "ip": [ + "0.0.0.0" + ], + "type": "host" + }, + "input": { + "type": "cel" + }, + "os": { + "family": "windows", + "name": "Windows", + "version": "string" + }, + "package": { + "license": [ + "string" + ], + "name": [ + "string" + ], + "path": [ + "string" + ], + "type": [ + "nodejs" + ], + "version": [ + "string" + ] + }, + "prisma_cloud": { + "host": { + "_id": "DESKTOP-6PQXXMS", + "agentless": false, + "all_compliance": { + "data": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0, + "description": "string", + "discovered": "2023-09-08T04:01:49.949Z", + "exploit": [ + "exploit-db" + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ], + "enabled": true + }, + "app_embedded": false, + "applications": [ + { + "installed_from_package": true, + "known_vulnerabilities": 0, + "layer_time": "1970-01-01T00:00:00.000Z", + "name": "string", + "path": "string", + "service": true, + "version": "string" + } + ], + "binaries": [ + { + "altered": true, + "cve_count": 0, + "deps": [ + "string" + ], + "file_mode": 0, + "function_layer": "string", + "md5": "string", + "missing_pkg": true, + "name": "string", + "path": "string", + "pkg_root_dir": "string", + "services": [ + "string" + ], + "version": "string" + } + ], + "cloud_metadata": { + "account_id": "Non-onboarded cloud accounts", + "aws_execution_env": "string", + "image": "string", + "labels": [ + { + "key": "string", + "source": { + "name": "string", + "type": [ + "namespace" + ] + }, + "timestamp": "2023-09-08T04:01:49.949Z", + "value": "string" + } + ], + "name": "string", + "provider": [ + "aws" + ], + "region": "string", + "resource": { + "id": "string", + "url": "string" + }, + "type": "string", + "vm": { + "id": "string", + "image_id": "string" + } + }, + "clusters": [ + "string" + ], + "collections": [ + "All" + ], + "compliance_distribution": { + "critical": 4, + "high": 0, + "low": 0, + "medium": 0, + "total": 4 + }, + "compliance_issues": { + "count": 4 + }, + "compliance_risk_score": 4000000, + "creation_time": "0001-01-01T00:00:00.000Z", + "devices": [ + { + "ip": "0.0.0.0", + "name": "string" + } + ], + "distro": "Microsoft Windows [Version 10.0.19045.2006]", + "external_labels": [ + { + "key": "string", + "source": { + "name": "string", + "type": [ + "namespace" + ] + }, + "timestamp": "2023-09-08T04:01:49.949Z", + "value": "string" + } + ], + "files": [ + { + "md5": "string", + "path": "string", + "sha1": "string", + "sha256": "string" + } + ], + "firewall_protection": { + "enabled": false, + "out_of_band_mode": "Observation", + "ports": [ + 0 + ], + "supported": false, + "tls_ports": [ + 0 + ], + "unprotected_processes": [ + { + "port": 0, + "process": "string", + "tls": true + } + ] + }, + "first_scan_time": "2023-08-11T06:53:57.456Z", + "history": [ + { + "base_layer": true, + "created": "1970-01-01T00:00:00.000Z", + "empty_layer": true, + "id": "string", + "instruction": "string", + "size_bytes": 0, + "tags": [ + "string" + ], + "vulnerabilities": [ + { + "applicable_rules": [ + "string" + ], + "binary_pkgs": [ + "string" + ], + "block": true, + "cause": "string", + "cri": true, + "custom": true, + "cve": "string", + "cvss": 0, + "description": "string", + "discovered": "2023-09-08T04:01:49.950Z", + "exploit": [ + "exploit-db" + ], + "exploits": [ + { + "kind": [ + "poc", + "in-the-wild" + ], + "link": "string", + "source": [ + "exploit-db" + ] + } + ], + "fix_date": "1970-01-01T00:00:00.000Z", + "fix_link": "string", + "function_layer": "string", + "grace_period_days": 0, + "id": "0", + "layer_time": "1970-01-01T00:00:00.000Z", + "link": "string", + "package": { + "name": "string", + "version": "string" + }, + "published": "1970-01-01T00:00:00.000Z", + "severity": "string", + "status": "string", + "templates": [ + [ + "PCI" + ] + ], + "text": "string", + "title": "string", + "twistlock": true, + "type": [ + "container" + ], + "vec_str": "string", + "vuln_tag_infos": [ + { + "color": "string", + "comment": "string", + "name": "string" + } + ], + "wild_fire_malware": { + "md5": "string", + "path": "string", + "verdict": "string" + } + } + ] + } + ], + "hostname": "DESKTOP-6PQXXMS", + "id": "string", + "image": { + "created": "0001-01-01T00:00:00.000Z", + "entrypoint": [ + "string" + ], + "env": [ + "string" + ], + "healthcheck": true, + "id": "string", + "layers": [ + "string" + ], + "os": "string", + "repo": { + "digest": [ + "string" + ], + "tags": [ + "string" + ] + }, + "user": "string", + "working_dir": "string" + }, + "instances": [ + { + "host": "string", + "image": "string", + "modified": "2023-09-08T04:01:49.951Z", + "registry": "string", + "repo": "string", + "tag": "string" + } + ], + "is_arm64": false, + "malware_analyzed_time": "0001-01-01T00:00:00.000Z", + "os_distro": { + "release": "Windows", + "value": "windows", + "version": "string" + }, + "package": { + "correlation_done": true, + "manager": true + }, + "packages": [ + { + "pkgs": [ + { + "binary_idx": [ + 0 + ], + "binary_pkgs": [ + "string" + ], + "cve_count": 0, + "default_gem": true, + "files": [ + { + "md5": "string", + "path": "string", + "sha1": "string", + "sha256": "string" + } + ], + "function_layer": "string", + "go_pkg": true, + "jar_identifier": "string", + "layer_time": "1970-01-01T00:00:00.000Z", + "license": "string", + "name": "string", + "os_package": true, + "path": "string", + "version": "string" + } + ], + "pkgs_type": "nodejs" + } + ], + "push_time": "0001-01-01T00:00:00.000Z", + "red_hat_non_rpm_image": false, + "scan": { + "time": "2023-08-23T11:48:41.803Z" + }, + "tags": [ + { + "digest": "string", + "id": "string", + "registry": "string", + "repo": "string", + "tag": "string" + } + ], + "trust_result": { + "hosts_statuses": [ + { + "host": "string", + "status": "trusted" + } + ] + }, + "type": "host", + "vulnerabilities": { + "count": 0 + }, + "vulnerability": { + "distribution": { + "critical": 0, + "high": 0, + "low": 0, + "medium": 0, + "total": 0 + }, + "risk_score": 0 + } + } + }, + "related": { + "hash": [ + "string" + ], + "hosts": [ + "string", + "DESKTOP-6PQXXMS" + ], + "ip": [ + "0.0.0.0" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-host" + ], + "vulnerability": { + "description": [ + "string" + ], + "id": [ + "string" + ], + "severity": [ + "string" + ] + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| event.dataset | Event dataset. | constant_keyword | +| event.module | Event module. | constant_keyword | +| input.type | Type of filebeat input. | keyword | +| log.offset | Log offset. | long | +| log.source.address | Source address from which the log event was read / sent from. | keyword | +| prisma_cloud.host._id | Image identifier (image ID or repo:tag). | keyword | +| prisma_cloud.host.agentless | Agentless indicates that the host was scanned with the agentless scanner. | boolean | +| prisma_cloud.host.all_compliance.data.applicable_rules | Rules applied on the package. | keyword | +| prisma_cloud.host.all_compliance.data.binary_pkgs | Names of the distro binary package names (packages which are built from the source of the package). | keyword | +| prisma_cloud.host.all_compliance.data.block | Indicates if the vulnerability has a block effect (true) or not (false). | boolean | +| prisma_cloud.host.all_compliance.data.cause | Additional information regarding the root cause for the vulnerability. | keyword | +| prisma_cloud.host.all_compliance.data.cri | Indicates if this is a CRI-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.all_compliance.data.custom | Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). | boolean | +| prisma_cloud.host.all_compliance.data.cve | CVE ID of the vulnerability (if applied). | keyword | +| prisma_cloud.host.all_compliance.data.cvss | CVSS score of the vulnerability. | float | +| prisma_cloud.host.all_compliance.data.description | Description of the vulnerability. | keyword | +| prisma_cloud.host.all_compliance.data.discovered | Specifies the time of discovery for the vulnerability. | date | +| prisma_cloud.host.all_compliance.data.exploit | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.all_compliance.data.exploits.kind | ExploitKind represents the kind of the exploit. | keyword | +| prisma_cloud.host.all_compliance.data.exploits.link | Link is a link to information about the exploit. | keyword | +| prisma_cloud.host.all_compliance.data.exploits.source | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.all_compliance.data.fix_date | Date/time when the vulnerability was fixed (in Unix time). | date | +| prisma_cloud.host.all_compliance.data.fix_link | Link to the vendor's fixed-version information. | keyword | +| prisma_cloud.host.all_compliance.data.function_layer | Specifies the serverless layer ID in which the vulnerability was discovered. | keyword | +| prisma_cloud.host.all_compliance.data.grace_period_days | Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. | long | +| prisma_cloud.host.all_compliance.data.id | ID of the violation. | keyword | +| prisma_cloud.host.all_compliance.data.layer_time | Date/time of the image layer to which the CVE belongs. | date | +| prisma_cloud.host.all_compliance.data.link | Vendor link to the CVE. | keyword | +| prisma_cloud.host.all_compliance.data.package.name | Name of the package that caused the vulnerability. | keyword | +| prisma_cloud.host.all_compliance.data.package.version | Version of the package that caused the vulnerability (or null). | keyword | +| prisma_cloud.host.all_compliance.data.published | Date/time when the vulnerability was published (in Unix time). | date | +| prisma_cloud.host.all_compliance.data.risk_factors | RiskFactors maps the existence of vulnerability risk factors. | flattened | +| prisma_cloud.host.all_compliance.data.severity | Textual representation of the vulnerability's severity. | keyword | +| prisma_cloud.host.all_compliance.data.status | Vendor status for the vulnerability. | keyword | +| prisma_cloud.host.all_compliance.data.templates | List of templates with which the vulnerability is associated. | keyword | +| prisma_cloud.host.all_compliance.data.text | Description of the violation. | keyword | +| prisma_cloud.host.all_compliance.data.title | Compliance title. | keyword | +| prisma_cloud.host.all_compliance.data.twistlock | Indicates if this is a Twistlock-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.all_compliance.data.type | Type represents the vulnerability type. | keyword | +| prisma_cloud.host.all_compliance.data.vec_str | Textual representation of the metric values used to score the vulnerability. | keyword | +| prisma_cloud.host.all_compliance.data.vuln_tag_infos.color | Color is a hexadecimal representation of color code value. | keyword | +| prisma_cloud.host.all_compliance.data.vuln_tag_infos.comment | Tag comment in a specific vulnerability context. | keyword | +| prisma_cloud.host.all_compliance.data.vuln_tag_infos.name | Name of the tag. | keyword | +| prisma_cloud.host.all_compliance.data.wild_fire_malware.md5 | MD5 is the hash of the malicious binary. | keyword | +| prisma_cloud.host.all_compliance.data.wild_fire_malware.path | Path is the path to malicious binary. | keyword | +| prisma_cloud.host.all_compliance.data.wild_fire_malware.verdict | Verdict is the malicious source like grayware, malware and phishing. | keyword | +| prisma_cloud.host.all_compliance.enabled | Enabled indicates whether passed compliance checks is enabled by policy. | boolean | +| prisma_cloud.host.app_embedded | Indicates that this image was scanned by an App-Embedded Defender. | boolean | +| prisma_cloud.host.applications.installed_from_package | Indicates that the app was installed as an OS package. | boolean | +| prisma_cloud.host.applications.known_vulnerabilities | Total number of vulnerabilities for this application. | long | +| prisma_cloud.host.applications.layer_time | Image layer to which the application belongs - layer creation time. | date | +| prisma_cloud.host.applications.name | Name of the application. | keyword | +| prisma_cloud.host.applications.path | Path of the detected application. | keyword | +| prisma_cloud.host.applications.service | Service indicates whether the application is installed as a service. | boolean | +| prisma_cloud.host.applications.version | Version of the application. | keyword | +| prisma_cloud.host.base_image | Image’s base image name. Used when filtering the vulnerabilities by base images. | keyword | +| prisma_cloud.host.binaries.altered | Indicates if the binary was installed from a package manager and modified/replaced (true) or not (false). | boolean | +| prisma_cloud.host.binaries.cve_count | Total number of CVEs for this specific binary. | long | +| prisma_cloud.host.binaries.deps | Third-party package files which are used by the binary. | keyword | +| prisma_cloud.host.binaries.file_mode | Represents the file's mode and permission bits. | long | +| prisma_cloud.host.binaries.function_layer | ID of the serverless layer in which the package was discovered. | keyword | +| prisma_cloud.host.binaries.md5 | Md5 hashset of the binary. | keyword | +| prisma_cloud.host.binaries.missing_pkg | Indicates if this binary is not related to any package (true) or not (false). | boolean | +| prisma_cloud.host.binaries.name | Name of the binary. | keyword | +| prisma_cloud.host.binaries.path | Path is the path of the binary. | keyword | +| prisma_cloud.host.binaries.pkg_root_dir | Path for searching packages used by the binary. | keyword | +| prisma_cloud.host.binaries.services | Names of services which use the binary. | keyword | +| prisma_cloud.host.binaries.version | Version of the binary. | keyword | +| prisma_cloud.host.cloud_metadata.account_id | Cloud account ID. | keyword | +| prisma_cloud.host.cloud_metadata.aws_execution_env | AWS execution environment (e.g. EC2/Fargate). | keyword | +| prisma_cloud.host.cloud_metadata.image | Image name. | keyword | +| prisma_cloud.host.cloud_metadata.labels.key | Label key. | keyword | +| prisma_cloud.host.cloud_metadata.labels.source.name | Source name (e.g., for a namespace, the source name can be 'twistlock'). | keyword | +| prisma_cloud.host.cloud_metadata.labels.source.type | ExternalLabelSourceType indicates the source of the labels. | keyword | +| prisma_cloud.host.cloud_metadata.labels.timestamp | Time when the label was fetched. | date | +| prisma_cloud.host.cloud_metadata.labels.value | Value of the label. | keyword | +| prisma_cloud.host.cloud_metadata.name | Instance name. | keyword | +| prisma_cloud.host.cloud_metadata.provider | CloudProvider specifies the cloud provider name. | keyword | +| prisma_cloud.host.cloud_metadata.region | Instance region. | keyword | +| prisma_cloud.host.cloud_metadata.resource.id | Unique ID of the resource. | keyword | +| prisma_cloud.host.cloud_metadata.resource.url | Server-defined URL for the resource. | keyword | +| prisma_cloud.host.cloud_metadata.type | Instance type. | keyword | +| prisma_cloud.host.cloud_metadata.vm.id | Azure unique vm ID. | keyword | +| prisma_cloud.host.cloud_metadata.vm.image_id | VMImageID holds the VM image ID. | keyword | +| prisma_cloud.host.cluster_type | ClusterType is the cluster type. | keyword | +| prisma_cloud.host.clusters | Cluster names. | keyword | +| prisma_cloud.host.collections | Collections to which this result applies. | keyword | +| prisma_cloud.host.compliance_distribution.critical | | long | +| prisma_cloud.host.compliance_distribution.high | | long | +| prisma_cloud.host.compliance_distribution.low | | long | +| prisma_cloud.host.compliance_distribution.medium | | long | +| prisma_cloud.host.compliance_distribution.total | | long | +| prisma_cloud.host.compliance_issues.count | Number of compliance issues. | long | +| prisma_cloud.host.compliance_issues.data.applicable_rules | Rules applied on the package. | keyword | +| prisma_cloud.host.compliance_issues.data.binary_pkgs | Names of the distro binary package names (packages which are built from the source of the package). | keyword | +| prisma_cloud.host.compliance_issues.data.block | Indicates if the vulnerability has a block effect (true) or not (false). | boolean | +| prisma_cloud.host.compliance_issues.data.cause | Additional information regarding the root cause for the vulnerability. | keyword | +| prisma_cloud.host.compliance_issues.data.cri | Indicates if this is a CRI-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.compliance_issues.data.custom | Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). | boolean | +| prisma_cloud.host.compliance_issues.data.cve | CVE ID of the vulnerability (if applied). | keyword | +| prisma_cloud.host.compliance_issues.data.cvss | CVSS score of the vulnerability. | float | +| prisma_cloud.host.compliance_issues.data.description | Description of the vulnerability. | keyword | +| prisma_cloud.host.compliance_issues.data.discovered | Specifies the time of discovery for the vulnerability. | date | +| prisma_cloud.host.compliance_issues.data.exploit | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.compliance_issues.data.exploits.kind | ExploitKind represents the kind of the exploit. | keyword | +| prisma_cloud.host.compliance_issues.data.exploits.link | Link is a link to information about the exploit. | keyword | +| prisma_cloud.host.compliance_issues.data.exploits.source | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.compliance_issues.data.fix_date | Date/time when the vulnerability was fixed (in Unix time). | date | +| prisma_cloud.host.compliance_issues.data.fix_link | Link to the vendor's fixed-version information. | keyword | +| prisma_cloud.host.compliance_issues.data.function_layer | Specifies the serverless layer ID in which the vulnerability was discovered. | keyword | +| prisma_cloud.host.compliance_issues.data.grace_period_days | Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. | long | +| prisma_cloud.host.compliance_issues.data.id | ID of the violation. | keyword | +| prisma_cloud.host.compliance_issues.data.layer_time | Date/time of the image layer to which the CVE belongs. | date | +| prisma_cloud.host.compliance_issues.data.link | Vendor link to the CVE. | keyword | +| prisma_cloud.host.compliance_issues.data.package.name | Name of the package that caused the vulnerability. | keyword | +| prisma_cloud.host.compliance_issues.data.package.version | Version of the package that caused the vulnerability (or null). | keyword | +| prisma_cloud.host.compliance_issues.data.published | Date/time when the vulnerability was published (in Unix time). | date | +| prisma_cloud.host.compliance_issues.data.risk_factors | RiskFactors maps the existence of vulnerability risk factors. | flattened | +| prisma_cloud.host.compliance_issues.data.severity | Textual representation of the vulnerability's severity. | keyword | +| prisma_cloud.host.compliance_issues.data.status | Vendor status for the vulnerability. | keyword | +| prisma_cloud.host.compliance_issues.data.text | Description of the violation. | keyword | +| prisma_cloud.host.compliance_issues.data.title | Compliance title. | keyword | +| prisma_cloud.host.compliance_issues.data.twistlock | Indicates if this is a Twistlock-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.compliance_issues.data.type | Type represents the vulnerability type. | keyword | +| prisma_cloud.host.compliance_issues.data.vec_str | Textual representation of the metric values used to score the vulnerability. | keyword | +| prisma_cloud.host.compliance_issues.data.vuln_tag_infos.color | Color is a hexadecimal representation of color code value. | keyword | +| prisma_cloud.host.compliance_issues.data.vuln_tag_infos.comment | Tag comment in a specific vulnerability context. | keyword | +| prisma_cloud.host.compliance_issues.data.vuln_tag_infos.name | Name of the tag. | keyword | +| prisma_cloud.host.compliance_issues.data.wildfire_malware.md5 | MD5 is the hash of the malicious binary. | keyword | +| prisma_cloud.host.compliance_issues.data.wildfire_malware.path | Path is the path to malicious binary. | keyword | +| prisma_cloud.host.compliance_issues.data.wildfire_malware.verdict | Verdict is the malicious source like grayware, malware and phishing. | keyword | +| prisma_cloud.host.compliance_risk_score | Compliance risk score for the image. | float | +| prisma_cloud.host.creation_time | Specifies the time of creation for the latest version of the image. | date | +| prisma_cloud.host.devices.ip | Network device IPv4 address. | ip | +| prisma_cloud.host.devices.name | Network device name. | keyword | +| prisma_cloud.host.distro | Full name of the distribution. | keyword | +| prisma_cloud.host.ecs_cluster_name | ECS cluster name. | keyword | +| prisma_cloud.host.err | Description of an error that occurred during image health scan. | keyword | +| prisma_cloud.host.external_labels.key | Label key. | keyword | +| prisma_cloud.host.external_labels.source.name | Source name (e.g., for a namespace, the source name can be 'twistlock'). | keyword | +| prisma_cloud.host.external_labels.source.type | ExternalLabelSourceType indicates the source of the labels. | keyword | +| prisma_cloud.host.external_labels.timestamp | Time when the label was fetched. | keyword | +| prisma_cloud.host.external_labels.value | Value of the label. | keyword | +| prisma_cloud.host.files.md5 | Hash sum of the file using md5. | keyword | +| prisma_cloud.host.files.path | Path of the file. | keyword | +| prisma_cloud.host.files.sha1 | Hash sum of the file using SHA-1. | keyword | +| prisma_cloud.host.files.sha256 | Hash sum of the file using SHA256. | keyword | +| prisma_cloud.host.firewall_protection.enabled | Enabled indicates if WAAS proxy protection is enabled (true) or not (false). | boolean | +| prisma_cloud.host.firewall_protection.out_of_band_mode | OutOfBandMode holds the app firewall out-of-band mode. | keyword | +| prisma_cloud.host.firewall_protection.ports | Ports indicates http open ports associated with the container. | long | +| prisma_cloud.host.firewall_protection.supported | Supported indicates if WAAS protection is supported (true) or not (false). | boolean | +| prisma_cloud.host.firewall_protection.tls_ports | TLSPorts indicates https open ports associated with the container. | long | +| prisma_cloud.host.firewall_protection.unprotected_processes.port | Port is the process port. | long | +| prisma_cloud.host.firewall_protection.unprotected_processes.process | Process is the process name. | keyword | +| prisma_cloud.host.firewall_protection.unprotected_processes.tls | TLS is the port TLS indication. | boolean | +| prisma_cloud.host.first_scan_time | Specifies the time of the scan for the first version of the image. This time is preserved even after the version update. | date | +| prisma_cloud.host.history.base_layer | Indicates if this layer originated from the base image (true) or not (false). | boolean | +| prisma_cloud.host.history.created | Date/time when the image layer was created. | date | +| prisma_cloud.host.history.empty_layer | Indicates if this instruction didn't create a separate layer (true) or not. | boolean | +| prisma_cloud.host.history.id | ID of the layer. | keyword | +| prisma_cloud.host.history.instruction | Docker file instruction and arguments used to create this layer. | keyword | +| prisma_cloud.host.history.size_bytes | Size of the layer (in bytes). | long | +| prisma_cloud.host.history.tags | Holds the image tags. | keyword | +| prisma_cloud.host.history.vulnerabilities.applicable_rules | Rules applied on the package. | keyword | +| prisma_cloud.host.history.vulnerabilities.binary_pkgs | Names of the distro binary package names (packages which are built from the source of the package). | keyword | +| prisma_cloud.host.history.vulnerabilities.block | Indicates if the vulnerability has a block effect (true) or not (false). | boolean | +| prisma_cloud.host.history.vulnerabilities.cause | Additional information regarding the root cause for the vulnerability. | keyword | +| prisma_cloud.host.history.vulnerabilities.cri | Indicates if this is a CRI-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.history.vulnerabilities.custom | Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). | boolean | +| prisma_cloud.host.history.vulnerabilities.cve | CVE ID of the vulnerability (if applied). | keyword | +| prisma_cloud.host.history.vulnerabilities.cvss | CVSS score of the vulnerability. | float | +| prisma_cloud.host.history.vulnerabilities.description | Description of the vulnerability. | keyword | +| prisma_cloud.host.history.vulnerabilities.discovered | Specifies the time of discovery for the vulnerability. | date | +| prisma_cloud.host.history.vulnerabilities.exploit | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.history.vulnerabilities.exploits.kind | ExploitKind represents the kind of the exploit. | keyword | +| prisma_cloud.host.history.vulnerabilities.exploits.link | Link is a link to information about the exploit. | keyword | +| prisma_cloud.host.history.vulnerabilities.exploits.source | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.history.vulnerabilities.fix_date | Date/time when the vulnerability was fixed (in Unix time). | date | +| prisma_cloud.host.history.vulnerabilities.fix_link | Link to the vendor's fixed-version information. | keyword | +| prisma_cloud.host.history.vulnerabilities.function_layer | Specifies the serverless layer ID in which the vulnerability was discovered. | keyword | +| prisma_cloud.host.history.vulnerabilities.grace_period_days | Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. | long | +| prisma_cloud.host.history.vulnerabilities.id | ID of the violation. | keyword | +| prisma_cloud.host.history.vulnerabilities.layer_time | Date/time of the image layer to which the CVE belongs. | date | +| prisma_cloud.host.history.vulnerabilities.link | Vendor link to the CVE. | keyword | +| prisma_cloud.host.history.vulnerabilities.package.name | Name of the package that caused the vulnerability. | keyword | +| prisma_cloud.host.history.vulnerabilities.package.version | Version of the package that caused the vulnerability (or null). | keyword | +| prisma_cloud.host.history.vulnerabilities.published | Date/time when the vulnerability was published (in Unix time). | date | +| prisma_cloud.host.history.vulnerabilities.risk_factors | RiskFactors maps the existence of vulnerability risk factors. | flattened | +| prisma_cloud.host.history.vulnerabilities.severity | Textual representation of the vulnerability's severity. | keyword | +| prisma_cloud.host.history.vulnerabilities.status | Vendor status for the vulnerability. | keyword | +| prisma_cloud.host.history.vulnerabilities.templates | List of templates with which the vulnerability is associated. | keyword | +| prisma_cloud.host.history.vulnerabilities.text | Description of the violation. | keyword | +| prisma_cloud.host.history.vulnerabilities.title | Compliance title. | keyword | +| prisma_cloud.host.history.vulnerabilities.twistlock | Indicates if this is a Twistlock-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.history.vulnerabilities.type | Type represents the vulnerability type. | keyword | +| prisma_cloud.host.history.vulnerabilities.vec_str | Textual representation of the metric values used to score the vulnerability. | keyword | +| prisma_cloud.host.history.vulnerabilities.vuln_tag_infos.color | Color is a hexadecimal representation of color code value. | keyword | +| prisma_cloud.host.history.vulnerabilities.vuln_tag_infos.comment | Tag comment in a specific vulnerability context. | keyword | +| prisma_cloud.host.history.vulnerabilities.vuln_tag_infos.name | Name of the tag. | keyword | +| prisma_cloud.host.history.vulnerabilities.wild_fire_malware.md5 | MD5 is the hash of the malicious binary. | keyword | +| prisma_cloud.host.history.vulnerabilities.wild_fire_malware.path | Path is the path to malicious binary. | keyword | +| prisma_cloud.host.history.vulnerabilities.wild_fire_malware.verdict | Verdict is the malicious source like grayware, malware and phishing. | keyword | +| prisma_cloud.host.hostname | Name of the host that was scanned. | keyword | +| prisma_cloud.host.hosts | ImageHosts is a fast index for image scan results metadata per host. | flattened | +| prisma_cloud.host.id | Image ID. | keyword | +| prisma_cloud.host.image.created | Date/time when the image was created. | date | +| prisma_cloud.host.image.entrypoint | Combined entrypoint of the image (entrypoint + CMD). | keyword | +| prisma_cloud.host.image.env | Image environment variables. | keyword | +| prisma_cloud.host.image.healthcheck | Indicates if health checks are enabled (true) or not (false). | boolean | +| prisma_cloud.host.image.history.base_layer | Indicates if this layer originated from the base image (true) or not (false). | boolean | +| prisma_cloud.host.image.history.created | Date/time when the image layer was created. | date | +| prisma_cloud.host.image.history.empty_layer | Indicates if this instruction didn't create a separate layer (true) or not. | boolean | +| prisma_cloud.host.image.history.id | ID of the layer. | keyword | +| prisma_cloud.host.image.history.instruction | Docker file instruction and arguments used to create this layer. | keyword | +| prisma_cloud.host.image.history.size_bytes | Size of the layer (in bytes). | long | +| prisma_cloud.host.image.history.tags | Holds the image tags. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.applicable_rules | Rules applied on the package. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.binaryPkgs | Names of the distro binary package names (packages which are built from the source of the package). | keyword | +| prisma_cloud.host.image.history.vulnerabilities.block | Indicates if the vulnerability has a block effect (true) or not (false). | boolean | +| prisma_cloud.host.image.history.vulnerabilities.cause | Additional information regarding the root cause for the vulnerability. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.cri | Indicates if this is a CRI-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.image.history.vulnerabilities.custom | Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). | boolean | +| prisma_cloud.host.image.history.vulnerabilities.cve | CVE ID of the vulnerability (if applied). | keyword | +| prisma_cloud.host.image.history.vulnerabilities.cvss | CVSS score of the vulnerability. | float | +| prisma_cloud.host.image.history.vulnerabilities.description | Description of the vulnerability. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.discovered | Specifies the time of discovery for the vulnerability. | date | +| prisma_cloud.host.image.history.vulnerabilities.exploit | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.exploits.kind | ExploitKind represents the kind of the exploit. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.exploits.link | Link is a link to information about the exploit. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.exploits.source | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.fix_date | Date/time when the vulnerability was fixed (in Unix time). | date | +| prisma_cloud.host.image.history.vulnerabilities.fix_link | Link to the vendor's fixed-version information. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.function_layer | Specifies the serverless layer ID in which the vulnerability was discovered. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.grace_period_days | Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. | long | +| prisma_cloud.host.image.history.vulnerabilities.id | ID of the violation. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.layer_time | Date/time of the image layer to which the CVE belongs. | date | +| prisma_cloud.host.image.history.vulnerabilities.link | Vendor link to the CVE. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.package.name | Name of the package that caused the vulnerability. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.package.version | Version of the package that caused the vulnerability (or null). | keyword | +| prisma_cloud.host.image.history.vulnerabilities.published | Date/time when the vulnerability was published (in Unix time). | date | +| prisma_cloud.host.image.history.vulnerabilities.risk_factors | RiskFactors maps the existence of vulnerability risk factors. | flattened | +| prisma_cloud.host.image.history.vulnerabilities.severity | Textual representation of the vulnerability's severity. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.status | Vendor status for the vulnerability. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.templates | List of templates with which the vulnerability is associated. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.text | Description of the violation. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.title | Compliance title. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.twistlock | Indicates if this is a Twistlock-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.image.history.vulnerabilities.type | Type represents the vulnerability type. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.vec_str | Textual representation of the metric values used to score the vulnerability. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.vuln_tag_infos.color | Color is a hexadecimal representation of color code value. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.vuln_tag_infos.comment | Tag comment in a specific vulnerability context. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.vuln_tag_infos.name | Name of the tag. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.wild_fire_malware.md5 | MD5 is the hash of the malicious binary. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.wild_fire_malware.path | Path is the path to malicious binary. | keyword | +| prisma_cloud.host.image.history.vulnerabilities.wild_fire_malware.verdict | Verdict is the malicious source like grayware, malware and phishing. | keyword | +| prisma_cloud.host.image.id | ID of the image. | keyword | +| prisma_cloud.host.image.labels | Image labels. | flattened | +| prisma_cloud.host.image.layers | Image filesystem layers. | keyword | +| prisma_cloud.host.image.os | Image os type. | keyword | +| prisma_cloud.host.image.repo.digest | Image repo digests. | keyword | +| prisma_cloud.host.image.repo.tags | Image repo tags. | keyword | +| prisma_cloud.host.image.user | Image user. | keyword | +| prisma_cloud.host.image.working_dir | Base working directory of the image. | keyword | +| prisma_cloud.host.installed_products.agentless | Agentless indicates whether the scan was performed with agentless approach. | boolean | +| prisma_cloud.host.installed_products.apache | Apache indicates the apache server version, empty in case apache not running. | keyword | +| prisma_cloud.host.installed_products.aws_cloud | AWSCloud indicates whether AWS cloud is used. | boolean | +| prisma_cloud.host.installed_products.cluster_type | ClusterType is the cluster type. | keyword | +| prisma_cloud.host.installed_products.crio | CRI indicates whether the container runtime is CRI (and not docker). | boolean | +| prisma_cloud.host.installed_products.docker | Docker represents the docker daemon version. | keyword | +| prisma_cloud.host.installed_products.docker_enterprise | DockerEnterprise indicates whether the enterprise version of Docker is installed. | boolean | +| prisma_cloud.host.installed_products.has_package_manager | HasPackageManager indicates whether package manager is installed on the OS. | boolean | +| prisma_cloud.host.installed_products.k8s_api_server | K8sAPIServer indicates whether a kubernetes API server is running. | boolean | +| prisma_cloud.host.installed_products.k8s_controller_manager | K8sControllerManager indicates whether a kubernetes controller manager is running. | boolean | +| prisma_cloud.host.installed_products.k8s_etcd | K8sEtcd indicates whether etcd is running. | boolean | +| prisma_cloud.host.installed_products.k8s_federation_api_server | K8sFederationAPIServer indicates whether a federation API server is running. | boolean | +| prisma_cloud.host.installed_products.k8s_federation_controller_manager | K8sFederationControllerManager indicates whether a federation controller manager is running. | boolean | +| prisma_cloud.host.installed_products.k8s_kubelet | K8sKubelet indicates whether kubelet is running. | boolean | +| prisma_cloud.host.installed_products.k8s_proxy | K8sProxy indicates whether a kubernetes proxy is running. | boolean | +| prisma_cloud.host.installed_products.k8s_scheduler | K8sScheduler indicates whether the kubernetes scheduler is running. | boolean | +| prisma_cloud.host.installed_products.kubernetes | Kubernetes represents the kubernetes version. | keyword | +| prisma_cloud.host.installed_products.managed_cluster_version | ManagedClusterVersion is the version of the managed Kubernetes service, e.g. AKS/EKS/GKE/etc. | keyword | +| prisma_cloud.host.installed_products.openshift | Openshift indicates whether openshift is deployed. | boolean | +| prisma_cloud.host.installed_products.openshift_version | OpenshiftVersion represents the running openshift version. | keyword | +| prisma_cloud.host.installed_products.os_distro | OSDistro specifies the os distribution. | keyword | +| prisma_cloud.host.installed_products.serverless | Serverless indicates whether evaluated on a serverless environment. | boolean | +| prisma_cloud.host.installed_products.swarm.manager | SwarmManager indicates whether a swarm manager is running. | boolean | +| prisma_cloud.host.installed_products.swarm.node | SwarmNode indicates whether the node is part of an active swarm. | boolean | +| prisma_cloud.host.instances.host | | keyword | +| prisma_cloud.host.instances.image | | keyword | +| prisma_cloud.host.instances.modified | | date | +| prisma_cloud.host.instances.registry | | keyword | +| prisma_cloud.host.instances.repo | | keyword | +| prisma_cloud.host.instances.tag | | keyword | +| prisma_cloud.host.is_arm64 | IsARM64 indicates if the architecture of the image is aarch64. | boolean | +| prisma_cloud.host.k8s_cluster_addr | Endpoint of the Kubernetes API server. | keyword | +| prisma_cloud.host.labels | Image labels. | keyword | +| prisma_cloud.host.malware_analyzed_time | MalwareAnalyzedTime is the WildFire evaluator analyzing time shown as progress in UI and cannot to be overwritten by a new scan result. | date | +| prisma_cloud.host.missing_distro_vuln_coverage | Indicates if the image OS is covered in the IS (true) or not (false). | boolean | +| prisma_cloud.host.namespaces | k8s namespaces of all the containers running this image. | keyword | +| prisma_cloud.host.os_distro.release | OS distribution release. | keyword | +| prisma_cloud.host.os_distro.value | Name of the OS distribution. | keyword | +| prisma_cloud.host.os_distro.version | OS distribution version. | keyword | +| prisma_cloud.host.package.correlation_done | PackageCorrelationDone indicates that the correlation to OS packages has been done. | boolean | +| prisma_cloud.host.package.manager | Indicates if the package manager is installed for the OS. | boolean | +| prisma_cloud.host.packages.pkgs.binary_idx | Indexes of the top binaries which use the package. | long | +| prisma_cloud.host.packages.pkgs.binary_pkgs | Names of the distro binary packages (packages which are built on the source of the package). | keyword | +| prisma_cloud.host.packages.pkgs.cve_count | Total number of CVEs for this specific package. | long | +| prisma_cloud.host.packages.pkgs.default_gem | DefaultGem indicates this is a gem default package (and not a bundled package). | boolean | +| prisma_cloud.host.packages.pkgs.files.md5 | Hash sum of the file using md5. | keyword | +| prisma_cloud.host.packages.pkgs.files.path | Path of the file. | keyword | +| prisma_cloud.host.packages.pkgs.files.sha1 | Hash sum of the file using SHA-1. | keyword | +| prisma_cloud.host.packages.pkgs.files.sha256 | Hash sum of the file using SHA256. | keyword | +| prisma_cloud.host.packages.pkgs.function_layer | ID of the serverless layer in which the package was discovered. | keyword | +| prisma_cloud.host.packages.pkgs.go_pkg | GoPkg indicates this is a Go package (and not module). | boolean | +| prisma_cloud.host.packages.pkgs.jar_identifier | JarIdentifier holds an additional identification detail of a JAR package. | keyword | +| prisma_cloud.host.packages.pkgs.layer_time | Image layer to which the package belongs (layer creation time). | date | +| prisma_cloud.host.packages.pkgs.license | License information for the package. | keyword | +| prisma_cloud.host.packages.pkgs.name | Name of the package. | keyword | +| prisma_cloud.host.packages.pkgs.os_package | OSPackage indicates that a python/java package was installed as an OS package. | boolean | +| prisma_cloud.host.packages.pkgs.path | Full package path (e.g., JAR or Node.js package path). | keyword | +| prisma_cloud.host.packages.pkgs.version | Package version. | keyword | +| prisma_cloud.host.packages.pkgs_type | PackageType describes the package type. | keyword | +| prisma_cloud.host.pull_duration | PullDuration is the time it took to pull the image. | long | +| prisma_cloud.host.push_time | PushTime is the image push time to the registry. | date | +| prisma_cloud.host.red_hat_non_rpm_image | RedHatNonRPMImage indicates whether the image is a Red Hat image with non-RPM content. | boolean | +| prisma_cloud.host.registry.namespace | IBM cloud namespace to which the image belongs. | keyword | +| prisma_cloud.host.registry.tags | RegistryTags are the tags of the registry this image is stored. | keyword | +| prisma_cloud.host.registry.type | RegistryType indicates the registry type where the image is stored. | keyword | +| prisma_cloud.host.repo_digests | Digests of the image. Used for content trust (notary). Has one digest per tag. | keyword | +| prisma_cloud.host.repo_tag.digest | Image digest (requires V2 or later registry). | keyword | +| prisma_cloud.host.repo_tag.id | ID of the image. | keyword | +| prisma_cloud.host.repo_tag.registry | Registry name to which the image belongs. | keyword | +| prisma_cloud.host.repo_tag.repo | Repository name to which the image belongs. | keyword | +| prisma_cloud.host.repo_tag.value | Image tag. | keyword | +| prisma_cloud.host.rhel_repos | RhelRepositories are the (RPM) repositories IDs from which the packages in this image were installed Used for matching vulnerabilities by Red Hat CPEs. | keyword | +| prisma_cloud.host.risk_factors | RiskFactors maps the existence of vulnerability risk factors. | flattened | +| prisma_cloud.host.runtime_enabled | HostRuntimeEnabled indicates if any runtime rule applies to the host. | boolean | +| prisma_cloud.host.scan.build_date | Scanner build date that published the image. | date | +| prisma_cloud.host.scan.duration | ScanDuration is the total time it took to scan the image. | long | +| prisma_cloud.host.scan.id | ScanID is the ID of the scan. | keyword | +| prisma_cloud.host.scan.time | Specifies the time of the last scan of the image. | date | +| prisma_cloud.host.scan.version | Scanner version that published the image. | keyword | +| prisma_cloud.host.secrets | Secrets are paths to embedded secrets inside the image Note: capital letter JSON annotation is kept to avoid converting all images for backward-compatibility support. | keyword | +| prisma_cloud.host.startup_binaries.altered | Indicates if the binary was installed from a package manager and modified/replaced (true) or not (false). | boolean | +| prisma_cloud.host.startup_binaries.cve_count | Total number of CVEs for this specific binary. | long | +| prisma_cloud.host.startup_binaries.deps | Third-party package files which are used by the binary. | keyword | +| prisma_cloud.host.startup_binaries.file_mode | Represents the file's mode and permission bits. | long | +| prisma_cloud.host.startup_binaries.function_layer | ID of the serverless layer in which the package was discovered. | keyword | +| prisma_cloud.host.startup_binaries.md5 | Md5 hashset of the binary. | keyword | +| prisma_cloud.host.startup_binaries.missing_pkg | Indicates if this binary is not related to any package (true) or not (false). | boolean | +| prisma_cloud.host.startup_binaries.name | Name of the binary. | keyword | +| prisma_cloud.host.startup_binaries.path | Path is the path of the binary. | keyword | +| prisma_cloud.host.startup_binaries.pkg_root_dir | Path for searching packages used by the binary. | keyword | +| prisma_cloud.host.startup_binaries.services | Names of services which use the binary. | keyword | +| prisma_cloud.host.startup_binaries.version | Version of the binary. | keyword | +| prisma_cloud.host.stopped | Stopped indicates whether the host was running during the agentless scan. | boolean | +| prisma_cloud.host.tags.digest | Image digest (requires V2 or later registry). | keyword | +| prisma_cloud.host.tags.id | ID of the image. | keyword | +| prisma_cloud.host.tags.registry | Registry name to which the image belongs. | keyword | +| prisma_cloud.host.tags.repo | Repository name to which the image belongs. | keyword | +| prisma_cloud.host.tags.tag | Image tag. | keyword | +| prisma_cloud.host.top_layer | SHA256 of the image's last layer that is the last element of the Layers field. | keyword | +| prisma_cloud.host.trust_result.groups._id | Name of the group. | keyword | +| prisma_cloud.host.trust_result.groups.disabled | Indicates if the rule is currently disabled (true) or not (false). | boolean | +| prisma_cloud.host.trust_result.groups.images | Image names or IDs (e.g., docker.io/library/ubuntu:16.04 / SHA264@...). | keyword | +| prisma_cloud.host.trust_result.groups.layers | Filesystem layers. The image is trusted if its layers have a prefix of the trusted groups layer in the same order. | keyword | +| prisma_cloud.host.trust_result.groups.modified | Datetime when the rule was last modified. | date | +| prisma_cloud.host.trust_result.groups.name | Name of the rule. | keyword | +| prisma_cloud.host.trust_result.groups.notes | Free-form text. | keyword | +| prisma_cloud.host.trust_result.groups.owner | User who created or last modified the rule. | keyword | +| prisma_cloud.host.trust_result.groups.previous_name | Previous name of the rule. Required for rule renaming. | keyword | +| prisma_cloud.host.trust_result.hosts_statuses.host | Host name. | keyword | +| prisma_cloud.host.trust_result.hosts_statuses.status | Status is the trust status for an image. | keyword | +| prisma_cloud.host.trust_status | Status is the trust status for an image. | keyword | +| prisma_cloud.host.twistlock_image | Indicates if the image is a Twistlock image (true) or not (false). | boolean | +| prisma_cloud.host.type | ScanType displays the components for an ongoing scan. | keyword | +| prisma_cloud.host.vulnerabilities.count | Total number of vulnerabilities. | long | +| prisma_cloud.host.vulnerabilities.data.applicable_rules | Rules applied on the package. | keyword | +| prisma_cloud.host.vulnerabilities.data.binary_pkgs | Names of the distro binary package names (packages which are built from the source of the package). | keyword | +| prisma_cloud.host.vulnerabilities.data.block | Indicates if the vulnerability has a block effect (true) or not (false). | boolean | +| prisma_cloud.host.vulnerabilities.data.cause | Additional information regarding the root cause for the vulnerability. | keyword | +| prisma_cloud.host.vulnerabilities.data.cri | Indicates if this is a CRI-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.vulnerabilities.data.custom | Indicates if the vulnerability is a custom vulnerability (e.g., openscap, sandbox) (true) or not (false). | boolean | +| prisma_cloud.host.vulnerabilities.data.cve | CVE ID of the vulnerability (if applied). | keyword | +| prisma_cloud.host.vulnerabilities.data.cvss | CVSS score of the vulnerability. | float | +| prisma_cloud.host.vulnerabilities.data.description | Description of the vulnerability. | keyword | +| prisma_cloud.host.vulnerabilities.data.discovered | Specifies the time of discovery for the vulnerability. | date | +| prisma_cloud.host.vulnerabilities.data.exploit | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.vulnerabilities.data.exploits.kind | ExploitKind represents the kind of the exploit. | keyword | +| prisma_cloud.host.vulnerabilities.data.exploits.source | ExploitType represents the source of an exploit. | keyword | +| prisma_cloud.host.vulnerabilities.data.fix_date | Date/time when the vulnerability was fixed (in Unix time). | date | +| prisma_cloud.host.vulnerabilities.data.fix_link | Link to the vendor's fixed-version information. | keyword | +| prisma_cloud.host.vulnerabilities.data.function_layer | Specifies the serverless layer ID in which the vulnerability was discovered. | keyword | +| prisma_cloud.host.vulnerabilities.data.grace_period_days | Number of grace days left for a vulnerability, based on the configured grace period. Nil if no block vulnerability rule applies. | long | +| prisma_cloud.host.vulnerabilities.data.id | ID of the violation. | keyword | +| prisma_cloud.host.vulnerabilities.data.layer_time | Date/time of the image layer to which the CVE belongs. | date | +| prisma_cloud.host.vulnerabilities.data.link | Vendor link to the CVE. | keyword | +| prisma_cloud.host.vulnerabilities.data.package.name | Name of the package that caused the vulnerability. | keyword | +| prisma_cloud.host.vulnerabilities.data.package.version | Version of the package that caused the vulnerability (or null). | keyword | +| prisma_cloud.host.vulnerabilities.data.published | Date/time when the vulnerability was published (in Unix time). | date | +| prisma_cloud.host.vulnerabilities.data.risk_factors | RiskFactors maps the existence of vulnerability risk factors. | flattened | +| prisma_cloud.host.vulnerabilities.data.severity | Textual representation of the vulnerability's severity. | keyword | +| prisma_cloud.host.vulnerabilities.data.status | Vendor status for the vulnerability. | keyword | +| prisma_cloud.host.vulnerabilities.data.templates | List of templates with which the vulnerability is associated. | keyword | +| prisma_cloud.host.vulnerabilities.data.text | Description of the violation. | keyword | +| prisma_cloud.host.vulnerabilities.data.title | | keyword | +| prisma_cloud.host.vulnerabilities.data.twistlock | Indicates if this is a Twistlock-specific vulnerability (true) or not (false). | boolean | +| prisma_cloud.host.vulnerabilities.data.type | Type represents the vulnerability type. | keyword | +| prisma_cloud.host.vulnerabilities.data.vec_str | Textual representation of the metric values used to score the vulnerability. | keyword | +| prisma_cloud.host.vulnerabilities.data.vuln_tag_infos.color | Color is a hexadecimal representation of color code value. | keyword | +| prisma_cloud.host.vulnerabilities.data.vuln_tag_infos.comment | Tag comment in a specific vulnerability context. | keyword | +| prisma_cloud.host.vulnerabilities.data.vuln_tag_infos.name | Name of the tag. | keyword | +| prisma_cloud.host.vulnerabilities.data.wild_fire_malware.md5 | MD5 is the hash of the malicious binary. | keyword | +| prisma_cloud.host.vulnerabilities.data.wild_fire_malware.path | Path is the path to malicious binary. | keyword | +| prisma_cloud.host.vulnerabilities.data.wild_fire_malware.verdict | Verdict is the malicious source like grayware, malware and phishing. | keyword | +| prisma_cloud.host.vulnerability.distribution.critical | | long | +| prisma_cloud.host.vulnerability.distribution.high | | long | +| prisma_cloud.host.vulnerability.distribution.low | | long | +| prisma_cloud.host.vulnerability.distribution.medium | | long | +| prisma_cloud.host.vulnerability.distribution.total | | long | +| prisma_cloud.host.vulnerability.risk_score | Image's CVE risk score. | long | +| prisma_cloud.host.wild_fire_usage.bytes | Bytes is the total number of bytes uploaded to the WildFire API. | long | +| prisma_cloud.host.wild_fire_usage.queries | Queries is the number of queries to the WildFire API. | long | +| prisma_cloud.host.wild_fire_usage.uploads | Uploads is the number of uploads to the WildFire API. | long | +| tags | User defined tags. | keyword | + + +### Host Profile + +This is the `Host Profile` dataset. + +#### Example + +An example event for `host_profile` looks as following: + +```json +{ + "@timestamp": "2023-10-18T12:15:22.607Z", + "agent": { + "ephemeral_id": "27dd294d-e02a-4b56-a204-034c7853e226", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "data_stream": { + "dataset": "prisma_cloud.host_profile", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "host" + ], + "created": "2023-08-11T06:53:48.855Z", + "dataset": "prisma_cloud.host_profile", + "ingested": "2023-10-18T12:15:23Z", + "kind": "asset", + "original": "{\"_id\":\"DESKTOP-6PQXXMS\",\"collections\":[\"All\"],\"created\":\"2023-08-11T06:53:48.855Z\",\"hash\":1,\"time\":\"0001-01-01T00:00:00Z\"}", + "type": [ + "info" + ] + }, + "host": { + "hostname": "DESKTOP-6PQXXMS" + }, + "input": { + "type": "cel" + }, + "prisma_cloud": { + "host_profile": { + "_id": "DESKTOP-6PQXXMS", + "collections": [ + "All" + ], + "created": "2023-08-11T06:53:48.855Z", + "hash": "1", + "time": "0001-01-01T00:00:00.000Z" + } + }, + "related": { + "hash": [ + "1" + ], + "hosts": [ + "DESKTOP-6PQXXMS" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-host_profile" + ] +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| event.dataset | Event dataset. | constant_keyword | +| event.module | Event module. | constant_keyword | +| input.type | Type of filebeat input. | keyword | +| log.offset | Log offset. | long | +| log.source.address | Source address from which the log event was read / sent from. | keyword | +| prisma_cloud.host_profile._id | ID is the profile ID (hostname). | keyword | +| prisma_cloud.host_profile.account_id | AccountID is the cloud account ID associated with the profile. | keyword | +| prisma_cloud.host_profile.apps.listening_ports.command | Command represents the command that triggered the connection. | keyword | +| prisma_cloud.host_profile.apps.listening_ports.modified | Modified is a timestamp of when the event occurred. | date | +| prisma_cloud.host_profile.apps.listening_ports.port | Port is the port number. | long | +| prisma_cloud.host_profile.apps.listening_ports.process_path | ProcessPath represents the path to the process that uses the port. | keyword | +| prisma_cloud.host_profile.apps.name | Name is the app name. | keyword | +| prisma_cloud.host_profile.apps.outgoing_ports.command | Command represents the command that triggered the connection. | keyword | +| prisma_cloud.host_profile.apps.outgoing_ports.country | Country is the country ISO code for the given IP address. | keyword | +| prisma_cloud.host_profile.apps.outgoing_ports.ip | IP is the IP address captured over this port. | ip | +| prisma_cloud.host_profile.apps.outgoing_ports.modified | Modified is a timestamp of when the event occurred. | date | +| prisma_cloud.host_profile.apps.outgoing_ports.port | Port is the port number. | long | +| prisma_cloud.host_profile.apps.outgoing_ports.process_path | ProcessPath represents the path to the process that uses the port. | keyword | +| prisma_cloud.host_profile.apps.processes.command | Command represents the command that triggered the connection. | keyword | +| prisma_cloud.host_profile.apps.processes.interactive | Interactive indicates whether the process belongs to an interactive session. | boolean | +| prisma_cloud.host_profile.apps.processes.md5 | MD5 is the process binary MD5 sum. | keyword | +| prisma_cloud.host_profile.apps.processes.modified | Modified indicates the process binary was modified after the container has started. | boolean | +| prisma_cloud.host_profile.apps.processes.path | Path is the process binary path. | keyword | +| prisma_cloud.host_profile.apps.processes.ppath | PPath is the parent process path. | keyword | +| prisma_cloud.host_profile.apps.processes.time | Time is the time in which the process was added. If the process was modified, Time is the modification time. | date | +| prisma_cloud.host_profile.apps.processes.user | User represents the username that started the process. | keyword | +| prisma_cloud.host_profile.apps.startup_process.command | Command represents the command that triggered the connection. | keyword | +| prisma_cloud.host_profile.apps.startup_process.interactive | Interactive indicates whether the process belongs to an interactive session. | boolean | +| prisma_cloud.host_profile.apps.startup_process.md5 | MD5 is the process binary MD5 sum. | keyword | +| prisma_cloud.host_profile.apps.startup_process.modified | Modified is a timestamp of when the event occurred. | boolean | +| prisma_cloud.host_profile.apps.startup_process.path | Path is the process binary path. | keyword | +| prisma_cloud.host_profile.apps.startup_process.ppath | PPath is the parent process path. | keyword | +| prisma_cloud.host_profile.apps.startup_process.time | Time is the time in which the process was added. If the process was modified, Time is the modification time. | date | +| prisma_cloud.host_profile.apps.startup_process.user | User represents the username that started the process. | keyword | +| prisma_cloud.host_profile.collections | Collections is a list of collections to which this profile applies. | keyword | +| prisma_cloud.host_profile.created | Created is the profile creation time. | date | +| prisma_cloud.host_profile.geoip.countries.code | Code is the country iso code. | keyword | +| prisma_cloud.host_profile.geoip.countries.ip | Ip is the Ip address. | ip | +| prisma_cloud.host_profile.geoip.countries.modified | Modified is the last modified time of this entry. | date | +| prisma_cloud.host_profile.geoip.modified | Modified is the last modified time of the cache. | date | +| prisma_cloud.host_profile.hash | ProfileHash represents the profile hash It is allowed to contain up to uint32 numbers, and represented by int64 since mongodb does not support unsigned data types. | keyword | +| prisma_cloud.host_profile.labels | Labels are the labels associated with the profile. | keyword | +| prisma_cloud.host_profile.ssh_events.command | Command represents the command that triggered the connection. | keyword | +| prisma_cloud.host_profile.ssh_events.country | Country represents the SSH client's origin country. | keyword | +| prisma_cloud.host_profile.ssh_events.interactive | Interactive indicates whether the process belongs to an interactive session. | boolean | +| prisma_cloud.host_profile.ssh_events.ip | IP address represents the connection client IP address. | keyword | +| prisma_cloud.host_profile.ssh_events.login_time | LoginTime represents the SSH login time. | date | +| prisma_cloud.host_profile.ssh_events.md5 | MD5 is the process binary MD5 sum. | keyword | +| prisma_cloud.host_profile.ssh_events.modified | Modified indicates the process binary was modified after the container has started. | boolean | +| prisma_cloud.host_profile.ssh_events.path | Path is the process binary path. | keyword | +| prisma_cloud.host_profile.ssh_events.ppath | PPath is the parent process path. | keyword | +| prisma_cloud.host_profile.ssh_events.time | Time is the time in which the process was added. If the process was modified, Time is the modification time. | date | +| prisma_cloud.host_profile.ssh_events.user | User represents the username that started the process. | keyword | +| prisma_cloud.host_profile.time | Time is the last time when this profile was modified. | date | +| tags | User defined tags. | keyword | + + +### Incident Audit + +This is the `Incident Audit` dataset. + +#### Example + +An example event for `incident_audit` looks as following: + +```json +{ + "@timestamp": "2023-08-30T08:42:17.834Z", + "agent": { + "ephemeral_id": "36c96ae0-38a3-4668-a74c-b41dae4b633f", + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.10.1" + }, + "cloud": { + "account": { + "id": [ + "accounttest" + ] + }, + "provider": [ + "aws" + ], + "region": "testregion" + }, + "container": { + "id": "testcontainer", + "image": { + "name": [ + "testimgname" + ] + }, + "name": [ + "testcontainername" + ] + }, + "data_stream": { + "dataset": "prisma_cloud.incident_audit", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.10.0" + }, + "elastic_agent": { + "id": "acedddc9-63e6-47f7-b4b0-ad41d6af2116", + "snapshot": false, + "version": "8.10.1" + }, + "event": { + "agent_id_status": "verified", + "dataset": "prisma_cloud.incident_audit", + "id": "thgry1736", + "ingested": "2023-10-18T12:17:39Z", + "kind": "event", + "original": "{\"_id\":\"thgry1736\",\"accountID\":\"accounttest\",\"acknowledged\":true,\"app\":\"test\",\"appID\":\"test123\",\"audits\":[{\"_id\":\"id1234\",\"accountID\":\"accounttest\",\"app\":\"test\",\"appID\":\"test123\",\"attackTechniques\":[\"exploitationForPrivilegeEscalation\"],\"attackType\":[\"cloudMetadataProbing\"],\"cluster\":\"clustertest\",\"collections\":[\"collectiontest\"],\"command\":\"commandtest\",\"container\":true,\"containerId\":\"testcontainerid\",\"containerName\":\"testcontainername\",\"count\":0,\"country\":\"in\",\"domain\":\"testdomain\",\"effect\":\"block\",\"err\":\"testerr\",\"filepath\":\"testfilepath\",\"fqdn\":\"testfqdn\",\"function\":\"testfun\",\"functionID\":\"testfunid\",\"hostname\":\"testhostname\",\"imageId\":\"testimgid\",\"imageName\":\"testimgname\",\"interactive\":true,\"ip\":\"81.2.69.142\",\"label\":\"testlabel\",\"labels\":{\"sjhia\": \"ifo\"},\"md5\":\"testmd5\",\"msg\":\"testmsg\",\"namespace\":\"testnamespace\",\"os\":\"testos\",\"pid\":0,\"port\":0,\"processPath\":\"testprocesspath\",\"profileId\":\"testprofileid\",\"provider\":\"aws\",\"rawEvent\":\"testrawevent\",\"region\":\"testregion\",\"requestID\":\"testrequestid\",\"resourceID\":\"testresourceid\",\"ruleName\":\"testrulename\",\"runtime\":\"python\",\"severity\":\"low\",\"time\":\"2023-08-30T08:42:17.834Z\",\"type\":\"processes\",\"user\":\"testuser\",\"version\":\"testversion\",\"vmID\":\"testvmid\",\"wildFireReportURL\":\"testwildfirereporturl\"}],\"category\":\"portScanning\",\"cluster\":\"testcluster\",\"collections\":[\"testcollections\"],\"containerID\":\"testcontainer\",\"containerName\":\"testcontainername\",\"customRuleName\":\"testcustomrulename\",\"fqdn\":\"testfqdn\",\"function\":\"testfunction\",\"functionID\":\"testfunctionid\",\"hostname\":\"testhostname\",\"imageID\":\"testimgid\",\"imageName\":\"testimgname\",\"labels\":{\"aaa\":\"bbb\"},\"namespace\":\"testnamespace\",\"profileID\":\"testprofileid\",\"provider\":\"aws\",\"region\":\"testregion\",\"resourceID\":\"testresourceid\",\"runtime\":\"testruntime\",\"serialNum\":0,\"shouldCollect\":true,\"time\":\"2023-08-30T08:42:17.834Z\",\"type\":\"host\",\"vmID\":\"testvmid\",\"windows\":true}" + }, + "host": { + "domain": [ + "testfqdn" + ], + "hostname": "testhostname" + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.48.7:57454" + } + }, + "prisma_cloud": { + "incident_audit": { + "_id": "thgry1736", + "account_id": "accounttest", + "acknowledged": true, + "app": { + "id": "test123", + "value": "test" + }, + "category": "portScanning", + "cluster": "testcluster", + "collections": [ + "testcollections" + ], + "container": { + "id": "testcontainer", + "name": "testcontainername" + }, + "custom_rule_name": "testcustomrulename", + "data": [ + { + "_id": "id1234", + "account_id": "accounttest", + "app": { + "id": "test123", + "value": "test" + }, + "attack": { + "techniques": [ + "exploitationForPrivilegeEscalation" + ], + "type": [ + "cloudMetadataProbing" + ] + }, + "cluster": "clustertest", + "collections": [ + "collectiontest" + ], + "command": "commandtest", + "container": { + "id": "testcontainerid", + "name": "testcontainername", + "value": true + }, + "count": 0, + "country": "in", + "domain": "testdomain", + "effect": "block", + "err": "testerr", + "filepath": "testfilepath", + "fqdn": "testfqdn", + "function": { + "id": "testfunid", + "value": "testfun" + }, + "hostname": "testhostname", + "image": { + "id": "testimgid", + "name": "testimgname" + }, + "interactive": true, + "ip": "81.2.69.142", + "label": "testlabel", + "labels": { + "sjhia": "ifo" + }, + "md5": "testmd5", + "msg": "testmsg", + "namespace": "testnamespace", + "os": "testos", + "pid": 0, + "port": 0, + "process_path": "testprocesspath", + "profile_id": "testprofileid", + "provider": "aws", + "raw_event": "testrawevent", + "region": "testregion", + "request_id": "testrequestid", + "resource_id": "testresourceid", + "rule_name": "testrulename", + "runtime": "python", + "severity": "low", + "time": "2023-08-30T08:42:17.834Z", + "type": "processes", + "user": "testuser", + "version": "testversion", + "vm_id": "testvmid", + "wild_fire_report_url": "testwildfirereporturl" + } + ], + "fqdn": "testfqdn", + "function": { + "id": "testfunctionid", + "value": "testfunction" + }, + "hostname": "testhostname", + "image": { + "id": "testimgid", + "name": "testimgname" + }, + "labels": { + "aaa": "bbb" + }, + "namespace": "testnamespace", + "profile_id": "testprofileid", + "provider": "aws", + "region": "testregion", + "resource_id": "testresourceid", + "runtime": "testruntime", + "serial_num": 0, + "should_collect": true, + "time": "2023-08-30T08:42:17.834Z", + "type": "host", + "vm_id": "testvmid", + "windows": true + } + }, + "related": { + "hosts": [ + "testfqdn", + "testhostname" + ], + "ip": [ + "81.2.69.142" + ], + "user": [ + "testuser" + ] + }, + "rule": { + "name": [ + "testrulename", + "testcustomrulename" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "prisma_cloud-incident_audit" + ], + "threat": { + "technique": { + "subtechnique": { + "name": [ + "{0=cloudMetadataProbing}" + ] + } + } + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| event.dataset | Event dataset. | constant_keyword | +| event.module | Event module. | constant_keyword | +| input.type | Type of filebeat input. | keyword | +| log.offset | Log offset. | long | +| log.source.address | Source address from which the log event was read / sent from. | keyword | +| prisma_cloud.incident_audit._id | Internal ID of the incident. | keyword | +| prisma_cloud.incident_audit.account_id | Cloud account ID. | keyword | +| prisma_cloud.incident_audit.acknowledged | Indicates if the incident has been acknowledged (true) or not (false). | boolean | +| prisma_cloud.incident_audit.app.id | Application Id. | keyword | +| prisma_cloud.incident_audit.app.value | Application that caused the incident. | keyword | +| prisma_cloud.incident_audit.category | | keyword | +| prisma_cloud.incident_audit.cluster | Cluster on which the incident was found. | keyword | +| prisma_cloud.incident_audit.collections | Collections to which this incident applies. | keyword | +| prisma_cloud.incident_audit.container.id | ID of the container that triggered the incident. | keyword | +| prisma_cloud.incident_audit.container.name | Container name. | keyword | +| prisma_cloud.incident_audit.custom_rule_name | Name of the custom runtime rule that triggered the incident. | keyword | +| prisma_cloud.incident_audit.data._id | Internal ID of the incident. | keyword | +| prisma_cloud.incident_audit.data.account_id | ID of the cloud account where the audit was generated. | keyword | +| prisma_cloud.incident_audit.data.app.id | Application id. | keyword | +| prisma_cloud.incident_audit.data.app.value | Name of the service which violated the host policy. | keyword | +| prisma_cloud.incident_audit.data.attack.techniques | Given list of techniques in documentation. | keyword | +| prisma_cloud.incident_audit.data.attack.type | Given list in documentation.RuntimeAttackType is the sub-category of the attack (e.g., malware process, process not in model, etc...). | keyword | +| prisma_cloud.incident_audit.data.cluster | Cluster name. | keyword | +| prisma_cloud.incident_audit.data.collections | Collections to which this audit applies. | keyword | +| prisma_cloud.incident_audit.data.command | ScrubbedCommand is the command executed by the process with scrubbed PII. | keyword | +| prisma_cloud.incident_audit.data.container.id | ID of the container that violates the rule. | keyword | +| prisma_cloud.incident_audit.data.container.name | Container name. | keyword | +| prisma_cloud.incident_audit.data.container.value | Indicates if this is a container audit (true) or host audit (false). | boolean | +| prisma_cloud.incident_audit.data.count | Attack type audits count. | long | +| prisma_cloud.incident_audit.data.country | Outbound country for outgoing network audits. | keyword | +| prisma_cloud.incident_audit.data.domain | Domain is the requested domain. | keyword | +| prisma_cloud.incident_audit.data.effect | Possible values: [block,prevent,alert,disable]RuleEffect is the effect that will be used in the runtime rule. | keyword | +| prisma_cloud.incident_audit.data.err | Unknown error in the audit process. | keyword | +| prisma_cloud.incident_audit.data.filepath | Filepath is the path of the modified file. | keyword | +| prisma_cloud.incident_audit.data.fqdn | Current full domain name used in audit alerts. | keyword | +| prisma_cloud.incident_audit.data.function.id | Id of function invoked. | keyword | +| prisma_cloud.incident_audit.data.function.value | Name of the serverless function that caused the audit. | keyword | +| prisma_cloud.incident_audit.data.hostname | current hostname. | keyword | +| prisma_cloud.incident_audit.data.image.id | Container image Id. | keyword | +| prisma_cloud.incident_audit.data.image.name | Container image name. | keyword | +| prisma_cloud.incident_audit.data.interactive | Indicates if the audit was triggered from a process that was spawned in interactive mode (e.g., docker exec ...) (true) or not (false). | boolean | +| prisma_cloud.incident_audit.data.ip | IP is the connection destination IP address. | ip | +| prisma_cloud.incident_audit.data.label | Container deployment label. | keyword | +| prisma_cloud.incident_audit.data.labels | | flattened | +| prisma_cloud.incident_audit.data.md5 | MD5 is the MD5 of the modified file (only for executables). | keyword | +| prisma_cloud.incident_audit.data.msg | Blocking message text. | keyword | +| prisma_cloud.incident_audit.data.namespace | K8s deployment namespace. | keyword | +| prisma_cloud.incident_audit.data.os | Operating system distribution. | keyword | +| prisma_cloud.incident_audit.data.pid | ID of the process that caused the audit event. | long | +| prisma_cloud.incident_audit.data.port | Port is the connection destination port. | long | +| prisma_cloud.incident_audit.data.process_path | Path of the process that caused the audit event. | keyword | +| prisma_cloud.incident_audit.data.profile_id | Profile ID of the audit. | keyword | +| prisma_cloud.incident_audit.data.provider | Possible values: [aws,azure,gcp,alibaba,oci,others]. CloudProvider specifies the cloud provider name. | keyword | +| prisma_cloud.incident_audit.data.raw_event | Unparsed function handler event input. | keyword | +| prisma_cloud.incident_audit.data.region | Region of the resource where the audit was generated. | keyword | +| prisma_cloud.incident_audit.data.request_id | ID of the lambda function invocation request. | keyword | +| prisma_cloud.incident_audit.data.resource_id | Unique ID of the resource where the audit was generated. | keyword | +| prisma_cloud.incident_audit.data.rule_name | Name of the rule that was applied, if blocked. | keyword | +| prisma_cloud.incident_audit.data.runtime | [python,python3.6,python3.7,python3.8,python3.9,nodejs12.x,nodejs14.x,dotnetcore2.1,dotnetcore3.1,dotnet6,java8,java11,ruby2.7]. | keyword | +| prisma_cloud.incident_audit.data.severity | Possible value [high, low, medium]. | keyword | +| prisma_cloud.incident_audit.data.time | Time of the audit event (in UTC time). | date | +| prisma_cloud.incident_audit.data.type | Possible values: [processes,network,kubernetes,filesystem] RuntimeType represents the runtime protection type. | keyword | +| prisma_cloud.incident_audit.data.user | Service user. | keyword | +| prisma_cloud.incident_audit.data.version | Defender version. | keyword | +| prisma_cloud.incident_audit.data.vm_id | Azure unique VM ID where the audit was generated. | keyword | +| prisma_cloud.incident_audit.data.wild_fire_report_url | WildFireReportURL is a URL link of the report generated by wildFire. | keyword | +| prisma_cloud.incident_audit.fqdn | Current hostname's full domain name. | keyword | +| prisma_cloud.incident_audit.function.id | ID of the function that triggered the incident. | keyword | +| prisma_cloud.incident_audit.function.value | Name of the serverless function. | keyword | +| prisma_cloud.incident_audit.hostname | Current hostname. | keyword | +| prisma_cloud.incident_audit.image.id | Container image id. | keyword | +| prisma_cloud.incident_audit.image.name | Container image name. | keyword | +| prisma_cloud.incident_audit.labels | | flattened | +| prisma_cloud.incident_audit.namespace | k8s deployment namespace. | keyword | +| prisma_cloud.incident_audit.profile_id | Runtime profile ID. | keyword | +| prisma_cloud.incident_audit.provider | Possible values: [aws,azure,gcp,alibaba,oci,others]. | keyword | +| prisma_cloud.incident_audit.region | Region of the resource on which the incident was found. | keyword | +| prisma_cloud.incident_audit.resource_id | Unique ID of the resource on which the incident was found. | keyword | +| prisma_cloud.incident_audit.runtime | Runtime of the serverless function. | keyword | +| prisma_cloud.incident_audit.serial_num | Serial number of incident. | long | +| prisma_cloud.incident_audit.should_collect | Indicates if this incident should be collected (true) or not (false). | boolean | +| prisma_cloud.incident_audit.time | Time of the incident (in UTC time). | date | +| prisma_cloud.incident_audit.type | Possible values: [host,container,function,appEmbedded,fargate]. | keyword | +| prisma_cloud.incident_audit.vm_id | Azure unique VM ID on which the incident was found. | keyword | +| prisma_cloud.incident_audit.windows | Windows indicates if defender OS type is Windows. | boolean | +| tags | User defined tags. | keyword | diff --git a/packages/prisma_cloud/img/prisma_cloud-logo.svg b/packages/prisma_cloud/img/prisma_cloud-logo.svg new file mode 100644 index 00000000000..58bca4f2bab --- /dev/null +++ b/packages/prisma_cloud/img/prisma_cloud-logo.svg @@ -0,0 +1,3 @@ + + + \ No newline at end of file diff --git a/packages/prisma_cloud/manifest.yml b/packages/prisma_cloud/manifest.yml new file mode 100644 index 00000000000..2880b88106d --- /dev/null +++ b/packages/prisma_cloud/manifest.yml @@ -0,0 +1,113 @@ +format_version: 2.8.0 +name: prisma_cloud +title: "Palo Alto Prisma Cloud" +version: 0.1.0 +description: "Collect logs from Prisma Cloud with Elastic Agent." +type: integration +categories: + - security +conditions: + kibana.version: "^8.10.1" + elastic.subscription: "basic" +icons: + - src: /img/prisma_cloud-logo.svg + title: Prisma Cloud logo + size: 32x32 + type: image/svg+xml +policy_templates: + - name: sample + title: Sample logs + description: Collect sample logs + inputs: + - type: cel + title: Collect Prisma Cloud logs via API + description: Collecting Prisma Cloud via API. + vars: + - name: username + type: text + title: Username + description: Access Key ID of the Prisma Cloud Console. + multi: false + required: true + show_user: true + - name: password + type: password + title: Password + description: Secret Key of the Prisma Cloud Console. + multi: false + required: true + show_user: true + - name: proxy_url + type: text + title: Proxy URL + multi: false + required: false + show_user: false + description: URL to proxy connections in the form of http[s]://:@:. Please ensure your username and password are in URL encoded format. + - name: ssl + type: yaml + title: SSL Configuration + description: i.e. certificate_authorities, supported_protocols, verification_mode etc. + multi: false + required: false + show_user: false + default: | + #certificate_authorities: + # - | + # -----BEGIN CERTIFICATE----- + # MIIDCjCCAfKgAwIBAgITJ706Mu2wJlKckpIvkWxEHvEyijANBgkqhkiG9w0BAQsF + # ADAUMRIwEAYDVQQDDAlsb2NhbGhvc3QwIBcNMTkwNzIyMTkyOTA0WhgPMjExOTA2 + # MjgxOTI5MDRaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB + # BQADggEPADCCAQoCggEBANce58Y/JykI58iyOXpxGfw0/gMvF0hUQAcUrSMxEO6n + # fZRA49b4OV4SwWmA3395uL2eB2NB8y8qdQ9muXUdPBWE4l9rMZ6gmfu90N5B5uEl + # 94NcfBfYOKi1fJQ9i7WKhTjlRkMCgBkWPkUokvBZFRt8RtF7zI77BSEorHGQCk9t + # /D7BS0GJyfVEhftbWcFEAG3VRcoMhF7kUzYwp+qESoriFRYLeDWv68ZOvG7eoWnP + # PsvZStEVEimjvK5NSESEQa9xWyJOmlOKXhkdymtcUd/nXnx6UTCFgnkgzSdTWV41 + # CI6B6aJ9svCTI2QuoIq2HxX/ix7OvW1huVmcyHVxyUECAwEAAaNTMFEwHQYDVR0O + # BBYEFPwN1OceFGm9v6ux8G+DZ3TUDYxqMB8GA1UdIwQYMBaAFPwN1OceFGm9v6ux + # 8G+DZ3TUDYxqMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAG5D + # 874A4YI7YUwOVsVAdbWtgp1d0zKcPRR+r2OdSbTAV5/gcS3jgBJ3i1BN34JuDVFw + # 3DeJSYT3nxy2Y56lLnxDeF8CUTUtVQx3CuGkRg1ouGAHpO/6OqOhwLLorEmxi7tA + # H2O8mtT0poX5AnOAhzVy7QW0D/k4WaoLyckM5hUa6RtvgvLxOwA0U+VGurCDoctu + # 8F4QOgTAWyh8EZIwaKCliFRSynDpv3JTUwtfZkxo6K6nce1RhCWFAsMvDZL8Dgc0 + # yvgJ38BRsFOtkRuAGSf6ZUwTO8JJRRIFnpUzXflAnGivK9M13D5GEQMmIl6U9Pvk + # sxSmbIUfc2SGJGCJD4I= + # -----END CERTIFICATE----- + - type: tcp + title: Collect Prisma Cloud logs via TCP input + description: Collecting logs from Prisma Cloud instance via TCP input. + vars: + - name: ssl + type: yaml + title: SSL Configuration + description: i.e. certificate_authorities, supported_protocols, verification_mode etc. + multi: false + required: false + show_user: false + default: | + #certificate_authorities: + # - | + # -----BEGIN CERTIFICATE----- + # MIIDCjCCAfKgAwIBAgITJ706Mu2wJlKckpIvkWxEHvEyijANBgkqhkiG9w0BAQsF + # ADAUMRIwEAYDVQQDDAlsb2NhbGhvc3QwIBcNMTkwNzIyMTkyOTA0WhgPMjExOTA2 + # MjgxOTI5MDRaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB + # BQADggEPADCCAQoCggEBANce58Y/JykI58iyOXpxGfw0/gMvF0hUQAcUrSMxEO6n + # fZRA49b4OV4SwWmA3395uL2eB2NB8y8qdQ9muXUdPBWE4l9rMZ6gmfu90N5B5uEl + # 94NcfBfYOKi1fJQ9i7WKhTjlRkMCgBkWPkUokvBZFRt8RtF7zI77BSEorHGQCk9t + # /D7BS0GJyfVEhftbWcFEAG3VRcoMhF7kUzYwp+qESoriFRYLeDWv68ZOvG7eoWnP + # PsvZStEVEimjvK5NSESEQa9xWyJOmlOKXhkdymtcUd/nXnx6UTCFgnkgzSdTWV41 + # CI6B6aJ9svCTI2QuoIq2HxX/ix7OvW1huVmcyHVxyUECAwEAAaNTMFEwHQYDVR0O + # BBYEFPwN1OceFGm9v6ux8G+DZ3TUDYxqMB8GA1UdIwQYMBaAFPwN1OceFGm9v6ux + # 8G+DZ3TUDYxqMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAG5D + # 874A4YI7YUwOVsVAdbWtgp1d0zKcPRR+r2OdSbTAV5/gcS3jgBJ3i1BN34JuDVFw + # 3DeJSYT3nxy2Y56lLnxDeF8CUTUtVQx3CuGkRg1ouGAHpO/6OqOhwLLorEmxi7tA + # H2O8mtT0poX5AnOAhzVy7QW0D/k4WaoLyckM5hUa6RtvgvLxOwA0U+VGurCDoctu + # 8F4QOgTAWyh8EZIwaKCliFRSynDpv3JTUwtfZkxo6K6nce1RhCWFAsMvDZL8Dgc0 + # yvgJ38BRsFOtkRuAGSf6ZUwTO8JJRRIFnpUzXflAnGivK9M13D5GEQMmIl6U9Pvk + # sxSmbIUfc2SGJGCJD4I= + # -----END CERTIFICATE----- + - type: udp + title: Collect Prisma Cloud logs via UDP input + description: Collecting logs from Prisma Cloud instance via UDP input. +owner: + github: elastic/security-external-integrations