[Prisma Cloud] Initial Release for Prisma Cloud (#8135)

* Initial Release for Prisma Cloud

.github/CODEOWNERS

@@ -186,6 +186,7 @@
 /packages/ping_one @elastic/security-external-integrations
 /packages/platform_observability @elastic/infra-monitoring-ui
 /packages/postgresql @elastic/obs-infraobs-integrations
+/packages/prisma_cloud @elastic/security-external-integrations
 /packages/problemchild @elastic/ml-ui @elastic/sec-applied-ml
 /packages/prometheus @elastic/obs-cloudnative-monitoring
 /packages/prometheus_input @elastic/obs-infraobs-integrations

packages/prisma_cloud/_dev/build/build.yml

@@ -0,0 +1,4 @@
+dependencies:
+  ecs:
+    reference: [email protected]
+    import_mappings: true

packages/prisma_cloud/_dev/build/docs/README.md

@@ -0,0 +1,164 @@
+# Prisma Cloud
+
+This [Prisma Cloud](https://docs.paloaltonetworks.com/prisma/prisma-cloud/prisma-cloud-admin-compute/welcome) is a cloud infrastructure security solution and a Security Operations Center (SOC) enablement tool that enables you to address risks and secure your workloads in a heterogeneous environment (hybrid and multi cloud) from a single console. It provides complete visibility and control over risks within your public cloud infrastructure—Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Oracle Cloud Infrastructure (OCI), Alibaba Cloud— and enables you to manage vulnerabilities, detect anomalies, ensure compliance, and provide runtime defense in heterogeneous environments, such as Windows, Linux, Kubernetes, Red Hat OpenShift, AWS Lambda, Azure Functions, and GCP Cloud Functions.
+
+## Prisma Cloud Security Posture Management (CSPM)
+
+Single pane of glass for both CSPM (Cloud Security Posture Management) & CWPP (Cloud Workload Protection Platform). Compute (formerly Twistlock, a CWPP solution) is delivered as part of the larger Prisma Cloud system. Palo Alto Networks runs, manages, and updates Compute Console for you. You deploy and manage Defenders in your environment. You access the Compute Console from a tab within the Prisma Cloud user interface.
+
+CSPM uses REST API mode to collect data. Elastic Agent fetches data via API endpoints.
+
+## Prisma Cloud Workload Protection (CWP)
+
+Self-hosted, stand-alone, self-operated version of Compute (formerly Twistlock). Download the entire software suite, and run it in any environment. You deploy and manage both Console and Defenders.
+
+CWP can be used in two different modes to collect data:
+- REST API mode.
+- Syslog mode: This includes TCP and UDP.
+
+## Compatibility
+
+This module has been tested against the latest CSPM version **v2** and CWP version **v30.03**.
+
+## Data streams
+
+The Prisma Cloud integration collects data for the following five events:
+
+| Event Type                    |
+|-------------------------------|
+| Alert                         |
+| Audit                         |
+| Host                          |
+| Host Profile                  |
+| Incident Audit                |
+
+**NOTE**:
+
+1. Alert and Audit data-streams are part of [CSPM](https://pan.dev/prisma-cloud/api/cspm/) module, whereas Host, Host Profile and Incident Audit are part of [CWP](https://pan.dev/prisma-cloud/api/cwpp/) module.
+2. Currently, we are unable to collect logs of Incident Audit datastream via defined API. Hence, we have not added the configuration of Incident Audit data stream via REST API.
+
+## Requirements
+
+- Elastic Agent must be installed.
+- You can install only one Elastic Agent per host.
+- Elastic Agent is required to stream data through the REST API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.
+
+### Installing and managing an Elastic Agent:
+
+You have a few options for installing and managing an Elastic Agent:
+
+### Install a Fleet-managed Elastic Agent (recommended):
+
+With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.
+
+### Install Elastic Agent in standalone mode (advanced users):
+
+With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.
+
+### Install Elastic Agent in a containerized environment:
+
+You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.
+
+There are some minimum requirements for running Elastic Agent and for more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
+
+The minimum **kibana.version** required is **8.10.1**.
+
+## Setup
+
+### To collect data through REST API, follow the below steps:
+
+### CSPM
+
+1. Considering you already have a Prisma Cloud account, to obtain an access key ID and secret access key from the Prisma Cloud system administrator, refer this [link](https://docs.paloaltonetworks.com/prisma/prisma-cloud/prisma-cloud-admin/manage-prisma-cloud-administrators/create-access-keys).
+2. The base URL of your CSPM API request depends on the region of your Prisma Cloud tenant and is similar to your Prisma Cloud administrative console URL. Obtain your URL from this [link](https://pan.dev/prisma-cloud/api/cspm/api-urls/).
+
+### CWP
+
+1. Assuming you've already generated your access key ID and secret access key from the Prisma Cloud Console; if not, see the section above.
+2. The base URL of your CWP API request depends on the console path and the API version of your Prisma Cloud Compute console.
+3. To find your API version, log in to your Prisma Cloud Compute console, click the bell icon in the top right of the page, your API version is displayed.
+4. To get your console path, navigate to Compute > Manage > System > Downloads. you can find your console path listed under Path to Console.
+5. Now you can create your base URL in this format: `https://<CONSOLE>/api/v<VERSION>`.
+
+**NOTE**: You can specify a date and time for the access key validity. If you do not select key expiry, the key is set to never expire; if you select it, but do not specify a date, the key expires in a month.
+
+### Enabling the integration in Elastic:
+
+1. In Kibana go to Management > Integrations
+2. In "Search for integrations" search bar, type Palo Alto Prisma Cloud.
+3. Click on the "Palo Alto Prisma Cloud" integration from the search results.
+4. Click on the Add Palo Alto Prisma Cloud Integration button to add the integration.
+5. While adding the integration, if you want to collect Alert and Audit data via REST API, then you have to put the following details:
+   - username
+   - password
+   - url
+   - interval
+   - time amount
+   - time unit
+   - batch size
+
+   or if you want to collect Host, Host Profile and Incident Audit data via REST API, then you have to put the following details:
+   - username
+   - password
+   - url
+   - interval
+   - offset
+   - batch size
+
+  or if you want to collect Host, Host Profile and Incident Audit data via TCP/UDP, then you have to put the following details:
+   - listen address
+   - listen port
+
+**NOTE**: Your Access key ID is your username and Secret Access key is your password.
+
+## Logs Reference
+
+### Alert
+
+This is the `Alert` dataset.
+
+#### Example
+
+{{event "alert"}}
+
+{{fields "alert"}}
+
+### Audit
+
+This is the `Audit` dataset.
+
+#### Example
+
+{{event "audit"}}
+
+{{fields "audit"}}
+
+### Host
+
+This is the `Host` dataset.
+
+#### Example
+
+{{event "host"}}
+
+{{fields "host"}}
+
+### Host Profile
+
+This is the `Host Profile` dataset.
+
+#### Example
+
+{{event "host_profile"}}
+
+{{fields "host_profile"}}
+
+### Incident Audit
+
+This is the `Incident Audit` dataset.
+
+#### Example
+
+{{event "incident_audit"}}
+
+{{fields "incident_audit"}}

packages/prisma_cloud/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,45 @@
+version: '2.3'
+services:
+  prisma_cloud-host-tcp:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9508 -p=tcp /sample_logs/host.log
+  prisma_cloud-host-udp:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9509 -p=udp /sample_logs/host.log
+  prisma_cloud-host_profile-tcp:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9510 -p=tcp /sample_logs/host_profile.log
+  prisma_cloud-host_profile-udp:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9511 -p=udp /sample_logs/host_profile.log
+  prisma_cloud-incident_audit-tcp:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9512 -p=tcp /sample_logs/incident_audit.log
+  prisma_cloud-incident_audit-udp:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9513 -p=udp /sample_logs/incident_audit.log
+  prisma_cloud:
+    image: docker.elastic.co/observability/stream:v0.10.0
+    hostname: prisma_cloud
+    ports:
+      - 8090
+    volumes:
+      - ./files:/files:ro
+    environment:
+      PORT: '8090'
+    command:
+      - http-server
+      - --addr=:8090
+      - --config=/files/config.yml

packages/prisma_cloud/_dev/deploy/docker/files/config.yml

@@ -0,0 +1,85 @@
+rules:
+  - path: /login
+    methods: ['POST']
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - 'application/json'
+        body: |
+          {"message":"login_successful","token":"xxxx","customerNames":[{"customerName":"Company (Tech Partner Only) - 84706136261xxxxxx32","prismaId":"1121575xxxx8690944","tosAccepted":true}]}
+  - path: /v2/alert
+    methods: ['GET']
+    request_headers:
+      x-redlock-auth:
+        - 'xxxx'
+    responses:
+      - status_code: 200
+        body: |
+          {"totalRows":1,"items":[{"id":"N-3910","alertAdditionalInfo":{"scannerVersion":"CS_2.0"},"alertAttribution":{"attributionEventList":[{"event":"first_event","event_ts":1694003441966,"username":"alex123"}],"resourceCreatedBy":"string","resourceCreatedOn":0},"status":"open","reason":"NEW_ALERT","firstSeen":1694003441966,"history":[{"modifiedOn":"1694003441966","modifiedBy":"alex123","reason":"Reason1","status":"OPEN"}],"lastSeen":1694003441966,"alertTime":1694003441966,"lastUpdated":1694003441966,"policyId":"ad23603d-754e-4499-8988-b801xxx85898","metadata":null,"policy":{"policyId":"ad23603d-754e-4499-8988-b8017xxxx98","name":"AWS EC2 instance that is internet reachable with unrestricted access (0.0.0.0\/0)","policyType":"network","systemDefault":true,"complianceMetadata":[{"complianceId":"qwer345bv","customAssigned":true,"policyId":"werf435tr","requirementDescription":"Description of policy compliance.","requirementId":"req-123-xyz","requirementName":"rigidity","sectionDescription":"Description of section.","sectionId":"sect-453-abc","sectionLabel":"label-1","standardDescription":"Description of standard.","standardId":"stand-543-pqr","standardName":"Class 1"}],"description":"This policy identifies AWS EC2 instances that are internet reachable with unrestricted access (0.0.0.0\/0). EC2 instances with unrestricted access to the internet may enable bad actors to use brute force on a system to gain unauthorised access to the entire network. As a best practice, restrict traffic from unknown IP addresses and limit the access to known hosts, services, or specific entities.","severity":"high","recommendation":"The following steps are recommended to restrict unrestricted access from the Internet:\n1. Visit the Network path Analysis from Source to Destination and review the network path components that allow internet access.\n2. Identify the network component on which restrictive rules can be implemented.\n3. Implement the required changes and make sure no other resources have been impacted due to these changes:\n a) The overly permissive Security Group rules can be made more restrictive.\n b) Move the instance inside a restrictive subnet if the instance does not need to be publicly accessible.\n c) Define a NAT rule to restrict traffic coming from the Internet to the respective instance.","labels":["Prisma_Cloud","Attack Path Rule"],"lastModifiedOn":1687474999057,"lastModifiedBy":"[email protected]","deleted":false,"findingTypes":[],"remediable":false,"remediation":{"actions":[{"operation":"buy","payload":"erefwsdf"}],"cliScriptTemplate":"temp1","description":"Description of CLI Script Template."}},"alertRules":[],"resource":{"rrn":"rrn:aws:instance:us-east-1:710000059376:e7ddce5a1ffcb47bxxxxxerf2635a3b4d9da3:i-04578e0008100947","id":"i-04578exxxx8100947","name":"IS-37133","account":"AWS Cloud Account","accountId":"710002259376","cloudAccountGroups":["Default Account Group"],"region":"AWS Virginia","regionId":"us-east-1","resourceType":"INSTANCE","resourceApiName":"aws-ec2-describe-instances","cloudServiceName":"Amazon EC2","url":"https:\/\/console.aws.amazon.com\/ec2\/v2\/home?region=us-east-1#Instances:instanceId=i-0457xxxxx00947","data":null,"additionalInfo":null,"cloudType":"aws","resourceTs":1694003441915,"unifiedAssetId":"66c543b6261c4d9edxxxxxb42e15f4","resourceConfigJsonAvailable":false,"resourceDetailsAvailable":true},"investigateOptions":{"alertId":"N-3910"}}]}
+  - path: /audit/redlock
+    methods: ['GET']
+    request_headers:
+      x-redlock-auth:
+        - 'xxxx'
+    responses:
+      - status_code: 200
+        body: |
+          [{"timestamp":1694594439068,"user":"[email protected]","ipAddress":"81.2.69.192","actionType":"LOGIN","resourceName":"[email protected]","action":"'[email protected]'(with role 'System Admin':'System Admin') logged in via access key.","resourceType":"Login","result":"Successful"}]
+  - path: /authenticate
+    methods: ['POST']
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - 'application/json'
+        body: |
+          {"token":"xxxx"}
+  - path: /hosts
+    methods: ['GET']
+    request_headers:
+      Authorization:
+        - 'Bearer xxxx'
+    query_params:
+      offset: 0
+      limit: 50
+    responses:
+      - status_code: 200
+        body: |
+          [{"_id":"DESKTOP-6PQXXMS","binaries":[{"altered":true,"cveCount":0,"deps":["string"],"fileMode":0,"functionLayer":"string","md5":"string","missingPkg":true,"name":"string","path":"string","pkgRootDir":"string","services":["string"],"version":"string"}],"cloudMetadata":{"accountID":"Non-onboarded cloud accounts","awsExecutionEnv":"string","image":"string","labels":[{"key":"string","sourceName":"string","sourceType":["namespace"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"name":"string","provider":["aws"],"region":"string","resourceID":"string","resourceURL":"string","type":"string","vmID":"string","vmImageID":"string"},"type":"host","hostname":"DESKTOP-6PQXXMS","scanTime":"2023-08-23T11:48:41.803Z","Secrets":[],"osDistro":"windows","osDistroVersion":"string","osDistroRelease":"Windows","distro":"Microsoft Windows [Version 10.0.19045.2006]","packageManager":true,"packages":[{"pkgs":[{"binaryIdx":[0],"binaryPkgs":["string"],"cveCount":0,"defaultGem":true,"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"functionLayer":"string","goPkg":true,"jarIdentifier":"string","layerTime":0,"license":"string","name":"string","osPackage":true,"path":"string","version":"string"}],"pkgsType":"nodejs"}],"isARM64":false,"packageCorrelationDone":true,"redHatNonRPMImage":false,"image":{"created":"0001-01-01T00:00:00Z","entrypoint":["string"],"env":["string"],"healthcheck":true,"id":"string","labels":{},"layers":["string"],"os":"string","repoDigest":["string"],"repoTags":["string"],"user":"string","workingDir":"string"},"allCompliance":{"compliance":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.949Z","exploit":["exploit-db"],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI"]],"text":"string","title":"string","twistlock":true,"type":["container"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}],"enabled":"true"},"clusters":["string"],"repoTag":null,"tags":[{"digest":"string","id":"string","registry":"string","repo":"string","tag":"string"}],"trustResult":{"hostsStatuses":[{"host":"string","status":"trusted"}]},"repoDigests":[],"creationTime":"0001-01-01T00:00:00Z","pushTime":"0001-01-01T00:00:00Z","vulnerabilitiesCount":0,"complianceIssuesCount":4,"vulnerabilityDistribution":{"critical":0,"high":0,"medium":0,"low":0,"total":0},"complianceDistribution":{"critical":4,"high":0,"medium":0,"low":0,"total":4},"vulnerabilityRiskScore":0,"complianceRiskScore":4000000,"riskFactors":{},"firstScanTime":"2023-08-11T06:53:57.456Z","history":[{"baseLayer":true,"created":0,"emptyLayer":true,"id":"string","instruction":"string","sizeBytes":0,"tags":["string"],"vulnerabilities":[{"applicableRules":["string"],"binaryPkgs":["string"],"block":true,"cause":"string","cri":true,"custom":true,"cve":"string","cvss":0,"description":"string","discovered":"2023-09-08T04:01:49.950Z","exploit":["exploit-db"],"exploits":[{"kind":["poc","in-the-wild"],"link":"string","source":["","exploit-db"]}],"fixDate":0,"fixLink":"string","functionLayer":"string","gracePeriodDays":0,"id":0,"layerTime":0,"link":"string","packageName":"string","packageVersion":"string","published":0,"riskFactors":{},"severity":"string","status":"string","templates":[["PCI"]],"text":"string","title":"string","twistlock":true,"type":["container"],"vecStr":"string","vulnTagInfos":[{"color":"string","comment":"string","name":"string"}],"wildfireMalware":{"md5":"string","path":"string","verdict":"string"}}]}],"hostDevices":[{"ip":"0.0.0.0","name":"string"}],"hosts":{},"id":"string","err":"","collections":["All"],"instances":[{"host":"string","image":"string","modified":"2023-09-08T04:01:49.951Z","registry":"string","repo":"string","tag":"string"}],"scanID":0,"trustStatus":"","externalLabels":[{"key":"string","sourceName":"string","sourceType":["namespace"],"timestamp":"2023-09-08T04:01:49.949Z","value":"string"}],"files":[{"md5":"string","path":"string","sha1":"string","sha256":"string"}],"firewallProtection":{"enabled":false,"supported":false,"outOfBandMode":"Observation","ports":[0],"tlsPorts":[0],"unprotectedProcesses":[{"port":0,"process":"string","tls":true}]},"applications":[{"installedFromPackage":true,"knownVulnerabilities":0,"layerTime":0,"name":"string","path":"string","service":true,"version":"string"}],"appEmbedded":false,"wildFireUsage":null,"agentless":false,"malwareAnalyzedTime":"0001-01-01T00:00:00Z"}]
+  - path: /hosts
+    methods: ['GET']
+    request_headers:
+      Authorization:
+        - 'Bearer xxxx'
+    query_params:
+      offset: 1
+      limit: 50
+    responses:
+      - status_code: 200
+        body: |
+          null
+  - path: /profiles/host
+    methods: ['GET']
+    request_headers:
+      Authorization:
+        - 'Bearer xxxx'
+    query_params:
+      offset: 0
+      limit: 50
+    responses:
+      - status_code: 200
+        body: |
+          [{"_id":"DESKTOP-6PQXXMS","hash":1,"created":"2023-08-11T06:53:48.855Z","time":"0001-01-01T00:00:00Z","collections":["All"]}]
+  - path: /profiles/host
+    methods: ['GET']
+    request_headers:
+      Authorization:
+        - 'Bearer xxxx'
+    query_params:
+      offset: 1
+      limit: 50
+    responses:
+      - status_code: 200
+        body: |
+          []