Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add attestation guide to README.md in ppml/docker-graphene #5568

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 39 additions & 4 deletions ppml/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ Protecting privacy and confidentiality is critical for large-scale data analysis
      [Step 0. Preparation your environment](#step-0-preparation-your-environment): detailed steps in [Prepare Environment](https://github.com/liu-shaojun/BigDL/blob/ppml_doc/ppml/docs/prepare_environment.md) \
      [Step 1. Encrypt and Upload Data](#step-1-encrypt-and-upload-data) \
      [Step 2. Build Big Data & AI applications](#step-2-build-big-data--ai-applications) \
      [Step 3. Submit Job](#step-3-submit-job): 4 deploy modes and 2 options to submit job \
      [Step 4. Decrypt and Read Result](#step-4-decrypt-and-read-result) \
      [Step 3. Attestation ](#step-3-attestation) \
      [Step 4. Submit Job](#step-4-submit-job): 4 deploy modes and 2 options to submit job \
      [Step 5. Decrypt and Read Result](#step-5-decrypt-and-read-result) \
  [3.3 More BigDL PPML Examples](#33-more-bigdl-ppml-examples) \
[4. Develop your own Big Data & AI applications with BigDL PPML](#4-develop-your-own-big-data--ai-applications-with-bigdl-ppml) \
  [4.1 Create PPMLContext](#41-create-ppmlcontext) \
Expand Down Expand Up @@ -175,7 +176,41 @@ you can use [generate_people_csv.py](https://github.com/analytics-zoo/ppml-e2e-e
#### Step 2. Build Big Data & AI applications
To build your own Big Data & AI applications, refer to [develop your own Big Data & AI applications with BigDL PPML](#4-develop-your-own-big-data--ai-applications-with-bigdl-ppml). The code of SimpleQuery is in [here](https://github.com/intel-analytics/BigDL/blob/main/scala/ppml/src/main/scala/com/intel/analytics/bigdl/ppml/examples/SimpleQuerySparkExample.scala), it is already built into bigdl-ppml-spark_3.1.2-2.1.0-SNAPSHOT.jar, and the jar is put into PPML image.

#### Step 3. Submit Job
#### Step 3. Attestation

To enable attestation, you should have a running Attestation Service (EHSM-KMS here for example) in your environment. (You can start a KMS refering to [this link](https://github.com/intel-analytics/BigDL/tree/main/ppml/services/kms-utils/docker)). Configure your KMS app_id and app_key with `kubectl`, and then configure KMS settings in `spark-driver-template.yaml` and `spark-executor-template.yaml` in the container.
``` bash
kubectl create secret generic kms-secret --from-literal=app_id=your-kms-app-id --from-literal=app_key=your-kms-app-key
```
Configure `spark-driver-template.yaml` for example. (`spark-executor-template.yaml` is similar)
``` yaml
apiVersion: v1
kind: Pod
spec:
containers:
- name: spark-driver
securityContext:
privileged: true
env:
- name: ATTESTATION
value: true
- name: ATTESTATION_URL
value: your_attestation_url
- name: ATTESTATION_ID
valueFrom:
secretKeyRef:
name: kms-secret
key: app_id
- name: ATTESTATION_KEY
valueFrom:
secretKeyRef:
name: kms-secret
key: app_key
...
```
You should get `Attestation Success!` in logs after you [submit a PPML job](#step-4-submit-job) if the quote generated with user report is verified successfully by Attestation Service, or you will get `Attestation Fail! Application killed!` and the job will be stopped.

#### Step 4. Submit Job
When the Big Data & AI application and its input data is prepared, you are ready to submit BigDL PPML jobs. You need to choose the deploy mode and the way to submit job first.

* **There are 4 modes to submit job**:
Expand Down Expand Up @@ -274,7 +309,7 @@ Here we use **k8s client mode** and **PPML CLI** to run SimpleQuery. Check other
</details>
<br />

#### Step 4. Decrypt and Read Result
#### Step 5. Decrypt and Read Result
When the job is done, you can decrypt and read result of the job. More details in [Decrypt Job Result](./services/kms-utils/docker/README.md#3-enroll-generate-key-encrypt-and-decrypt).

```
Expand Down