[Pal/Linux-SGX] Get rid of sgx driver submodule

.ci/lib/stage-build-sgx.jenkinsfile

@@ -18,19 +18,12 @@ stage('build') {
         # test the build with the DCAP driver v1.6 and clean up afterwards
 
         cd "$WORKSPACE"
-
-        ISGX_DRIVER_PATH=/opt/intel/SGXDataCenterAttestationPrimitives/driver/linux \
-            make ${MAKEOPTS} -C Pal/src/host/Linux-SGX/sgx-driver
-        make ${MAKEOPTS}
-
+        make ${MAKEOPTS} ISGX_DRIVER_PATH=/opt/intel/SGXDataCenterAttestationPrimitives/driver/linux
         make ${MAKEOPTS} clean
-        make ${MAKEOPTS} -C Pal/src/host/Linux-SGX/sgx-driver distclean
     '''
 
     sh '''
-        ISGX_DRIVER_PATH=/opt/intel/linux-sgx-driver \
-            make ${MAKEOPTS} -C Pal/src/host/Linux-SGX/sgx-driver
-        make ${MAKEOPTS}
+        make ${MAKEOPTS} ISGX_DRIVER_PATH=/opt/intel/linux-sgx-driver
     '''
 
     try {

.ci/run-pylint

@@ -18,7 +18,6 @@ find . -name \*.py \
     -and -not -path ./LibOS/shim/test/ltp/build/\* \
     -and -not -path ./LibOS/shim/test/ltp/install/\* \
     -and -not -path ./Examples/pytorch/\* \
-    -and -not -path ./Pal/src/host/Linux-SGX/sgx-driver/\* \
 | sed 's/./\\&/g' \
 | xargs "${PYLINT}" "$@" \
     Pal/src/host/Linux-SGX/signer/pal-sgx-get-token \

.gitmodules

@@ -1,6 +1,3 @@
-[submodule "Pal/src/host/Linux-SGX/sgx-driver"]
-	path = Pal/src/host/Linux-SGX/sgx-driver
-	url = https://github.com/oscarlab/graphene-sgx-driver.git
 [submodule "LibOS/shim/test/ltp/src"]
 	path = LibOS/shim/test/ltp/src
 	url = https://github.com/linux-test-project/ltp.git

Documentation/building.rst

@@ -72,23 +72,24 @@ Run the following commands on Ubuntu to install SGX-related dependencies::
     sudo apt install -y python3-pip
     sudo /usr/bin/pip3 install protobuf
 
-2. Install the Linux kernel patched with FSGSBASE
-"""""""""""""""""""""""""""""""""""""""""""""""""
+2a. Install the Linux kernel patched with FSGSBASE
+""""""""""""""""""""""""""""""""""""""""""""""""""
 
 FSGSBASE is a feature in recent processors which allows direct access to the FS
 and GS segment base addresses. For more information about FSGSBASE and its
 benefits, see `this discussion <https://lwn.net/Articles/821719>`__.
+FSGSBASE patchset was merged in 5.9. For older kernels it is available as
+`separate patches <https://github.com/oscarlab/graphene-sgx-driver/tree/master/fsgsbase_patches>`__.
 
-Work is being done to include FSGSBASE enabling in the upstream Linux kernel.
-Currently, the FSGSBASE enabling code is out-of-tree, requiring some patches to
-the kernel.
+The following instructions to patch and compile a Linux kernel with FSGSBASE
+support below are written around Ubuntu 18.04 LTS (Bionic Beaver) with a Linux
+5.4 LTS stable kernel but can be adapted for other distros as necessary. These
+instructions ensure that the resulting kernel has FSGSBASE support and up to
+date security mitigations.
 
-Enabling FSGSBASE support requires building and installing a custom kernel with
-backported patches. The instructions to patch and compile a Linux kernel with
-FSGSBASE support below are written around Ubuntu 18.04 LTS (Bionic Beaver) with
-a Linux 5.4 LTS stable kernel but can be adapted for other distros as necessary.
-These instructions ensure that the resulting kernel has FSGSBASE support and up
-to date security mitigations.
+#. Clone the repository with patches::
+
+       git clone https://github.com/oscarlab/graphene-sgx-driver
 
 #. Setup a build environment for kernel development following `the instructions
    in the Ubuntu wiki <https://wiki.ubuntu.com/KernelTeam/GitKernelBuild>`__.
@@ -100,7 +101,7 @@ to date security mitigations.
 
 #. Apply the provided FSGSBASE patches to the kernel source tree::
 
-       git am <graphene-dir>/Pal/src/host/Linux-SGX/sgx-driver/fsgsbase_patches/*.patch
+       git am <graphene-sgx-driver>/fsgsbase_patches/*.patch
 
    The conversation regarding this patchset can be found in the kernel mailing
    list archives `here
@@ -126,6 +127,27 @@ these software packages may not work with recent Linux kernels like 5.4. We
 recommend to use commit ``b7ccf6f`` of the Intel SGX Linux Driver for Intel SGX
 DCAP and commit ``0e71c22`` of the Intel SGX SDK/PSW.
 
+2b. Install the Graphene FSGSBASE driver (not for production)
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+If you followed step 2a and installed the patched Linux kernel, skip this step.
+Otherwise, you will need a Graphene-specific Linux driver that enables the
+FSGSBASE feature available in recent processors.
+
+.. warning::
+
+   This module is a |~| quick-and-dirty hack with dangerous security hole
+   (allows unauthorized local privilege escalation). "Do not use for production"
+   is not a |~| joke. We use it only for testing on very old kernels where the
+   patchset does not apply cleanly.
+
+To install the Graphene FSGSBASE driver, run the following commands::
+
+   git clone https://github.com/oscarlab/graphene-sgx-driver
+   cd graphene-sgx-driver
+   make
+   sudo insmod gsgx.ko
+
 3. Generate signing keys
 """"""""""""""""""""""""
 
@@ -158,22 +180,6 @@ download and install it from:
 
 - https://github.com/intel/SGXDataCenterAttestationPrimitives
 
-5. Install the Graphene SGX driver (not for production)
-"""""""""""""""""""""""""""""""""""""""""""""""""""""""
-
-If you followed step 1 and installed the patched Linux kernel, skip this step.
-Otherwise, you will need a Graphene-specific Linux driver that enables the
-FSGSBASE feature available in recent processors.
-
-To install the Graphene SGX driver, run the following commands::
-
-   git submodule update --init -- Pal/src/host/Linux-SGX/sgx-driver
-   cd Pal/src/host/Linux-SGX/sgx-driver
-   make
-   # the console will prompt you for the path to the Intel SGX driver code
-   # (simply press ENTER if you use the in-kernel Intel SGX driver)
-   sudo insmod gsgx.ko
-
 Building
 ^^^^^^^^
 

Documentation/cloud-deployment.rst

@@ -40,7 +40,6 @@ Building
 
        git clone https://github.com/oscarlab/graphene.git
        cd graphene
-       git submodule update --init -- Pal/src/host/Linux-SGX/sgx-driver/
 
 #. Prepare the signing keys::
 

Documentation/quickstart.rst

@@ -55,7 +55,6 @@ second command should list the process status of :command:`aesm_service`.
 
       git clone https://github.com/oscarlab/graphene.git
       cd graphene
-      git submodule update --init -- Pal/src/host/Linux-SGX/sgx-driver/
       export GRAPHENE_DIR=$PWD
 
 #. Prepare a signing key::

LICENSE.addendum.txt

@@ -1,9 +1,5 @@
 Graphene itself is licensed under the LGPL-3.0-or-later.
 
-Graphene on an SGX host requires a kernel driver (in
-Pal/src/host/Linux-SGX/sgx-driver), which is separately licensed under
-the GPL.
-
 Graphene also includes the following third party sources (and licenses):
 
 mbedtls crypto libraries - Apache 2.0

Pal/Makefile

@@ -33,7 +33,6 @@ format:
 	                               -path ./src/host/Linux-SGX/tools/common/cJSON.h -prune -o \
 	                               -path ./src/host/Linux-SGX/tools/common/cJSON-*/cJSON.c -prune -o \
 	                               -path ./src/host/Linux-SGX/tools/common/cJSON-*/cJSON.h -prune -o \
-	                               -path ./src/host/Linux-SGX/sgx-driver -prune -o \
 	                               \( -name '*.h' -o -name '*.c' \) -print)
 
 .PHONY: distclean

Pal/src/host/Linux-SGX/.gitignore

@@ -1,6 +1,7 @@
 /asm-offsets.h
 /generated-offsets.s
 /generated_offsets.py
+/gsgx.h
 /pal-sgx
 /quote/aesm.pb-c.c
 /quote/aesm.pb-c.h

Pal/src/host/Linux-SGX/Makefile

@@ -14,8 +14,7 @@ CFLAGS += \
 	-I../../../include/lib \
 	-I../../../lib/crypto/mbedtls/include \
 	-I../../../lib/crypto/mbedtls/crypto/include \
-	-Iprotected-files \
-	-Isgx-driver
+	-Iprotected-files
 
 # Some of the code uses alignof on expressions, which is a GNU extension.
 # Silence Clang - it complains but does support it.
@@ -89,7 +88,7 @@ urts-asm-objs = sgx_entry.o
 graphene_lib = .lib/graphene-lib.a
 
 .PHONY: all
-all: sgx-driver/sgx.h $(host_files) tools
+all: gsgx.h $(host_files) tools
 
 libpal-Linux-SGX.a: $(enclave-objs) $(enclave-asm-objs)
 	$(call cmd,ar_a_o)
@@ -138,8 +137,8 @@ gdb_integration/sgx_gdb.so: gdb_integration/sgx_gdb.c
 
 enclave_entry.o sgx_entry.o: asm-offsets.h
 
-sgx-driver/sgx.h:
-	$(MAKE) -C sgx-driver $(notdir $@)
+gsgx.h: gsgx.h.in
+	./link-intel-driver.py < $< > $@
 
 ifeq ($(filter %clean,$(MAKECMDGOALS)),)
 include $(wildcard *.d) $(wildcard gdb_integration/*.d)
@@ -154,6 +153,7 @@ tools:
 CLEAN_FILES += $(notdir $(pal_static) $(pal_lib) $(pal_loader))
 CLEAN_FILES += gdb_integration/sgx_gdb.so
 CLEAN_FILES += quote/aesm.pb-c.c quote/aesm.pb-c.h quote/aesm.pb-c.d quote/aesm.pb-c.o
+CLEAN_FILES += gsgx.h
 
 .PHONY: clean_
 clean_:
@@ -163,12 +163,10 @@ clean_:
 
 .PHONY: clean
 clean: clean_
-	$(MAKE) -C sgx-driver $@
 	$(MAKE) -C tools $@
 
 .PHONY: distclean
 distclean: clean_
-	$(MAKE) -C sgx-driver $@
 	$(MAKE) -C tools $@
 
 .PHONY: test

Pal/src/host/Linux-SGX/db_misc.c

@@ -18,12 +18,7 @@
 #include "pal_linux.h"
 #include "pal_linux_defs.h"
 #include "pal_security.h"
-/* sgx.h is required to define SGX_DCAP,
- * and doesn't have a definition for __packed */
-#ifndef __packed
-#define __packed __attribute__((packed))
-#endif
-#include "sgx.h"
+#include "gsgx.h"
 #include "sgx_api.h"
 #include "sgx_attest.h"
 #include "toml.h"

Pal/src/host/Linux-SGX/generated-offsets.c

@@ -11,12 +11,7 @@
 #include "sgx_arch.h"
 #include "sgx_tls.h"
 
-/* sgx.h header from the Intel SGX driver assumes that `__packed` macro was defined */
-#ifndef __packed
-#define __packed __attribute__((packed))
-#endif
-#include "sgx.h"
-#undef __packed
+#include "gsgx.h"
 
 __attribute__((__used__)) static void dummy(void) {
     /* defines in sgx_arch.h */
@@ -180,7 +175,4 @@ __attribute__((__used__)) static void dummy(void) {
 #ifdef SGX_DCAP
     DEFINE(SGX_DCAP, SGX_DCAP);
 #endif
-#ifdef SGX_DCAP_16_OR_LATER
-    DEFINE(SGX_DCAP_16_OR_LATER, SGX_DCAP_16_OR_LATER);
-#endif
 }

Pal/src/host/Linux-SGX/gsgx.h.in

@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: LGPL-3.0-or-later */
+/* (C) Copyright 2020 Intel Corporation
+ *                    Dmitrii Kuvaiskii <[email protected]>
+ */
+
+#ifndef __ARCH_GSGX_H__
+#define __ARCH_GSGX_H__
+
+#ifndef __packed
+#define __packed __attribute__((packed))
+#endif
+
+#include <linux/stddef.h>
+#include <linux/types.h>
+
+#include "@DRIVER_SGX_H@"
+
+#define GSGX_FILE  "/dev/gsgx"
+
+/* Graphene needs the below subset of SGX instructions' return values */
+#ifndef SGX_INVALID_SIG_STRUCT
+#define SGX_INVALID_SIG_STRUCT  1
+#endif
+
+#ifndef SGX_INVALID_ATTRIBUTE
+#define SGX_INVALID_ATTRIBUTE   2
+#endif
+
+#ifndef SGX_INVALID_MEASUREMENT
+#define SGX_INVALID_MEASUREMENT 4
+#endif
+
+#ifndef SGX_INVALID_SIGNATURE
+#define SGX_INVALID_SIGNATURE   8
+#endif
+
+#ifndef SGX_INVALID_EINITTOKEN
+#define SGX_INVALID_EINITTOKEN  16
+#endif
+
+#ifndef SGX_INVALID_CPUSVN
+#define SGX_INVALID_CPUSVN      32
+#endif
+
+/* SGX_INVALID_LICENSE was renamed to SGX_INVALID_EINITTOKEN in SGX driver 2.1:
+ *   https://github.com/intel/linux-sgx-driver/commit/a7997dafe184d7d527683d8d46c4066db205758d */
+#ifndef SGX_INVALID_LICENSE
+#define SGX_INVALID_LICENSE     SGX_INVALID_EINITTOKEN
+#endif
+
+#define ISGX_FILE "@ISGX_FILE@"
+
+@DEFINE_DCAP@
+
+#endif /* __ARCH_GSGX_H__ */

Pal/src/host/Linux-SGX/link-intel-driver.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+
+import os
+import string
+import sys
+
+DRIVER_VERSIONS = {
+        'sgx_user.h':                 '/dev/isgx',
+        'include/uapi/asm/sgx_oot.h': '/dev/sgx/enclave',
+        'sgx_in_kernel.h':            '/dev/sgx/enclave',
+}
+
+def find_intel_sgx_driver(isgx_driver_path):
+    '''
+    Graphene only needs one header from the Intel SGX Driver:
+      - sgx_user.h for non-DCAP, older version of the driver
+        (https://github.com/intel/linux-sgx-driver)
+      - include/uapi/asm/sgx_oot.h for DCAP 1.6+ version of the driver
+        (https://github.com/intel/SGXDataCenterAttestationPrimitives)
+      - default sgx_in_kernel.h for in-kernel 32+ version of the driver
+        (https://lore.kernel.org/linux-sgx/[email protected])
+
+    This function returns the required header from the SGX driver.
+    '''
+    for header_path, dev_path in DRIVER_VERSIONS.items():
+        abs_header_path = os.path.abspath(os.path.join(isgx_driver_path, header_path))
+        if os.path.exists(abs_header_path):
+            return abs_header_path, dev_path
+
+    print('Could not find the header from the Intel SGX Driver (ISGX_DRIVER_PATH={!r})'.format(
+        isgx_driver_path), file=sys.stderr)
+    sys.exit(1)
+
+class MesonTemplate(string.Template):
+    pattern = '''
+        @(?:
+            (?P<escaped>@) |
+            (?P<named>[A-Za-z0-9_]+)@ |
+            (?P<braced>[A-Za-z0-9_]+)@ |
+            (?P<invalid>)
+        )
+    '''
+
+def main():
+    '''
+    Find and copy header/device paths from Intel SGX Driver
+    '''
+    try:
+        isgx_driver_path = os.environ['ISGX_DRIVER_PATH']
+    except KeyError:
+        print(
+            'ISGX_DRIVER_PATH environment variable is undefined. You can define\n'
+            'ISGX_DRIVER_PATH="" to use the default in-kernel driver\'s C header.',
+            file=sys.stderr)
+        sys.exit(1)
+
+    if not isgx_driver_path:
+        # user did not specify any driver path, use default in-kernel driver's C header
+        isgx_driver_path = os.path.dirname(os.path.abspath(__file__))
+
+    header_path, dev_path = find_intel_sgx_driver(isgx_driver_path)
+
+    with sys.stdin:
+        template = MesonTemplate(sys.stdin.read())
+
+    sys.stdout.write(template.safe_substitute(
+        DRIVER_SGX_H=header_path,
+        ISGX_FILE=dev_path,
+        DEFINE_DCAP=('#define SGX_DCAP 1' if dev_path == '/dev/sgx/enclave' else '')
+    ))
+
+
+if __name__ == '__main__':
+    main()