gramine-sgx-get-token fails to connect to the AESMD service

[jogi343]

Issue:
gramine-sgx-get-token fails to connect to the AESMD service.

Steps to reproduce:
In the the /examples/gcc/ directory, run the command SGX=1 make check. This is also happening with other examples like curl.

Expected result:
The regression tests should run.

Actual result:
Getting the following output:

gramine-manifest \
	-Dlog_level=error \
	-Darch_libdir=/lib/x86_64-linux-gnu \
	-Dgcc_lib_path=/usr/lib/gcc/x86_64-linux-gnu \
	-Dgcc_major_version=9 \
	gcc.manifest.template >gcc.manifest
gramine-sgx-sign \
	--key /home/signer/enclave-key.pem \
	--manifest gcc.manifest \
	--output gcc.manifest.sgx
Attributes:
    size:        0x40000000
    thread_num:  4
    isv_prod_id: 0
    isv_svn:     0
    attr.flags:  0x4
    attr.xfrm:   0x3
    misc_select: 0x0
Memory:
    000000003fd6b000-0000000040000000 [REG:R--] (manifest) measured
    000000003fd4b000-000000003fd6b000 [REG:RW-] (ssa) measured
    000000003fd47000-000000003fd4b000 [TCS:---] (tcs) measured
    000000003fd43000-000000003fd47000 [REG:RW-] (tls) measured
    000000003fd03000-000000003fd43000 [REG:RW-] (stack) measured
    000000003fcc3000-000000003fd03000 [REG:RW-] (stack) measured
    000000003fc83000-000000003fcc3000 [REG:RW-] (stack) measured
    000000003fc43000-000000003fc83000 [REG:RW-] (stack) measured
    000000003fc33000-000000003fc43000 [REG:RW-] (sig_stack) measured
    000000003fc23000-000000003fc33000 [REG:RW-] (sig_stack) measured
    000000003fc13000-000000003fc23000 [REG:RW-] (sig_stack) measured
    000000003fc03000-000000003fc13000 [REG:RW-] (sig_stack) measured
    000000003f7de000-000000003f82b000 [REG:R-X] (code) measured
    000000003f82b000-000000003fc03000 [REG:RW-] (data) measured
    0000000000010000-000000003f7de000 [REG:RWX] (free)
Measurement:
    03d67895f6500af8eb7bb703409fcd2aad86c5f48377ede2f0b0627278c88652
gramine-sgx-get-token --output gcc.token --sig gcc.sig  #original
Attributes:
    mr_enclave:  03d67895f6500af8eb7bb703409fcd2aad86c5f48377ede2f0b0627278c88652
    mr_signer:   d4f51b18e558ee7496b81b93b4dfe92fd8f619f5e78d700a476b8dbf1a62c6df
    isv_prod_id: 0
    isv_svn:     0
    attr.flags:  0000000000000004
    attr.xfrm:   0000000000000007
    mask.flags:  ffffffffffffffff
    mask.xfrm:   fffffffffff9ff1b
    misc_select: 00000000
    misc_mask:   ffffffff
    modulus:     cf8d5982f6dd68250f926364ed34aca5...
    exponent:    3
    signature:   142c4c787b99bf83c396911e9b694aba...
    date:        2022-04-19
Traceback (most recent call last):
  File "/usr/local/bin/gramine-sgx-get-token", line 20, in <module>
    main() # pylint: disable=no-value-for-parameter
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/bin/gramine-sgx-get-token", line 16, in main
    token = get_token(sig, verbose=verbose)
  File "/usr/local/lib/python3.8/dist-packages/graminelibos/sgx_get_token.py", line 152, in get_token
    token = connect_aesmd(sig['enclave_hash'], sig['modulus'], sig['attribute_flags'], xfrms)
  File "/usr/local/lib/python3.8/dist-packages/graminelibos/sgx_get_token.py", line 89, in connect_aesmd
    raise Exception(f'Failed. (Error Code = {ret_msg.ret.error})')
Exception: Failed. (Error Code = 6)
make: *** [Makefile:57: gcc.token] Error 1

It seems this is the same issue as this in the gsc repo.

Other related info:

1. commit hash: 7ddf010
2. OS: Ubuntu 20.04, kernel: 5.9.16-050916-generic
3. SGX machine details: SGX1 with no FLC
4. SGX drivers used: OOT
5. Gramine is built from the source
6. The output of sudo service aesmd status:

    ● aesmd.service - Intel(R) Architectural Enclave Service Manager
        Loaded: loaded (/lib/systemd/system/aesmd.service; enabled; vendor preset: enabled)
        Active: active (running) since Wed 2022-04-13 08:37:33 CEST; 5 days ago
       Process: 953 ExecStartPre=/opt/intel/sgx-aesm-service/aesm/linksgx.sh (code=exited, status=0/SUCCESS)
       Process: 989 ExecStartPre=/bin/mkdir -p /var/run/aesmd/ (code=exited, status=0/SUCCESS)
       Process: 998 ExecStartPre=/bin/chown -R aesmd:aesmd /var/run/aesmd/ (code=exited, status=0/SUCCESS)
       Process: 1003 ExecStartPre=/bin/chmod 0755 /var/run/aesmd/ (code=exited, status=0/SUCCESS)
       Process: 1005 ExecStartPre=/bin/chown -R aesmd:aesmd /var/opt/aesmd/ (code=exited, status=0/SUCCESS)
       Process: 1014 ExecStartPre=/bin/chmod 0750 /var/opt/aesmd/ (code=exited, status=0/SUCCESS)
       Process: 1015 ExecStart=/opt/intel/sgx-aesm-service/aesm/aesm_service (code=exited, status=0/SUCCESS)
      Main PID: 1023 (aesm_service)
         Tasks: 4 (limit: 38145)
        Memory: 8.9M
        CGroup: /system.slice/aesmd.service
                └─1023 /opt/intel/sgx-aesm-service/aesm/aesm_service
   
   Apr 13 08:37:33 t-ubuntu-20 systemd[1]: Starting Intel(R) Architectural Enclave Service Manager...
   Apr 13 08:37:33 t-ubuntu-20 systemd[1]: Started Intel(R) Architectural Enclave Service Manager.
   Apr 13 08:37:33 t-ubuntu-20 aesm_service[1023]: [ADMIN]White List update requested
   Apr 13 08:37:33 t-ubuntu-20 aesm_service[1023]: The server sock is 0x559ec143a5e0
   Apr 13 08:37:33 t-ubuntu-20 aesm_service[1023]: [ADMIN]White List update failed due to network error
   Apr 19 01:51:58 t-ubuntu-20 aesm_service[1023]: [ADMIN]White List update requested
   Apr 19 01:51:58 t-ubuntu-20 aesm_service[1023]: [ADMIN]White list update request successful for Version: 110
   

[dimakuv]

SGX machine details: SGX1 with no FLC

Since you have this "old-style" machine, I think what happens is that AESMD looks into its White List and fails to find your built gcc enclave (more particularly, fails to find MRSIGNER with which your gcc enclave is signed).

This is because SGX1 (with no FLC) uses a very restrictive policy for non-debug SGX enclaves. Since SGX2 (with FLC), the policy is more relaxed and allows to run non-debug SGX enclaves.

Anyway, my assumption is that your machine doesn't allow you to run non-debug enclaves. And in Gramine, enclaves are non-debug by default. So please add sgx.debug = true in your gcc.manifest.template file, rebuild and run again. Hopefully this solves your problem (AESMD will be satisfied with the debug enclave and let it run).

[jogi343]

Thank you @dimakuv . This solution works!!