Alibaba Cloud Linux:Use Inclavare Containers to implement remote attestation

Procedure

Step 1: Make preparations

1. Create a security-enhanced instance that uses an Alibaba Cloud Linux (UEFI) public image.

   The instance simulates the rats-tls-client that is on the cloud. When you create a security-enhanced instance, you must select an image that supports security features. For more information, see Create a trusted instance.

   Note

   When you create an instance, you must select a virtual private cloud (VPC) and a vSwitch for the instance and assign a public IP address to enable Internet access for the instance.

2. Build an SGX encrypted computing environment on the security-enhanced instance.

   For more information, see Build an SGX confidential computing environment.

3. Create an instance that uses a CentOS 8.2 public image.

   The instance simulates the on-promises host of Verdictd. For information about how to create an ECS instance, see Create an instance on the Custom Launch tab. For more information about Verdictd, visit Verdictd.

   Note

   When you create an instance, you must select a VPC and a vSwitch for the instance and assign a public IP address to enable Internet access for the instance.

4. Add a rule to a security group of the CentOS 8.2 instance to allow inbound traffic on port 1111.

   For more information, see Add a security group rule.

Step 2: Install a confidential container

1. Connect to the security-enhanced instance that uses an Alibaba Cloud Linux (UEFI) public image.

   For more information, see Connection method overview.

2. Run the following commands to install the rats-tls-sgx confidential container.

   The rats-tls-sgx installation package provides sample programs and library files for establishing secure communication channels based on the hardware-based TEE. For more information, visit rats-tls.

   • Run the following commands if the instance runs the Alibaba Cloud Linux 3.2104 64-bit (UEFI) operating system:

     sudo yum-config-manager --add-repo https://mirrors.openanolis.cn/inclavare-containers/alinux3-repo && \
       sudo rpm --import https://mirrors.openanolis.cn/inclavare-containers/alinux3-repo/RPM-GPG-KEY-rpm-sign && \
       sudo yum install -y rats-tls-sgx

   • Run the following commands if the instance runs the Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI) operating system:

     sudo yum-config-manager --add-repo https://mirrors.openanolis.org/inclavare-containers/alinux2-repo && \
       sudo rpm --import https://mirrors.openanolis.org/inclavare-containers/alinux2-repo/RPM-GPG-KEY-rpm-sign && \
       sudo yum install -y rats-tls-sgx

3. Run the following command to check whether rats-tls-sgx is installed:

   ls /usr/share/rats-tls/samples

   The following command output indicates that rats-tls-sgx is installed:

   rats-tls-client  rats-tls-server  sgx_stub_enclave.signed.so

Step 3: Install and start Verdictd

1. Use a terminal to connect to the instance that uses a CentOS 8.2 public image.

   For more information, see Connection method overview.

2. Change the CentOS 8 repository address.

   CentOS 8 reached end of life (EOL). In accordance with Linux community rules, all content was removed from the following CentOS 8 repository address: http://mirror.centos.org/centos/8/. If you continue to use the default CentOS 8 repository on Alibaba Cloud, an error is reported. To use specific installation packages of CentOS 8, change the CentOS 8 repository address. For more information, see Change CentOS 8 repository addresses. .

3. Run the following commands to install the SGX Platform Software (PSW).

   Verdictd must rely on the dynamic libraries provided by the SGX PSW to verify the TLS certificate that is embedded with SGX information. For more information, see SGX PSW.

   sudo yum install -y yum-utils && \
       sudo wget -c https://download.01.org/intel-sgx/sgx-linux/2.13/distro/centos8.2-server/sgx_rpm_local_repo.tgz && \
       sudo tar xzf sgx_rpm_local_repo.tgz && \
       sudo yum-config-manager --add-repo sgx_rpm_local_repo && \
       sudo yum makecache && rm -f sgx_rpm_local_repo.tgz && \
       sudo yum install --nogpgcheck -y libsgx-dcap-quote-verify \
       libsgx-dcap-default-qpl libsgx-dcap-ql \
       libsgx-uae-service

4. Configure the public URL of Alibaba Cloud PCCS.

   The Alibaba Cloud SGX remote attestation service is deployed on a per-region basis. For optimal stability, we recommend that you access this service in the region where the vSGX instance is deployed. You must manually modify the /etc/sgx_default_qcnl.conf file to adapt to the Alibaba Cloud SGX remote attestation service that is deployed in the region where the vSGX instance is deployed.

   Note

   The Alibaba Cloud SGX remote attestation service is supported only in regions inside the Chinese mainland. For more information about regions, see Regions and zones.

   • If a public IP address is assigned to the vSGX instance, you must modify /etc/sgx_default_qcnl.conf as described in the following code. You can use the vi or vim editor to modify the file. The sample command for using the vi editor is sudo vi /etc/sgx_default_qcnl.conf. The sample command for using the vim editor is sudo vi /etc/sgx_default_qcnl.conf.

     # PCCS server address
     PCCS_URL=https://sgx-dcap-server.<Region-ID>.aliyuncs.com/sgx/certification/v3/
     # To accept insecure HTTPS cert, set this option to FALSE
     USE_SECURE_CERT=TRUE

     <Region-ID> is a variable. You must replace it with the ID of the region where the vSGX instance is deployed. For example, if the instance is deployed in the China (Hangzhou) region, you must replace <Region-ID> with cn-hangzhou.

   • If the vSGX instance in a VPC has only internal IP addresses, you must modify /etc/sgx_default_qcnl.conf as described in the following code. You can use the vi or vim editor to modify the file. The sample command for using the vi editor is sudo vi /etc/sgx_default_qcnl.conf. The sample command for using the vim editor is sudo vi /etc/sgx_default_qcnl.conf.

     # PCCS server address
     PCCS_URL=https://sgx-dcap-server-vpc.<Region-ID>.aliyuncs.com/sgx/certification/v3/
     # To accept insecure HTTPS cert, set this option to FALSE
     USE_SECURE_CERT=TRUE

     <Region-ID> is a variable. You must replace it with the ID of the region where the vSGX instance is deployed. For example, if the instance is deployed in the China (Hangzhou) region, you must replace <Region-ID> with cn-hangzhou.

5. Run the following command to install the Verdictd software stack.

   Verdictd uses the library files of rats-tls-host to verify the TLS certificate that is embedded with SGX information.

   sudo yum-config-manager --add-repo https://mirrors.openanolis.cn/inclavare-containers/rpm-repo/ && \
       sudo rpm --import https://mirrors.openanolis.cn/inclavare-containers/rpm-repo/RPM-GPG-KEY-rpm-sign && \
       sudo yum install -y rats-tls-host verdictd

6. Run the following command to check whether Verdictd is installed:

   which verdictd verdict

   The following command output indicates that Verdictd is installed:

   /usr/local/bin/verdictd
   /usr/local/bin/verdict

7. Run the following command to check whether rats-tls-host is installed:

   ls /usr/share/rats-tls/samples

   The following command output indicates that rats-tls-host is installed:

   rats-tls-client  rats-tls-server

8. Run the following command to start Verdictd:

   verdictd --client-api 127.0.0.1:10001 --listen 0.0.0.0:1111 --mutual --attester nullattester --verifier sgx_ecdsa --tls openssl --crypto openssl

   Note

   After Verdictd is started, Verdictd listens on 0.0.0.0:1111 for remote attestation requests from rats-tls-client and listens on 127.0.0.1:10001 for configuration policy requests from Verdict.

   In the subsequent steps, you must make sure that Verdictd continues to run and the ECS instance remains connected.

Step 4: Configure reference values for remote attestation

1. Create a new terminal to connect to the ECS instance that uses a CentOS 8.2 public image.

   For more information, see Connection method overview.

   Note

   When you create the new terminal, make sure that the existing terminal on which Verdictd is started is not closed or disconnected. Two terminals are connected to the ECS instance that uses a CentOS 8.2 public image.

2. Use Verdict to configure the reference values of SGX Open Policy Agent (OPA) for Verdictd so that only specific SGX enclaves can be started.

   OPA is an open source, general-purpose policy engine that enables unified policy enforcement across the entire stack. For more information, visit OPA.

   1. Run the following command to export the SGX OPA reference value file:

      verdict --client-api 127.0.0.1:10001 --export-opa-reference sgxData

      The sgxData file contains the following content:

      cat sgxData
      {
          "mrEnclave": [],
          "mrSigner": [],
          "productId": 0,
          "svn": 0
      }

      Parameters in the sgxData reference value file:

      • mrEnclave: The mrEnclave value of an SGX enclave is added to the mrEnclave reference value array.

      • mrSigner: The mrSigner value of an SGX enclave is added to the mrSigner reference value array.

      • productId: This parameter ensures that the productId value of an SGX enclave is greater than or equal to the reference value of productId.

      • svn: This parameter ensures that the security version number (SVN) of an SGX enclave is greater than or equal to the reference value of svn.

   2. Obtain SGX OPA reference values.

      In actual business environments, you deploy applications in the cloud. Therefore, you must obtain reference values of the applications based on your trusted applications. The following steps show how to obtain the SGX OPA reference values. In this example, the reference values of the sgx_stub_enclave.signed.so SGX application that is deployed on the cloud are obtained.

      1. Run one of the following commands to download the application that is deployed on the cloud.

         • Run the following command if the instance runs the Alibaba Cloud Linux 3.2104 64-bit (UEFI) operating system:

           wget https://mirrors.openanolis.cn/inclavare-containers/alinux3-repo/rats-tls-sgx-0.6.5-1.al8.x86_64.rpm

         • Run the following command if the instance runs the Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI) operating system:

           wget https://mirrors.openanolis.cn/inclavare-containers/alinux2-repo/rats-tls-sgx-0.6.5-1.al7.x86_64.rpm

      2. Run one of the following commands to decompress the downloaded RPM package.

         • Run the following command if the instance runs the Alibaba Cloud Linux 3.2104 64-bit (UEFI) operating system:

           rpm2cpio rats-tls-sgx-0.6.5-1.al8.x86_64.rpm | cpio -div

         • Run the following command if the instance runs the Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI) operating system:

           rpm2cpio rats-tls-sgx-0.6.5-1.al7.x86_64.rpm | cpio -div

      3. Run the following commands to download and install sgx_sign:

         export SGX_SDK_VERSION=2.14 && \
             export SGX_SDK_RELEASE_NUMBER=2.14.100.2 && \
             sudo wget -c https://download.01.org/intel-sgx/sgx-linux/$SGX_SDK_VERSION/distro/centos8.2-server/sgx_linux_x64_sdk_$SGX_SDK_RELEASE_NUMBER.bin && \
             sudo chmod +x sgx_linux_x64_sdk_$SGX_SDK_RELEASE_NUMBER.bin && \
             echo -e 'no\n/opt/intel\n' | sudo ./sgx_linux_x64_sdk_$SGX_SDK_RELEASE_NUMBER.bin && \
             sudo rm -f sgx_linux_x64_sdk_$SGX_SDK_RELEASE_NUMBER.bin

      4. Run the following command to use sgx_sign to dump the metadata file and signature structure (SIGSTRUCT) of the enclave:

         /opt/intel/sgxsdk/bin/x64/sgx_sign dump -enclave ./usr/share/rats-tls/samples/sgx_stub_enclave.signed.so -dumpfile metadata.txt -cssfile sigstruct.bin

      5. Run the following command to obtain the Base64-encoded mrEnclave value:

         dd skip=960 count=32 if=sigstruct.bin of=mrenclave.binary bs=1 && \
             cat mrenclave.binary | base64

         • A command output similar to the following one is returned if the instance runs the Alibaba Cloud Linux 3.2104 64-bit (UEFI) operating system:

           zqs/chEBBNrInHlSCDO+0eV/pnlAB5qieF3M0hKfekA=

         • A command output similar to the following one is returned if the instance runs the Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI) operating system:

           +PLc9QPrEZ6ad9e3BgihgOINOVPYvIj0v/ixt2Kbwyw=

      6. Run the following command to obtain the Base64-encoded mrSigner value:

         grep -A 2 -i "mrsigner->value" metadata.txt | awk -F ":" 'NR==2,NR==3 {print $1}' | xargs echo -n | sed 's/[[:space:]]//g;s/0x//g' | xxd -r -p | base64

         A command output similar to the following one is returned:

         g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4=

      7. Run the following commands to obtain the productId value:

         dd skip=1024 count=2 if=sigstruct.bin of=productId.binary bs=1 && \
             od -An -tx2 productId.binary

         A command output similar to the following one is returned for rats-tls-sgx-0.6.5-1.al8.x86_64.rpm:

         0000

      8. Run the following commands to obtain the svn value:

         dd skip=1026 count=2 if=sigstruct.bin of=svn.binary bs=1 && \
             od -An -tx2 svn.binary

         A command output similar to the following one is returned for rats-tls-sgx-0.6.5-1.al8.x86_64.rpm:

         0000

   3. Run one of the following commands to generate an evidence file based on the SGX OPA reference values.

      Generate an evidence file based on the obtained SGX OPA reference values. The evidence file is a JSON file that contains the information of stub_enclave evidence. It is used to test whether the updated sgxData reference value file can work as expected.

      • Run the following command if the instance runs the Alibaba Cloud Linux 3.2104 64-bit (UEFI) operating system:

        cat <<- EOF >evidence
        {
            "mrEnclave": "zqs/chEBBNrInHlSCDO+0eV/pnlAB5qieF3M0hKfekA=",
            "mrSigner": "g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4=",
            "productId": 0,
            "svn": 0
        }
        EOF

      • Run the following command if the instance runs the Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI) operating system:

        cat <<- EOF >evidence
        {
            "mrEnclave": "+PLc9QPrEZ6ad9e3BgihgOINOVPYvIj0v/ixt2Kbwyw=",
            "mrSigner": "g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4=",
            "productId": 0,
            "svn": 0
        }
        EOF

   4. Run one of the following commands to update the local file that contains the SGX OPA reference values.

      Update the sgxData file based on the obtained SGX OPA reference values.

      • Run the following command if the instance runs the Alibaba Cloud Linux 3.2104 64-bit (UEFI) operating system:

        cat <<- EOF >./sgxData
        {
            "mrEnclave": ["zqs/chEBBNrInHlSCDO+0eV/pnlAB5qieF3M0hKfekA="],
            "mrSigner": ["g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4="],
            "productId": 0,
            "svn": 0
        }
        EOF

      • Run the following command if the instance runs the Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI) operating system:

        cat <<- EOF >./sgxData
        {
            "mrEnclave": ["+PLc9QPrEZ6ad9e3BgihgOINOVPYvIj0v/ixt2Kbwyw="],
            "mrSigner": ["g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4="],
            "productId": 0,
            "svn": 0
        }
        EOF

   5. (Optional) Run the following command to test the updated local reference value file.

      If you want to test the updated local reference value file, such as the format of the new reference value file and whether the added reference values can work as expected, perform this step.

      verdict --client-api 127.0.0.1:10001 --test-opa-local-reference sgxPolicy.rego ./sgxData ./evidence

      Parameters in the command:

      • --test-opa-local-reference: the test of the local reference value file.

      • sgxPolicy.rego: the name of the OPA policy file used for the test. This parameter must be set to sgxPolicy.rego in the command.

      • ./sgxData: the file to be tested, which is the updated reference value file.

      • ./evidence: a JSON file that contains the stub_enclave evidence information. This file is generated by running a shell script. This file is used to test whether the updated sgxData reference value file can work as expected.

      If the test succeeds, a command output similar to the following one is returned.

      • Alibaba Cloud Linux 3.2104 64-bit (UEFI):

        

      • Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI):

   6. Run the following command to upload the reference value file.

      After the reference value file is updated, you can upload the latest local reference value file to Verdictd.

      verdict --client-api 127.0.0.1:10001 --set-opa-reference sgxData ./sgxData

      A command output similar to the following one is returned.You can run the cat /opt/verdictd/opa/sgxData command to check whether the reference value file is updated. Sample command output:

      • Alibaba Cloud Linux 3.2104 64-bit (UEFI):

        cat /opt/verdictd/opa/sgxData
        {
            "mrEnclave": ["zqs/chEBBNrInHlSCDO+0eV/pnlAB5qieF3M0hKfekA="],
            "mrSigner": ["g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4="],
            "productId": 0,
            "svn": 0
        }

      • Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI):

        cat /opt/verdictd/opa/sgxData
        {
            "mrEnclave": ["+PLc9QPrEZ6ad9e3BgihgOINOVPYvIj0v/ixt2Kbwyw="],
            "mrSigner": ["g9cZ533qyhRw9rr2Kk13QwPImdtpAg+ccO4d/AjHzp4="],
            "productId": 0,
            "svn": 0
        }

Step 5: Verify whether remote attestation is implemented

1. Connect to the security-enhanced instance that uses an Alibaba Cloud Linux (UEFI) public image.

   For more information, see Connection method overview.

   Note

   Make sure that the terminal that has Verdictd started is not closed or disconnected.

2. Run the following commands to allow rats-tls-client to initiate a remote attestation request to Verdictd to check whether the workloads in the cloud are running in a hardware-based TEE:

   cd /usr/share/rats-tls/samples/ && \
   sudo ./rats-tls-client -a sgx_ecdsa -v nullverifier -t openssl -c openssl -i <Public IP address of the instance on which Verdictd is deployed> -p 1111 --mutual --verdictd

   <Public IP address of the instance on which Verdictd is deployed> is a variable. Replace it with the public IP address of the instance on which Verdictd is deployed.

   The following command output indicates that remote attestation is implemented.