Build a TDX confidential computing environment and verify remote attestation - Elastic Compute Service

This topic describes how to build a Trust Domain Extensions (TDX) confidential computing environment on an Intel^® TDX-enabled Elastic Compute Service (ECS) instance and how to verify the TDX feature on the instance.

Background information

Intel^® TDX is a CPU hardware-based technology that provides hardware-assisted isolation and encryption for ECS instances to protect runtime data, such as CPU registers, memory data, and interrupt injections. TDX helps achieve high levels of data privacy and mitigate risks that are associated with unauthorized access to running processes or sensitive data that is being processed. For more information about Intel^® TDX, see Intel® Trust Domain Extensions (Intel® TDX).

Intel^® TDX provides default out-of-the-box protection for your ECS instances and applications. You can migrate applications to a TDX-enabled instance without the need to modify the application code.

Create a TDX-enabled instance

Create a TDX-enabled instance in the ECS console

The steps for creating a TDX-enabled instance in the ECS console are similar to the steps for creating a regular instance. Take note of specific options of the TDX-enabled instance. The following section describes specific configurations of the TDX-enabled instance. For information about other common configurations, see Create an instance on the Custom Launch tab.

Log on to the ECS console.
In the left-side navigation pane, choose Instances & Images > Instances.
In the top navigation bar, select the region and resource group to which the resource belongs.

Click Create Instance and configure the parameters described in the following table.

Parameter	Description
Network and Zone	Select Beijing Zone I.
Instance Type	Select ecs.g8i.xlarge or a larger instance type.
Image	Select Alibaba Cloud Linux 3.2104 LTS 64-bit (UEFI) as the image version and then select the Confidential VM check box.

Create the instance as promoted.

Create a TDX-enabled instance by calling an API operation or using Alibaba Cloud CLI

You can call the RunInstances operation or use Alibaba Cloud CLI to create a TDX-enabled ECS instance. The following table describes the related parameters.

Parameter	Description	Example
RegionId	Select the China (Beijing) region.	cn-beijing
ZoneId	Select Beijing Zone I.	cn-beijing-j
InstanceType	Select ecs.g8i.xlarge or a larger instance type.	ecs.g8i.2xlarge
ImageId	Specify the ID of a TDX-capable image. Only Alibaba Cloud Linux 3.2104 LTS 64-bit Unified Extensible Firmware Interface (UEFI) images whose kernel version is 5.10.134-16.al8.x86_64 or later are supported.	aliyun_3_9_x64_20G_uefi_alibase_20231219.vhd
SecurityOptions.ConfidentialComputingMode	Specify the confidential computing mode.	TDX

Example for creating a TDX-enabled instance by using Alibaba Cloud CLI

aliyun ecs RunInstances \
  --SecurityOptions.ConfidentialComputingMode TDX \
  --Region cn-beijing \
  --ZoneId cn-hangzhou-i \
  --SystemDisk.Category cloud_essd \
  --ImageId 'aliyun_3_9_x64_20G_uefi_alibase_20231219.vhd' \
  --InstanceType 'ecs.g8i.2xlarge' \
  --SecurityGroupId 'sg-[SecurityGroupId]' \
  --VSwitchId 'vsw-[VSwitchID]' \
  --KeyPairName [KEY_PAIR_NAME] \

Procedure

Step 1: Check whether TDX is enabled

Before you build a TDX confidential computing environment on an ECS instance, make sure that TDX is enabled on the instance.

Check whether TDX is enabled.
```
lscpu |grep -i tdx_guest
```
The following command output indicates that TDX is enabled.
Check whether the TDX driver is installed.
```
ls -l /dev/tdx_guest
```
The following command output indicates that the TDX driver is installed.

Step 2: Build a TDX confidential computing environment

Note

In most cases, you can migrate applications from a non-TDX environment to a TDX environment without the need to modify the application code. You can use TDX-enabled instances in the same manner that you use TDX-disabled instances. We recommend that you use the features that are described in the following sections to maximize TDX protection for your ECS instances.

TDX Report is a data structure that is generated by CPU hardware to represent the identities of TDX-enabled instances. TDX Report includes critical information about TDX-enabled instances, such as attributes (ATTRIBUTES), runtime-extendable measurement registers (RTMR), and trusted-computing base (TCB) security-version number (SVN). TDX Report uses cryptographic methods to ensure information integrity. For more information, see Intel TDX Module.

Import the Yellowdog Updater Modified (YUM) software repository for Alibaba Cloud confidential computing.
- The public URLs of the YUM software repository are in the following format: https://enclave-[Region-ID].oss-[Region-ID].aliyuncs.com/repo/alinux/enclave-expr.repo.
- The internal URLs of the YUM software repository are in the following format: https://enclave-[Region-ID].oss-[Region-ID]-internal.aliyuncs.com/repo/alinux/enclave-expr.repo.
Replace [Region-ID] in the preceding URLs with the actual region ID of the TDX-enabled instance. For example, to import the YUM software repository from an internal URL that is specific to the China (Beijing) region, run the following commands:
```
region="cn-beijing"

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://enclave-${region}.oss-${region}-internal.aliyuncs.com/repo/alinux/enclave-expr.repo
```

Install a compilation tool and sample code.

sudo yum groupinstall -y "Development Tools"
sudo yum install -y sgxsdk libtdx-attest-devel

Compile the sample TDXReportParse code.
1. Access the TDXReportParse directory.
```
cd /opt/alibaba/teesdk/intel/sgxsdk/SampleCode/TDXReportParse
```
2. Compile TDXReportParse.
```
sudo make
```
Run the compiled executable file.
```
sudo ./tdx_report_parse
```
The following command output is returned. If the TDX-enabled instance is under protection, the instance is in a non-debugging state and NO_DEBUG is displayed on the attributes line.

Verify the TDX feature by using the remote attestation service

In an Alibaba Cloud TDX-enabled encryption environment, the remote attestation service can be used to verify the trustworthiness of the platform and the integrity and confidentiality of the code running on the platform. For more information, see Remote attestation service.

Platform indicates the hardware platform and virtualization software stacks used by Alibaba Cloud.
Code running on the platform indicates an operating system such as Alibaba Cloud Linux and applications such as NGINX and Java running in a TDX-enabled environment.

Install dependency packages required for TDX remote attestation.

sudo yum install -y gcc gcc-c++ make openssl-devel git jq
sudo yum install -y tdx-quote-generation-sample tee-appraisal-tool libsgx-dcap-ql-devel libsgx-dcap-quote-verify-devel libsgx-dcap-default-qpl-devel tdx-quote-verification-sample

Configure the Alibaba Cloud TDX remote attestation service.
Specify the PCCS_URL parameter in the /etc/sgx_default_qcnl.conf file. For example, specify the PCCS_URL parameter to point to the Distributed Cooperative Access Points (DCAP) service in the China (Beijing) region.
```
sudo sed -i.$(date "+%m%d%y") 's|PCCS_URL=.*|PCCS_URL=https://sgx-dcap-server.cn-beijing.aliyuncs.com/sgx/certification/v4/|' /etc/sgx_default_qcnl.conf
```
Access the tdx-quote-generation-sample directory and compile the tdx-quote-generation-sample file.
```
cd /opt/alibaba/tdx-quote-generation-sample
sudo make
```
Perform one of the following operations to enable the system to generate a remote attestation report, which is also referred to as a quote.
- Enable the system to randomly generate report_data and issue a quote.
```
sudo ./app
```
- Specify report_data and issue a quote.
```
sudo ./app -d <report_data_in_hex>
```
If the outputs similar to the outputs shown in the following figure appear, the remote attestation report (quote) is generated.

Verify the remote attestation report (quote).

Configure and sign an appraisal policy.
Important
We recommend that you use a known and secure environment, but do not use the production environment to generate the appraisal policy as needed.
You can describe the appraisal policy in JSON format.
- For example, you can configure the following appraisal policy to verify whether your TDX-enabled ECS instance runs in an encrypted and non-debugging state, which indicates that the instance is under protection.
- You can also configure additional parameters in the appraisal policy to verify the integrity of the operating system and applications. For more information about the appraisal policy, see Intel DCAP Appraisal Engine Developer Guide.
```
{
    "policy_array":[
        {
            "environment":{
                "class_id":"45b734fc-aa4e-4c3d-ad28-e43d08880e68",
                "description":"Application TD TCB 1.5"
            },
            "reference":{
                "tdx_attributes":"0000000010000000",
                "#NOTE": "0000000010000000 means for NO_DEBUG and SPTE_VE_DISABLE"
            }
        }
    ]
}
```
You can use the default policy in the /opt/alibaba/tdx-quote-verification-sample/Policies//tenant_td_policy.json file or compile a custom policy to use a policy token for subsequent verification. The policy token can be transferred to any environment that generates a remote attestation report for subsequent verification.
```
cd /opt/alibaba/tdx-quote-verification-sample
sudo make Policies/tenant_td_policy.jwt
```
If the outputs similar to the outputs shown in the following figure appear, the policy token is generated.
Compile other components on which TDX remote attestation depends.
```
cd /opt/alibaba/tdx-quote-verification-sample
sudo make all
```
If the outputs similar to the outputs shown in the following figure appear, the dependent components are compiled.

Verify the remote attestation report (quote).

Important

Replace the value of the <path_to_quote> parameter in the following command with the actual quote directory. Example: /opt/alibaba/tdx-quote-generation-sample/quote.dat.

Based on the policy in Policies/tenant_td_policy.json, the quote generated for remote attestation is verified, and the corresponding verification results are converted from the JSON Web Token (JWT) format to the standard format.
```
./verifier -quote <path_to_quote>
```
The outputs similar to the outputs shown in the following figure appear.

Use RelyingParty to verify the identity of the JWT signature.

./verifier -quote <path_to_quote> |./relying_party -v |grep "json payload" |awk -F 'payload:' '{print $2}'|jq

The outputs similar to the outputs shown in the following figure appear. In the following example, only specific fields are displayed.

[
  {
    "result": {
      "appraisal_check_date": 1710400829000000000,
      "nonce": 502551065253582,
      "certification_data": [
        {
          "certification_data": {
            "qe_identity_issuer_chain": "LS0t...",
            "root_ca_crl": "MzA4...",
            "pck_crl": "LS0t...",
            "pck_crl_issuer_chain": "LS0t...",
            "tcb_info": "eyJ0...",
            "qe_identity": "eyJl...",
            "tcb_info_issuer_chain": "LS0t..."
          }
        }
      ],
      "overall_appraisal_result": 1,
      "appraised_reports": [
        {
          "appraisal_result": 1,
          "detailed_result": [
            {
              "td_mrownerconfig_check": true,
              "td_xfam_check": true,
              "td_mrservicetd_check": true,
              "td_attributes_check": true,
              "td_rtmr3_check": true,
              "td_mrtd_check": true,
              "td_mrowner_check": true,
              "td_rtmr0_check": true,
              "td_mrconfigid_check": true,
              "td_rtmr1_check": true,
              "td_rtmr2_check": true
            }
          ],
          "policy": {
            "environment": {
              "description": "Application TD TCB 1.5",
              "class_id": "45b734fc-aa4e-4c3d-ad28-e43d08880e68"
            },
            "signature": "-6C0-...",
            "reference": {
              ...
            },
            "signing_key": {
              "kty": "EC",
              "crv": "P-384",
              "alg": "ES384",
              "y": "CeW8...",
              "x": "NmSa..."
            }
          },
          "report": {
            "environment": {
              "Description": "Application TD TCB",
              "class_id": "45b734fc-aa4e-4c3d-ad28-e43d08880e68"
            },
            "measurement": {
              "tdx_mrownerconfig": "0000...",
              "tdx_mrservicetd": "3D03...",
              "tdx_xfam": "00000000000642E7",
              "tdx_mrtd": "0A40...",
              "tdx_mrowner": "0000...",
              "tdx_attributes": "0000000010000000",
              "tdx_mrconfigid": "0000...",
              "tdx_reportdata": "D98B...",
              "tdx_rtmr3": "0000...",
              "tdx_rtmr2": "0000...",
              "tdx_rtmr1": "6368...",
              "tdx_rtmr0": "D0FD..."
            }
          }
        },
        {
          "appraisal_result": 1,
          "detailed_result": [
            {
              "platform_provider_id_check": true,
              "sgx_types_check": true,
              "cached_keys_check": true,
              "smt_enabled_check": true,
              "accepted_tcb_level_date_tag_check": true,
              "advisory_ids_check": true,
              "expiration_date_check": true,
              "tcb_eval_num_check": true,
              "earliest_accepted_tcb_level_date_tag_check": true,
              "tcb_status_check": true,
              "dynamic_platform_check": true
            }
          ],
          "policy": {
            "environment": {
              "description": "Alibaba Cloud Evaluation Num Policy for TDX Platform",
              "class_id": "f708b97f-0fb2-4e6b-8b03-8a5bcd1221d3"
            },
            "signature": "l00p...",
            "reference": {
              "accepted_tcb_status": [
                "UpToDate"
              ],
              "allow_dynamic_plaform": true,
              "min_eval_num": 16
            },
            "signing_key": {
              "kty": "EC",
              "crv": "P-384",
              "alg": "ES384",
              "y": "7hlr...",
              "x": "OSbD..."
            }
          },
          "report": {
            "environment": {
              "description": "TDX Platform TCB",
              "class_id": "f708b97f-0fb2-4e6b-8b03-8a5bcd1221d3"
            },
            "measurement": {
              "earliest_issue_date": "2018-05-21T10:45:10Z",
              "is_cached_keys_policy": true,
              "fmspc": "90C06F000000",
              "is_smt_enabled": true,
              "earliest_expiration_date": "2024-04-02T10:22:51Z",
              "root_ca_crl_num": 1,
              "root_key_id": "9309...",
              "pck_crl_num": 1,
              "tcb_eval_num": 16,
              "latest_issue_date": "2024-03-13T15:45:02Z",
              "tcb_status": [
                "UpToDate"
              ],
              "tcb_level_date_tag": "2023-08-09T00:00:00Z",
              "is_dynamic_platform": true,
              "sgx_types": 1
            }
          }
        },
        {
          "appraisal_result": 1,
          "detailed_result": [
            {
              "td_qe_expiration_date_check": true,
              "td_qe_tcb_eval_num_check": true,
              "td_qe_earliest_accepted_tcb_level_date_tag_check": true,
              "td_qe_tcb_status_check": true,
              "td_qe_accepted_tcb_level_date_tag_check": true
            }
          ],
          "policy": {
            "environment": {
              "description": "Alibaba Cloud Num Policy for Verified TDQE",
              "class_id": "3769258c-75e6-4bc7-8d72-d2b0e224cad2"
            },
            "signature": "l00p...",
            "reference": {
              "accepted_tcb_status": [
                "UpToDate"
              ],
              "min_eval_num": 16
            },
            "signing_key": {
              "kty": "EC",
              "crv": "P-384",
              "alg": "ES384",
              "y": "7hlr...",
              "x": "OSbD..."
            }
          },
          "report": {
            "environment": {
              "Description": "RAW TDX QE Report",
              "class_id": "3769258c-75e6-4bc7-8d72-d2b0e224cad2"
            },
            "measurement": {
              "earliest_expiration_date": "2025-05-21T10:50:10Z",
              "earliest_issue_date": "2018-05-21T10:45:10Z",
              "root_key_id": "9309...",
              "tcb_eval_num": 16,
              "latest_issue_date": "2018-05-21T10:50:10Z",
              "tcb_status": [
                "UpToDate"
              ],
              "tcb_level_date_tag": "2023-08-09T00:00:00Z"
            }
          }
        }
      ]
    }
  }
]

Note

For simplicity, you can check the overall_appraisal_result field in appraisal_result to check whether the evidence complies with the appraisal policy.

./verifier -quote <path_to_quote> |./relying_party -v |grep "json payload" |awk -F 'payload:' '{print $2}'|jq '.[0].result.overall_appraisal_result'

In addition, the report_data parameter specified when the quote is generated is also reflected in the verification result of the remote attestation service. You can use the report_data value to implement business logic such as key exchange.
```
./verifier -quote <path_to_quote> |./relying_party -v -a|grep "json payload" |awk -F 'payload:' '{print $2}'|jq '.. | .tdx_reportdata? | select(. != null)'
```

Known issues and functionality limits

TDX-enabled instances provide CPU hardware-based encryption to protect CPU registers and memory states. As a result, the overall performance of the instances is affected.
You cannot restart TDX-enabled instances from within the instances due to the design limitations of RTMR, interrupt protection, and register state recovery. You cannot run the reboot command in the operating system of a TDX-enabled instance to restart the instance.
You cannot log on to a TDX-enabled instance by using Virtual Network Computing (VNC). You can obtain the logs that are generated during the runtime of the instance. For more information, see View system logs and screenshots or GetInstanceConsoleOutput.
Limits for images:
- Only UEFI images are supported.
- Windows images are not supported.
- Images whose kernel version is 5.10.134-16.al8.x86_64 or later are supported.
Elastic Remote Direct Memory Access (eRDMA) and accelerators are not supported.
Known issues about the Software Input Output Translation Lookaside Buffer (SWIOTLB) in the Linux kernel:
- The amount of available memory on TDX-enabled instances is smaller than the amount of available memory on TDX-disabled instances. TDX-enabled instances use a specific area, called SWIOTLB, in unencrypted memory for peripheral communication. By default, the SWIOTLB takes up 6% of the total available memory of a TDX-enabled instance and does not exceed 1 GiB in size.
- If you bind multiple elastic network interfaces (ENIs) to a TDX-enabled instance of a large instance type, the instance may freeze when memory allocation fails due to insufficient SWIOTLB in network interface controller (NIC) multi-queue scenarios. You can stop the instance and unbind ENIs from the instance in the ECS console to recover the instance.
- If your instance workload involves a large amount of I/O communication, the performance of the instance may degrade due to insufficient SWIOTLB. You can run the following command to check whether the preceding issue occurred:
```
dmesg| grep 'swiotlb buffer is full'
```
  Important
  Incorrect SWIOTLB-related parameter settings may cause your instance to fail to start. To facilitate data rollback when an error occurs, we recommend that you create snapshots for disks on the instance before you proceed. For more information, see Create a snapshot for a disk.
  If the preceding issue occurred, you can perform the following steps to improve the performance of the instance by increasing the size of the SWIOTLB:
  1. Open the /etc/default/grub file.
```
vim /etc/default/grub
```
  2. Append the swiotlb parameter to the GRUB_CMDLINE_LINUX field.
    For example, change GRUB_CMDLINE_LINUX to GRUB_CMDLINE_LINUX="... swiotlb=524288" to increase the size of the SWIOTLB to 1 GiB.
    Note
    You can calculate the value of the swiotlb parameter by using the following formula: Size of the SWIOTLB that you require in MiB × 512. For more information, see The kernel's command-line parameters.
  3. Update the grub.cfg file to make the preceding modifications take effect.
```
grub2-mkconfig -o /boot/efi/EFI/alinux/grub.cfg
```