Elastic GPU Service:Build measured LLM inference environment on a heterogeneous confidential computing instance

Why confidential computing for LLM inference

As Large Language Models (LLMs) become more widespread, securely managing models and user data has become critically important. Heterogeneous confidential computing instances (gn8v-tee) extend Intel TDX confidential computing by integrating a GPU into the TEE, protecting data in transit between the CPU and GPU and data processed within the GPU.

This solution enforces two core security properties:

• Confidentiality -- Model and user data stay within the instance's confidential security boundary. No plaintext data is exposed externally.

• Integrity -- All components in the inference stack (framework, model files, and web interface) are tamper-proof and auditable by third parties.

Measurement and remote attestation

The solution relies on two Intel TDX mechanisms:

Measurement

During startup, the TDX module records the state of the Trusted Domain (TD) guest in two register sets:

• MRTD (Build Time Measurement Register) -- Captures the initial configuration and boot image of the guest VM.

• RTMRs (Runtime Measurement Registers) -- Record the kernel, kernel command line, and runtime services as they load. In this solution, the Ollama service, DeepSeek model, and Open WebUI framework are reflected in the RTMRs.

Remote attestation

Remote attestation lets a client obtain cryptographic proof that the inference environment is intact:

1. The client requests a remote attestation report from Open WebUI.

2. Open WebUI obtains a hardware-signed TD Quote from the CPU. The Quote includes the MRTD and RTMRs, reflecting the current integrity state.

3. The client submits the Quote to a trusted attestation service, which validates the digital signature, certificate chain, and security policy.

Step 1: Create a heterogeneous confidential computing instance

aliyun ecs RunInstances \
  --RegionId cn-beijing \
  --ZoneId cn-beijing-l \
  --SystemDisk.Category cloud_essd \
  --ImageId 'aliyun_3_x64_20G_alibase_20250117.vhd' \
  --InstanceType 'ecs.gn8v-tee.4xlarge' \
  --SecurityGroupId '<SECURITY_GROUP_ID>' \
  --VSwitchId '<VSWITCH_ID>' \
  --KeyPairName <KEY_PAIR_NAME>

Step 2: Build the TDX remote attestation environment

TDX Report is a hardware-generated data structure representing the identity of a TDX-enabled instance. It includes attributes, RTMRs, and TCB security-version number (SVN), protected by cryptographic integrity. For more details, see Intel TDX Module specification.

1. Import the YUM repository for Alibaba Cloud confidential computing. Repository URL formats: Replace [Region-ID] with the actual region ID. Currently, only the China (Beijing) region supports TDX instances. To use the internal URL:

   • Public: https://enclave-[Region-ID].oss-[Region-ID].aliyuncs.com/repo/alinux/enclave-expr.repo

   • Internal: https://enclave-[Region-ID].oss-[Region-ID]-internal.aliyuncs.com/repo/alinux/enclave-expr.repo

      region="cn-beijing"
   
      sudo yum install -y yum-utils
      sudo yum-config-manager --add-repo https://enclave-${region}.oss-${region}-internal.aliyuncs.com/repo/alinux/enclave-expr.repo

2. Install build tools and sample code.

      sudo yum groupinstall -y "Development Tools"
      sudo yum install -y sgxsdk libtdx-attest-devel

3. Configure the remote attestation service. Set the PCCS_URL in /etc/sgx_default_qcnl.conf to point to the Data Center Attestation Primitives (DCAP) service in the China (Beijing) region:

      sudo sed -i.$(date "+%m%d%y") 's|PCCS_URL=.*|PCCS_URL=https://sgx-dcap-server.cn-beijing.aliyuncs.com/sgx/certification/v4/|' /etc/sgx_default_qcnl.conf

Step 3: Install Ollama

1. Connect to the instance using Workbench.

2. Run the official installation script:

   Note

   If the installation fails due to network issues, see the Ollama Linux installation guide for alternative methods.

      curl -fsSL https://ollama.com/install.sh | sh

Step 4: Download and run DeepSeek-R1

Use tmux to maintain the session during the large model download.

1. Install tmux.

      sudo yum install -y tmux

2. Download and run DeepSeek-R1. When the download completes and the model starts, the output looks like: Enter /bye to exit the interactive session.

      # Create a tmux session
      tmux new -s "run-deepseek"
      # Download and run the model (~43 GB)
      ollama run deepseek-r1:70b

      ......
      verifying sha256 digest
      writing manifest
      success
      >>>
      >>> Send a message (/? for help)

3. (Optional) Reconnect to the tmux session after a disconnection:

      tmux attach -t run-deepseek

Step 5: Compile Open WebUI with TDX measurement support

To enable TDX security measurement, download the TDX plugin from CCZoo and compile Open WebUI from source.

Step 6: Run Open WebUI and verify TDX attestation