Container Service for Kubernetes:Secure confidential container environments with remote attestation

Pre-launch environment verification

Before any image is pulled or container started, the Attestation Agent inside the confidential VM automatically generates hardware evidence and submits it to the Trustee. Only after the Trustee confirms the evidence is valid does the pod proceed.

1. An administrator submits a pod manifest to the API server. The kubelet requests a pod sandbox from containerd via the Container Runtime Interface (CRI). containerd checks runtimeClassName and invokes kata-runtime.

2. Kata Remote calls the Alibaba Cloud OpenAPI to launch an ECS TDX confidential VM as the pod.

3. The Attestation Agent inside the VM generates evidence — including hardware signatures — and sends it to the Trustee.

4. After the Trustee validates the evidence, the pod pulls the application image and starts the container.

Runtime remote attestation

Applications running inside a PeerPod can proactively prove the security of their environment to external services, establishing a chain of trust across systems.

1. The application calls the local API at http://127.0.0.1:8006/aa/evidence to request a real-time attestation report.

2. The application sends this evidence to an external service. That service forwards it to its own Trustee for validation, confirming the sender is running inside a protected confidential environment.

Step 1: Configure the Trustee

On the Trustee server, configure two policies: one that controls which container images are allowed, and one that defines what constitutes a trusted hardware environment.

1. Create the container security launch policy. This policy file is stored as a confidential resource and is read during pod startup to trigger the remote attestation flow.

   # Create the policy directory
   mkdir -p /opt/trustee/kbs/repository/default/container-policy
   
   # Write a policy that allows all images (for testing only — tighten this before going to production)
   cat << EOF > /opt/trustee/kbs/repository/default/container-policy/insecure-accept-all
   {
       "default": [{"type": "insecureAcceptAnything"}],
       "transports": {}
   }
   EOF

2. Create the remote attestation policy. This Rego policy defines what counts as a trusted environment. The policy below requires Intel TDX hardware, identified by TEE type code 81000000 and Intel's vendor UUID.

   cat << EOF > /opt/trustee/attestation-service/policies/opa/default.rego
   package policy
   
   import rego.v1
   
   default executables := 32
   default hardware := 97
   default configuration := 32
   default file_system := 32
   
   ##### ECS TDX PeerPod
   executables := 3 if {
   input.tdx
   }
   
   hardware := 2 if {
   # Verify that the evidence is of the Intel TDX type
   input.tdx.quote.header.tee_type == "81000000"
   # Verify the vendor ID
   input.tdx.quote.header.vendor_id == "939a7233f79c4ca9940a0db3957f0607"
   }
   
   configuration := 2 if {
   input.tdx
   }
   
   file_system := 2 if {
   input.tdx
   }
   EOF

3. Restart the Trustee to apply the policies.

   service as restart

Step 2: Deploy a pod with remote attestation

1. Generate the initdata configuration. initdata passes the Trustee endpoint address to the PeerPod so the Attestation Agent knows where to send evidence.

   # Replace with your actual Trustee IP address
   TRUSTEE_SERVICE_ENDPOINT=1.2.3.4
   
   cat <<EOF > initdata.toml
   version = "0.1.0"
   algorithm = "sha256"
   [data]
   "aa.toml" = '''
   [eventlog_config]
   enable_eventlog = true
   [token_configs]
   [token_configs.kbs]
   url = "http://${TRUSTEE_SERVICE_ENDPOINT}:8080"
   '''
   "cdh.toml" = '''
   [kbc]
   name = "cc_kbc"
   url = "http://${TRUSTEE_SERVICE_ENDPOINT}:8080"
   [image]
   image_security_policy_uri = "kbs:///default/container-policy/insecure-accept-all"
   '''
   EOF
   
   # Compress and Base64-encode the initdata
   initdata=$(cat initdata.toml | gzip | base64 -w0)

   Important

   initdata.toml contains your Trustee endpoint address. Strictly control how this file is generated, distributed, and stored.

   Run echo $initdata to verify the encoded output.

2. Create the pod manifest. Write the encoded initdata string to the pod's annotations field.

   cat << EOF > pod.yaml
   apiVersion: v1
   kind: Pod
   metadata:
     name: pod-caa-demo
     annotations:
       io.katacontainers.config.hypervisor.cc_init_data: ${initdata}
   spec:
     runtimeClassName: kata-remote
     containers:
       - image: alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com/alinux3/alinux3:latest
         name: hello
         command: ["sh", "-c", "echo hello && sleep infinity"]
   EOF

3. Deploy the pod.

   kubectl apply -f pod.yaml

   Run kubectl get po pod-caa-demo to check the pod status.

4. Verify attestation on the Trustee server.

   journalctl -u as -f

   The message Tdx Verifier/endorsement check passed confirms the attestation succeeded and the pod is running in a trusted environment.

Step 1: Get remote attestation evidence from within the container

(Optional) Add dynamic measurements

Dynamic measurements let an application embed custom runtime information into the attestation report. This is useful for auditing and compliance: standard remote attestation verifies that the environment (the "safe") is intact, while dynamic measurements further prove that the specific application state (the "contents of the safe") matches expectations.

From inside the pod container, send a POST request to http://127.0.0.1:8006/aa/aael. The request body fields domain, operation, and content are customizable printable strings (no spaces allowed). The data is saved to the Eventlog of the TDX pod and bound to the final attestation report.

# Run this command inside the container
curl -X POST http://127.0.0.1:8006/aa/aael \
     -H "Content-Type: application/json" \
     -d '{"domain":"test","operation":"test","content":"test"}'

Get the attestation evidence

From inside the pod container, send a GET request to retrieve the attestation evidence for the current TEE.

curl 127.0.0.1:8006/aa/evidence?runtime_data=AAAA

The runtime_data parameter accepts a Base64-encoded nonce provided by the verifier to protect against replay attacks.

The command outputs a JSON string. Copy the entire string (from { to }) and save it for the next steps.

Step 2: Prepare the verification environment

On the verifier server:

1. (Optional) Install the Trustee if not already installed.

   yum install trustee -y

2. Create the client-side validation policy at /opt/trustee/attestation-service/policies/opa/client.rego. This policy uses the same scoring rules as the default policy.

   cat <<EOF > /opt/trustee/attestation-service/policies/opa/client.rego
   
   # Policy to verify evidence from the confidential container (TEE)
   package policy
   import rego.v1
   
   # --- Default scores ---
   default executables := 32
   default hardware := 97
   default configuration := 32
   default file_system := 32
   
   # --- Validation rules for Alibaba Cloud ECS TDX PeerPods ---
   executables := 3 if {
       input.tdx
   }
   
   hardware := 2 if {
       # TEE type must be the Intel TDX identifier
       input.tdx.quote.header.tee_type == "81000000"
       # Vendor ID must be the official Intel UUID
       input.tdx.quote.header.vendor_id == "939a7233f79c4ca9940a0db3957f0607"
   }
   
   configuration := 2 if {
       input.tdx
   }
   
   file_system := 2 if {
       input.tdx
   }
   EOF

3. Restart the Trustee to apply the policy.

   service as-restful restart

Step 3: Submit the evidence for verification

1. Save the attestation evidence to a file to avoid issues with special characters.

   cat > evidence.json

   Paste the JSON string, then save and exit.

2. Encode the evidence and build the request body.

   EVIDENCE=$(cat evidence.json)
   BASE64EVIDENCE=$(echo ${EVIDENCE} | base64 -w 0 2>/dev/null | tr '+/' '-_' | tr -d '=' )
   
   cat << EOF > request.json
   {
       "verification_requests": [
           {
               "tee": "tdx",
               "evidence": "${BASE64EVIDENCE}"
           }
       ],
       "policy_ids": ["client"]
   }
   EOF

3. Send a POST request to the Trustee's attestation endpoint on port 50005.

   curl -k -X POST http://127.0.0.1:50005/attestation \
        -i \
        -H 'Content-Type: application/json' \
        -d @request.json

   The response contains a token field with a JSON Web Token (JWT). Copy this token.

Step 4: Parse and evaluate the result

1. Store the JWT.

   # Replace <...> with the actual JWT string (without quotes)
   TOKEN='<Paste the JWT token here>'

2. Extract the trustworthiness scores.

   echo $TOKEN | cut -d'.' -f2 | base64 -d | jq '.submods.cpu0."ear.trustworthiness-vector"'

   Expected output:

   base64: invalid input
   {
     "configuration": 2,
     "executables": 3,
     "file-system": 2,
     "hardware": 2
   }

   The base64: invalid input message is harmless and can be ignored. The scores confirm the container is running in a compliant TDX environment.