Encrypt cloud disks in confidential instances - Elastic Compute Service

In confidential computing scenarios, data used by confidential computing applications is typically protected by register and memory encryption. However, the program code and the data it processes are often stored directly on cloud disks. In these confidential computing environments, you can implement data encryption inside the confidential instance using technologies such as LUKS or BitLocker. This encrypts data independently of the cloud service provider before the data leaves the security boundary of the confidential instance. This also lets you manage the encryption keys for the cloud disks. This topic describes how to encrypt cloud disks within a confidential computing instance to provide a higher level of security. An encrypted cloud disk in this context is called a confidential cloud disk.

Note

To encrypt disks on non-confidential computing instances, use the disk encryption feature provided by Alibaba Cloud Elastic Block Storage (EBS).

Encryption overview

How encryption works

Confidential cloud disks provide data encryption and protection at the block storage level based on the Linux LUKS2 technology. To create a confidential cloud disk from a cloud disk on a confidential instance, you must perform an initialization procedure and an open procedure.

Initialization procedure: First, identify the block device to initialize. This is usually a block device that corresponds to a partition of a cloud disk in the instance, such as /dev/nvme1n1p1. In this topic, this block device is called a volume. Use the Cryptpilot tool to initialize the selected volume. You can then use this volume to store confidential data in subsequent procedures.
Important
The initialization procedure erases the original data on the block device. Proceed with caution to prevent operational errors.
Open procedure: Use the Cryptpilot tool to open the initialized volume. A new virtual block device is created in the instance. Data written to this block device is encrypted and stored as ciphertext on the underlying Elastic Compute Service (ECS) cloud disk. When data is read from this virtual block device, the data on the underlying ECS cloud disk is automatically decrypted. The encryption and decryption processes occur in the Linux kernel. This effectively prevents plaintext from leaking outside the instance.

Encryption algorithm

The encryption process for a confidential cloud disk uses a two-layer key design with a key encryption key (KEK) and a master key.

During the initialization procedure, a master key is generated using a secure random number algorithm. This key is used directly for data encryption using the industry-standard 256-bit AES encryption algorithm. Alternatively, you can provide a passphrase to use as the key. The initialization procedure generates a KEK from this key using a key derivation algorithm and then uses the KEK to encrypt the master key.
When you open the volume, a KEK is generated from the key you provide. The KEK is then used to decrypt the master key, which in turn decrypts the confidential data. The AES hardware encryption capability of the instance accelerates the encryption and decryption processes. This reduces the performance overhead of encryption.

Key storage

Confidential cloud disks support multiple key storage methods. You can configure the key information required to open a volume in advance. This prevents you from having to manually enter the key each time the instance starts.

One-time key (OTP): This is a special key storage method. If you configure this type of key, a secure temporary key is automatically generated to initialize and open the volume. You do not need to separately initialize the volume.
Important
A different key is generated each time the volume is reopened. This method is suitable for temporary data disks or swap partitions. Do not store confidential data that must be persisted after the instance is shut down on this type of volume.
Key Management Service (KMS): This method stores the key in Alibaba Cloud KMS. You must configure the KMS access information in the ECS instance. When the volume is opened, the ECS instance automatically accesses Alibaba Cloud KMS to retrieve the key for encrypting and decrypting the volume. This method is suitable for storing confidential data that requires persistence.

Integrity support

In addition to preventing unauthorized access to and disclosure of confidential data, you must also prevent attackers from tampering with confidential data. This is important in scenarios such as storing large language model (LLM) weights and training datasets. Confidential cloud disks support data integrity using the Authenticated Encryption with Associated Data (AEAD) encryption mode.

Considerations

You can encrypt data disks only on TDX instances or heterogeneous confidential instances.
The confidential cloud disk solution does not prevent disk rollbacks. Although a rollback does not compromise data integrity or confidentiality, it can revert data to an older state. Therefore, this solution cannot help you identify such potential attacks. You must check data versions at the application layer based on your security requirements to handle these issues.

Procedure

Step 1: Prepare the environment

Create a confidential computing instance.
You can encrypt data disks only on TDX confidential instances or heterogeneous confidential instances.
Remotely connect to the confidential computing instance.
For more information, see Log on to a Linux instance using Workbench.
Install Cryptpilot.
```
sudo yum install -y cryptpilot-crypt
```
Check the Cryptpilot version.
```
cryptpilot-crypt --version
```

Step 2: Create and attach a cloud disk

Important

Encrypting a cloud disk erases the original data on the block device. We recommend creating a new data disk for this operation.

Create an empty data disk to be encrypted.
Attach the created data disk to the ECS instance.
Initialize the data disk.
When you initialize the data disk, create only a partition and a file system. Do not mount the file system yet. The file system must be mounted after the disk is encrypted.
View the attached cloud disk and partition information.
```
lsblk
```
The following output indicates that two cloud disks are attached to the current ECS instance:
- /dev/nvme0n1: The original system disk.
- /dev/nvme1n1: The new data disk with two empty partitions (/dev/nvme1n1p1 and /dev/nvme1n1p2).

Step 3: Create a volume with an ext4 file system

For a temporary volume, the confidential data in the volume is lost after the volume is closed. This is suitable for scenarios that use temporary data disks or swap partitions.
A persistent volume ensures that confidential data persists after the volume is closed. This is suitable for scenarios that require persistent storage of confidential data.

Create a temporary volume with an ext4 file system

Create and configure the /etc/cryptpilot/volumes/data0.toml file.

Note

The configuration file is a text file in the TOML format. The filename can be any name that ends with .toml and does not need to match the volume name.

Modify the configuration information in the following configuration file as needed.

sudo bash -c 'cat > /etc/cryptpilot/volumes/data0.toml << EOF
# Volume configuration
dev = "/dev/nvme1n1p1"    # Specify the block device to use, which is an empty partition of the cloud disk. This example uses the first partition of the data disk, /dev/nvme1n1p1.
volume = "data0"          # Specify the name of the volume. This example names it data0.
auto_open = false         # true means the volume is automatically opened when the system starts. false means the volume must be opened manually.
makefs = "ext4"           # Specify the file system type to create. This example uses ext4.
integrity = true          # true means data integrity support is enabled. false means data integrity support is disabled.

# Key configuration
[encrypt.otp]             # Specify the key storage type. Here, the one-time key (OTP) type is selected.
EOF'

View the volume status.
```
sudo cryptpilot-crypt show data0
```
The output displays the configuration information of the data0 volume:
- Initialized column: Indicates that the volume does not need to be initialized because its storage type is OTP (one-time key).
- Opened column: Indicates that the volume is not open.
Open the volume and view its status.
1. Open the volume.
```
sudo cryptpilot-crypt open data0
```
2. You can view the volume status.
```
sudo cryptpilot-crypt show data0
```
  The following output shows Opened: True, which indicates that the volume is open.
3. Check the virtual device corresponding to the volume.
```
sudo file -Ls /dev/mapper/data0
```
  When a volume is opened, a virtual block device is created at the path /dev/mapper/<volume name>.
Mount the file system and view the result.
1. Mount the data0 volume to the /mnt/data0 path.
```
sudo mkdir -p /mnt/data0
sudo mount -t ext4 /dev/mapper/data0 /mnt/data0
```
2. View the mount result.
```
mount | grep data0
ls -la /mnt/data0/
```
  The following output indicates that the volume was successfully mounted and contains an empty ext4 file system.
Take the volume offline.
Important
- This example uses the one-time key (OTP) model. After the volume is closed, the confidential data stored in it is completely lost.
- Before you close the volume, stop all processes that are reading from or writing to the volume. Unmount the file system from the volume to ensure that the volume is not in use.
1. Unmount the file system.
```
sudo umount /mnt/data0
```
2. Take the volume offline.
```
sudo cryptpilot-crypt close data0
```
3. View the volume status.
```
cryptpilot-crypt show data0
```
  The following output indicates that the volume is closed.

Create a persistent volume with an ext4 file system

In a secure environment, randomly generate a Base64-encoded key to encrypt the volume.
```
sudo yum install -y openssl
openssl rand -base64 40
```
Save the generated key to be uploaded later.
Select a backend to store the key
A persistent volume requires a stable and reusable encryption key. For security, you should not store this key directly on the business host. Instead, you should store it in a dedicated key management backend. Cryptpilot retrieves the key from the backend when the volume is opened.
This topic describes two common backend options. Choose one based on your environment:
- Option 1: Use a self-built Trustee service to store the key. This is suitable for scenarios where you build your own key storage backend and require remote attestation (trusted computing) for the instance.
- Option 2: Use Alibaba Cloud KMS to store the key. This is suitable for scenarios where you want to use a cloud-hosted key management service and do not need to build your own service.
The preparation steps for these two methods are described below.
Use a self-built Trustee service
Trustee is an open source trust management service that consists of the following components: a remote attestation service (AS), a reference value provider service (RVPS), and a key brokering service (KBS). In this solution, the encryption key is stored in the KBS. Cryptpilot retrieves the key from the KBS when opening the volume.
Step 1: Deploy and configure the Trustee service
1. In a trusted environment, such as a separate ECS instance, install the Trustee package:
  sudo yum install -y trustee
2. (Optional) Configure HTTPS access
  To enable access over HTTPS, edit /etc/trustee/gateway.yml and enter your TLS certificate configuration:
  server: insecure_http: false tls: cert_file: "<path_to_https_certificate_pem_file>" key_file: "<path_to_https_certificate_private_key_file>" # other configurations...
3. On an Alibaba Cloud Linux 3 instance, you can start the Trustee service using systemd:
  sudo systemctl restart trustee
  After Trustee starts, it listens on port 8081 on the local machine.
4. Modify the default remote attestation policy
  Trustee checks the TEE hardware environment, file system, programs, and configurations in the confidential instance and provides the check results.
  For simplicity, this example only checks for a valid TEE hardware environment. Modify the `/opt/trustee/kbs/policy.rego` file to contain the following content.
  package policy import rego.v1 default allow = false # --------------------------- # Resource path helpers # --------------------------- resource_path := data["resource-path"] path_parts := split(resource_path, "/") is_repo(name) if { count(path_parts) == 3; path_parts[0] == name } is_type(t) if { count(path_parts) == 3; path_parts[1] == t } is_tag(tag) if { count(path_parts) == 3; path_parts[2] == tag } # --------------------------- # Default decision # --------------------------- allow if { # cpu0 must exist s := input.submods["cpu0"] # cpu0 must carry a trustworthiness vector tv := s["ear.trustworthiness-vector"] # and it must pass the hardware check tv["hardware"] <= 32 }
  Note
  If you need more comprehensive validation, first refer to the remote attestation validation policy at /opt/trustee/attestation-service/policies/opa/default.rego. Use the Reference Value Configuration document and the Remote Attestation Configuration document to configure the reference values. This ensures that the validation results for all four dimensions of remote attestation are considered trusted (a value less than or equal to 32). You can then allow key resource requests that meet the strict verification conditions of the default resource policy.
5. Prepare and upload the encryption key for the volume
  Create a key storage directory (KBS uses /opt/trustee/kbs/repository/default/ by default):
  sudo mkdir -p /opt/trustee/kbs/repository/default/local-resources/
  Write the previously generated key to a local resource file. For example:
  sudo sh -c 'echo -n "<data1_volume_encryption_key>" > /opt/trustee/kbs/repository/default/local-resources/data1'
  Replace <data1-volume-encryption-key> with the previously generated random key. The corresponding KBS resource identifier is kbs:///default/local-resources/data1, which you can reference in the Cryptpilot volume configuration in subsequent steps.
Step 2: Install dependent components
1. To enable remote attestation in the confidential instance and access the Trustee service to retrieve keys, install the additional dependent components.
  sudo yum install confidential-data-hub attestation-agent -y sudo systemctl enable --now attestation-agent.service
Step 3: Create the encrypted volume configuration file
1. Create the /etc/cryptpilot/volumes/data1.toml file.
2. Configuration file
  Note
  The configuration file uses the TOML configuration language format, and its filename can be any text file that ends with .toml and does not need to match the volume name.
  Modify the configuration information in the following configuration file as needed.
  # Volume configuration dev = "/dev/nvme1n1p2" # Specify the block device to use, which is an empty partition of the cloud disk. This example uses the second partition of the data disk, /dev/nvme1n1p2. volume = "data1" # Specify the name of the volume. This example names it data1. auto_open = false # true means the volume is automatically opened when the system starts. false means the volume must be opened manually. makefs = "ext4" # Specify the file system type to create. This example uses ext4. integrity = true # true means data integrity support is enabled. false means data integrity support is disabled. # Key configuration (obtained through Trustee KBS) [encrypt.kbs] kbs_url = "http://<trustee-ip>:8081/api" # Replace <trustee-ip> with the public IP address of the Trustee instance. key_uri = "kbs:///default/local-resources/data1" # The resource URI of the key for the data1 volume stored in KBS.
Use Alibaba Cloud KMS to store the key
Alibaba Cloud KMS is a cloud-based key management service. It is suitable for scenarios where you want to host keys on Alibaba Cloud. In this solution, the encryption key is stored in KMS as a generic secret. Cryptpilot retrieves the key from KMS when opening the volume.
Step 1: Create an Alibaba Cloud KMS instance and a generic secret
1. Purchase and enable a KMS instance.
2. Create a software-protected key.
3. Create a generic secret.
  Secret Name: Enter a name for the generic secret. In this example, confidential_cloud_disk_passphrase is used.
  Secret Value: Select Plain Text and enter the random key generated in the previous step.
  Customer Master Key (CMK): Select the software-protected key created in the previous step.
4. Create an application access point, and obtain the CA certificate of the KMS instance.
  Make sure that you have obtained the following items:
  Application identity credentials (ClientKeyContent): The filename is clientKey_****.json by default.
  Credential security token (ClientKeyPassword): The filename is clientKey_****_Password.txt by default.
  KMS instance CA certificate: The filename is PrivateKmsCA_kst-******.pem by default.
Step 2: Create the encrypted volume configuration file
1. Create the /etc/cryptpilot/volumes/data1.toml file.
2. Configuration file.
  Note
  The configuration file is in the TOML format. The filename must end with .toml but does not need to match the volume name.
  Modify the configuration information in the following configuration file as needed.
  # Volume configuration dev = "/dev/nvme1n1p2" # Specify the block device to use, which is an empty partition of the cloud disk. This example uses the first partition of the data disk, /dev/nvme1n1p2. volume = "data1" # Specify the name of the volume. This example names it data1. auto_open = false # true means the volume is automatically opened when the system starts. false means the volume must be opened manually. makefs = "ext4" # Specify the file system type to create. This example uses ext4. integrity = true # true means data integrity support is enabled. false means data integrity support is disabled. # Key configuration [encrypt.kms] # Specify the key storage type. Here, the Key Management Service (KMS) type is selected. secret_name = "confidential_cloud_disk_passphrase" # The name of the secret stored in the KMS instance, which is confidential_cloud_disk_passphrase in the previous example. # The content of the application identity credentials file (clientKey_****.json file). client_key = ''' { "KeyId": "KAAP.b183152a-c3bf-43ae-9399-a101607b****", "PrivateKeyData": "MIIJ0wIBAzCCCZ8GCSqGSIb3DQE****" } ''' client_key_password = "258372e962f5204ce648bc66243f****" # The content of the credential security token file (clientKey_****_Password.txt file). kms_instance_id = "kst-bjj67d004a85ykq80****" # The KMS instance ID. # The content of the KMS instance CA certificate file (PrivateKmsCA_kst-******.pem file). kms_cert_pem = """ -----BEGIN CERTIFICATE----- MIIDuzCCAqOgAwIBAgIJALTKwWAjvbMiMA0GCSqGSIb3DQEBCwUAMH**** -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIID3zCCAsegAwIBAgIJAO8qnQyTy8/kMA0GCSqGSIb3DQEBCwUAMH**** -----END CERTIFICATE----- """
View the volume status.
```
sudo cryptpilot-crypt show data1
```
You can view the configuration for the data0 volume:
- Initialized column: Indicates that the volume has not been initialized.
- Opened column: Indicates that the volume is not open.
Initialize the volume and view its status.
1. Initialize the volume.
```
sudo cryptpilot-crypt init data1
```
2. View the volume status.
```
sudo cryptpilot-crypt show data1
```
  The following output indicates that the volume has been initialized (Initialized: True).
Open the volume and view its status.
1. Open the volume.
```
sudo cryptpilot-crypt open data1
```
2. View the volume status.
```
sudo cryptpilot-crypt show
```
  The following output indicates that the volume is open (Opened: True).
3. Check the virtual device corresponding to the volume.
```
sudo file -Ls /dev/mapper/data1
```
  When a volume is opened, a virtual block device is created at the following path: /dev/mapper/<volume_name>.
Mount the file system and write data.
Mount the data1 volume to the /mnt/data1 path and create the /mnt/data1/data.txt file to represent confidential data.
```
sudo mkdir -p /mnt/data1
sudo mount -t ext4 /dev/mapper/data1 /mnt/data1
echo "secret data" | sudo tee /mnt/data1/data.txt > /dev/null
```
Close and reopen the volume, and then check the data written previously.
Important
- This example uses the one-time key (OTP) model. When the volume is shut down, confidential data stored on it is permanently deleted.
- Before you close the volume, stop all processes that are reading from or writing to the volume. Unmount the file system from the volume to ensure that the volume is not in use.
1. Unmount the file system and close the volume.
```
sudo umount /mnt/data1
sudo cryptpilot-crypt close data1
```
2. Reopen the volume, mount the file system, and access the file in the volume.
```
sudo cryptpilot-crypt open data1
sudo mount -t ext4 /dev/mapper/data1 /mnt/data1
cat /mnt/data1/data.txt
```
  The output shows that the previously written data still exists. This verifies the persistent storage capability.

Step 4: Configure the instance to automatically open the volume on startup

Modify the configuration file.
For volumes that you want to open automatically, change the value of the auto_open field in their configuration file to true.
(Optional) For a persistent volume, you must manually initialize the volume by running the sudo cryptpilot-crypt init <volume_name> command before it can be automatically opened when the instance starts.
Start the cryptpilot service and enable it to start automatically on boot.
Important
After each modification to the cryptpilot configuration, you must start the cryptpilot service and enable it to start automatically on boot. This ensures that the auto-open volume feature takes effect.
```
sudo systemctl enable --now cryptpilot.service
```
Procedure
View the volume status.
```
sudo cryptpilot-crypt show
```
The following output indicates that the volume was automatically opened.

Cryptpilot configuration

The default configuration directory for Cryptpilot is /etc/cryptpilot/, which contains the following configuration files:

Volume configuration: The default path is /etc/cryptpilot/volumes/. Each volume corresponds to a configuration file.

Temporary storage type (OTP)

# The name of resulting volume with decrypted data, which will be set up below `/dev/mapper/`.
volume = "data0"
# The path to the underlying encrypted device.
dev = "/dev/nvme1n1p1"
# Whether or not to open the LUKS2 device and set up mapping during booting. The default value is false.
auto_open = true
# The file system to initialize on the volume. Allowed values are ["swap", "ext4", "xfs", "vfat"]. If is not specified, or the device is not "empty", i.e. it contains any signature, the operation will be skipped.
makefs = "ext4"
# Whether or not to enable support for data integrity. The default value is false. Note that integrity cannot prevent a replay (rollback) attack.
integrity = true

# One Time Password (Temporary volume)
[encrypt.otp]

Persistent storage type (self-built Trustee service)

# The name of resulting volume with decrypted data, which will be set up below `/dev/mapper/`.
volume = "data0"
# The path to the underlying encrypted device.
dev = "/dev/nvme1n1p1"
# Whether or not to open the LUKS2 device and set up mapping during booting. The default value is false.
auto_open = true
# The file system to initialize on the volume. Allowed values are ["swap", "ext4", "xfs", "vfat"]. If is not specified, or the device is not "empty", i.e. it contains any signature, the operation will be skipped.
makefs = "ext4"
# Whether or not to enable support for data integrity. The default value is false. Note that integrity cannot prevent a replay (rollback) attack.
integrity = true

# Key Broker Service
[encrypt.kbs]
# The HTTP url of the KBS instance.
kbs_url = "http://<trustee-ip>:8081/api"
# The Resource URI in the KBS instance which refer to the KBS resource that will be used as passphrase. Should be in format `kbs:///<repo>/<type>/<tag>`
key_uri = "kbs:///default/mykey/volume_data0"
# The X.509 Root Cert used for HTTPS connection to the KBS instance, in PEM format. If not specified, the native Root CA certificate store in the system will be used.
kbs_root_cert = """
-----BEGIN CERTIFICATE-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----
"""

Persistent storage type (Alibaba Cloud KMS)

# The name of resulting volume with decrypted data, which will be set up below `/dev/mapper/`.
volume = "data0"
# The path to the underlying encrypted device.
dev = "/dev/nvme1n1p1"
# Whether or not to open the LUKS2 device and set up mapping during booting. The default value is false.
auto_open = true
# The file system to initialize on the volume. Allowed values are ["swap", "ext4", "xfs", "vfat"]. If is not specified, or the device is not "empty", i.e. it contains any signature, the operation will be skipped.
makefs = "ext4"
# Whether or not to enable support for data integrity. The default value is false. Note that integrity cannot prevent a replay (rollback) attack.
integrity = true

# Aliyun KMS
[encrypt.kms]
# The id of KMS instance
kms_instance_id = "kst-XXXXXXXXX"
# The name of the secret store in the KMS instance.
secret_name = "XXXXXXXXX"
# Content of the clientKey_****.json file.
client_key = '''
{
  "KeyId": "KAAP.XXXXXXXXX",
  "PrivateKeyData": "XXXXXXXXX"
}'''
# Content of the clientKey_****_Password.txt file.
client_key_password = "XXXXXXXXX"
# The CA cert of the KMS (the content of PrivateKmsCA_kst-******.pem file).
kms_cert_pem = """
-----BEGIN CERTIFICATE-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
-----END CERTIFICATE-----
"""

Encryption overview

How encryption works

Encryption algorithm

Key storage

Integrity support

Considerations

Procedure

Step 1: Prepare the environment

Step 2: Create and attach a cloud disk

Step 3: Create a volume with an ext4 file system

Create a temporary volume with an ext4 file system

Create a persistent volume with an ext4 file system

Use a self-built Trustee service

Step 1: Deploy and configure the Trustee service

Step 2: Install dependent components

Step 3: Create the encrypted volume configuration file

Use Alibaba Cloud KMS to store the key

Step 1: Create an Alibaba Cloud KMS instance and a generic secret

Step 2: Create the encrypted volume configuration file

Step 4: Configure the instance to automatically open the volume on startup

Cryptpilot configuration