In Container Service for Kubernetes (ACK) Serverless Pro clusters, you can use keys that are created in Key Management Service (KMS) to encrypt Kubernetes Secrets. This reduces the risk of information leakage. This topic describes how to use a key that is managed by KMS to encrypt Secrets for an existing ACK Serverless Pro cluster.
Prerequisites
Item | Description |
KMS key | A KMS key is created in the KMS console. The key belongs to the region where your ACK Serverless Pro cluster resides. ACK Serverless Pro clusters support default keys, software-protected keys, and hardware-protected keys. For more information about the key management feature of KMS, see Getting started with keys. For more information about KMS billing, see Billing. Important After you enable Secret encryption, do not use the KMS API or the KMS console to disable or delete the key that is used to encrypt and decrypt Secrets, or create a schedule to delete the key. Otherwise, the API server becomes unavailable and cannot retrieve Secrets or service account objects. As a result, service interruptions occur. |
Authorization | The following requirements must be met:
|
Limits
Secret encryption can be enabled only for existing ACK Serverless Pro clusters. This feature cannot be enabled when you create ACK Serverless Pro clusters.
Overview of Secret encryption
Kubernetes Secrets are used to store and manage sensitive data, such as the passwords of applications, Transport Layer Security (TLS) certificates, and credentials to download Docker images. Kubernetes stores Secrets in etcd of a cluster. For more information, see Secrets.
ACK Pro clusters allow you to use a key created in KMS to encrypt Secrets. The KMS provider mechanism of Kubernetes is used during encryption. A KMS provider uses envelope encryption to encrypt and decrypt Secrets that are stored in etcd. Procedures for Secret encryption and decryption:
When you use a Kubernetes Secret to store a password, the API server generates a random data encryption key (DEK) to encrypt the Secret. Then, the API server sends the DEK to KMS. KMS uses the specified key to encrypt the DEK and returns the encrypted DEK to the API server. The API server then stores the encrypted Secret and DEK in etcd.
When you decrypt the Kubernetes Secret, the system calls the Decrypt operation of KMS to decrypt the DEK first. Then, the system uses the plaintext DEK to decrypt the Kubernetes Secret and returns the decrypted Secret.
For more information, see The KMS provider and Use envelope encryption.
KMS v2
In Kubernetes 1.31 and later, ACK Pro clusters use KMS v2 to implement disk encryption. KMS v2 allows you to manage more Secret instances in the cluster and provides better encryption and decryption performance. For more information about the encryption and decryption process of KMS v2, see Kubernetes 1.27: KMS V2 Moves to Beta.
For more information about how to upgrade KMS v1 to KMS v2, see How do I upgrade KMS v2 for Secret encryption?
The Kubernetes community has marked KMS v1 as obsolete in Kubernetes 1.27, and the v1 interface is disabled by default in Kubernetes 1.29. To ensure compatibility, you can configure KMS v1 feature gates to enable v1 interface in ACK Pro clusters .
Enable Secret encryption for an existing ACK Serverless Pro cluster
Log on to the ACK console. In the navigation pane on the left, click Clusters.
On the Clusters page, click the name of the ACK Serverless Pro cluster for which you want to enable Secret encryption. On the details page of the cluster, click the Basic Information tab. In the Security and Auditing section, enable Secret Encryption.
In the dialog box that appears, select the key that you created and click OK.
On the Clusters page, if the status of the cluster changes from Updating to Running, the Secret encryption feature is enabled for the cluster.
If the Secret encryption feature is no longer required, you can turn off Secret Encryption in the Basic Information section.
Use automatic key rotation to encrypt Secrets
You can use the automatic key rotation feature provided by KMS to encrypt Secrets. During a key rotation, the system still uses the original key to encrypt existing Secrets. New Secrets are encrypted by using the new key. For more information about automatic key rotation, see Configure key rotation.
To force the system to use the new key to encrypt existing Secrets, run the following command after the key is rotated:
kubectl get secrets --all-namespaces -o json | kubectl annotate --overwrite -f - encryption-key-rotation-time="$(date -u +'%Y-%m-%dT%H:%M:%S%z')"FAQ
How do I upgrade KMS v2 to use Secret encryption?
If you use KMS v2 for Secret encryption, make sure that the version of your ACK Pro cluster is 1.31 or later.
For more information about how to enable Secret encryption for your cluster, see Enable Secret encryption for an ACK Pro cluster. After Secret encryption is enabled, Secret instances in the cluster are automatically encrypted and decrypted by using KMS v2 during disk loading and reading.
If Secret encryption is enabled for your cluster, you can use one of the following methods to rotate the encryption method for your cluster during off-peak hours:
Method 1: Run the
kubectl get secrets --all-namespaces -o json | kubectl replace -f -command. This command rotates the encryption method of all the Secrets that have been installed on the disk by using the encryption method of KMS v2.Method 2: Log on to the ACK console. On the Basic Information tab of the Cluster Information page, disable Secret Encryption. The disable operation triggers the rotation of all Secrets instances in the cluster. After a period of time, enable Secret encryption for the cluster. For more information, see Use KMS to encrypt Kubernetes Secrets. After the Secret encryption feature is enabled, all Secrets in the cluster are encrypted by using KMS v2.
After Secret encryption is enabled, is ciphertext returned if I use kubectl to query a Secret?
No. After Secret encryption is enabled, plaintext is returned if you use kubectl to query a Secret. The Secret encryption feature encrypts the Secrets that are stored in etcd. After you enable Secret encryption, Secrets are stored in etcd as ciphertext. However, if you use a kubectl client to query a Secret by calling the Secret API provided by the API server of the cluster, plaintext is returned for the Secret.
How do I prohibit RAM users or RAM roles from enabling or disabling the Secret encryption feature for existing ACK Pro clusters?
To prohibit RAM users or RAM roles from enabling or disabling the Secret encryption feature for existing ACK Pro clusters, attach the following policy to the RAM users or RAM roles. For more information, see Create a custom RAM policy.
{
"Action": [
"cs:UpdateKMSEncryption"
],
"Effect": "Deny",
"Resource": [
"*"
]
}