In an ACK Pro cluster or an , you can use keys created in Alibaba Cloud Key Management Service (KMS) to encrypt Kubernetes Secrets. This topic describes how to use KMS-managed keys to enable encryption at rest for Secrets in an ACK Pro cluster.
Prerequisites
|
Item |
Description |
|
KMS key |
A KMS key is created in the Key Management Service console. The key and the target ACK Pro cluster must be in the same region. ACK Pro clusters support default keys, software keys, and hardware keys. You can choose a key type based on your business requirements. For more information about how to manage KMS keys, see Key management quick start. For more information about the billing of KMS, see Billing. Important
After you enable encryption at rest, do not disable or delete the key that the cluster uses to encrypt and decrypt Secrets in the KMS console or by calling API operations. Otherwise, the cluster's API server will become unavailable. You will not be able to retrieve objects such as Secrets and ServiceAccounts, and your applications may fail. |
|
Cluster network ACL rule configuration |
When you enable encryption at rest, the KMS plug-in on the control plane needs to access the Alibaba Cloud KMS OpenAPI to encrypt and decrypt Secrets. Therefore, you must ensure that the Outbound rules of the cluster's security group and the inbound and outbound rules of the VPC network ACL allow access to the Alibaba Cloud service CIDR block ( |
|
Authorization |
Make sure that the following authorization requirements are met based on your account type.
|
Secret encryption
In Kubernetes clusters, you can use the Secret resource to store and manage sensitive information for your applications, such as passwords, TLS certificates, and Docker image pull credentials. Kubernetes stores all Secret objects in the cluster's etcd. For more information, see Secrets.
In an ACK Pro cluster or an , you can use a key created in KMS to encrypt Kubernetes Secrets. KMS encryption is based on the KMS Encryption Provider mechanism provided by Kubernetes. It uses envelope encryption to automatically encrypt and decrypt Kubernetes Secrets stored in etcd. The encryption and decryption workflow is as follows:
-
When you store a Secret by using the Kubernetes Secret API, the API server generates a random data encryption key (DEK) to encrypt the Secret data. The DEK is then encrypted with the specified KMS key, and the resulting ciphertext is stored in etcd.
-
When you read a Secret, the system calls the KMS Decrypt OpenAPI operation to decrypt the ciphertext key. The system then uses the resulting plaintext key to decrypt the Secret data and returns the original content.
For more information, see Using a KMS provider for data encryption and Use a KMS key for envelope encryption.
KMS v2
Starting from Kubernetes 1.31, ACK Pro clusters and use the community KMS v2 API for encryption at rest by default. KMS v2 supports more Secrets in a cluster and provides better encryption and decryption performance. For more information about the KMS v2 encryption and decryption process, see KMS v2.
To upgrade from the KMS v1 API to the KMS v2 API, see How do I upgrade to use KMS v2 for Secret encryption at rest?
The Kubernetes community deprecated the KMS v1 API in version 1.27 and disabled it by default in version 1.29. For compatibility, ACK Pro clusters and still enable the KMS v1 API by using its feature gate.
Enable secret encryption in ACK Pro clusters
New ACK Pro cluster
Log on to the ACK console. In the left navigation pane, click Clusters.
-
On the Clusters page, click Create Kubernetes Cluster.
-
On the ACK Managed Cluster tab, expand Advanced Options (Optional). Find Secret Encryption, select the Select Key checkbox, and then select a KMS key ID from the drop-down list. Configure other parameters as required and click Create Kubernetes Cluster.
For more information about how to configure other parameters for an ACK Pro cluster, see Parameters of an ACK managed cluster.
Log on to the ActionTrail console. In the left-side navigation pane, click Cluster Events Query. If the Cluster Events Query page shows encryption and decryption event logs that use the aliyuncsmanagedsecurityrole system role, encryption at rest is enabled for the cluster.
If you no longer need this feature, go to the cluster details page. On the Cluster Information page, click the Basic Information tab. In the Security and Auditing section, turn off the Secret Encryption switch.
Existing ACK Pro cluster
Log on to the ACK console. In the left navigation pane, click Clusters.
-
On the Clusters page, click the name of the target cluster. On the cluster details page, click the Basic Information tab. In the Security and Auditing section, turn on the Secret Encryption switch.
The first time you enable this feature, click Go to RAM console as prompted to go to the RAM Quick Authorization page, and then click Confirm to complete the authorization.
Note-
To enable encryption at rest, make sure that the current RAM user or RAM role has RBAC administrator or O&M permissions on the cluster. For more information, see Authorize users and user groups by using RBAC.
-
To grant permissions to the
aliyuncsmanagedsecurityrolerole, make sure that you are logged on with an Alibaba Cloud account or a RAM user or RAM role that has RAM management permissions.
-
-
In the Secret Encryption dialog box, select an existing KMS key and click OK.
If you have not created a KMS key, click Create Key to go to the Key Management Service console to create a key. For more information, see Create a key.
When the cluster state changes from Updating to Running, encryption at rest for Secrets is enabled for the cluster.
If you no longer need this feature, you can turn off the Secret Encryption switch in the Security and Auditing section.
Automatic key rotation
You can use the automatic key rotation feature of KMS to encrypt Secrets at rest. After a key is automatically rotated, existing Secrets are still encrypted with the previous key version, while new Secrets are encrypted with the new key version. For more information about automatic key rotation, see Key rotation.
To ensure that existing Secrets are also encrypted with the new key version, run the following command after the key is automatically rotated. This forces all existing Secrets to be re-encrypted with the new key version.
kubectl get secrets --all-namespaces -o json | kubectl annotate --overwrite -f - encryption-key-rotation-time="$(date -u +'%Y-%m-%dT%H:%M:%S%z')"FAQ
How to upgrade to KMS v2?
To use KMS v2 for Secret encryption at rest, make sure that your ACK Pro cluster or is version 1.31 or later.
-
If you have not enabled encryption at rest for Secrets for your cluster, see Enable encryption at rest for Secrets in an ACK Pro cluster to enable it. After you enable the feature, the KMS v2 API is automatically used to encrypt and decrypt Secrets.
-
If you have already enabled encryption at rest for Secrets, you can use one of the following methods during off-peak hours to upgrade the encryption provider to KMS v2.
-
Method 1: Run the
kubectl get secrets --all-namespaces -o json | kubectl replace -f -command. This command re-encrypts all Secrets by using the KMS v2 provider. -
Method 2: Log on to the Container Service for Kubernetes console. On the Cluster Information tab of the Basic Information page, find and turn off Secret Encryption. This process triggers the re-encryption of all Secrets in the cluster. After a period of time, see Enable encryption at rest for Secrets in an ACK Pro cluster to re-enable the feature. After the feature is re-enabled, all Secrets in the cluster are encrypted at rest by using the KMS v2 API.
-
Are secrets retrieved by kubectl encrypted?
No. The encryption at rest feature encrypts the raw data stored in etcd. This means that after you enable the feature, the Secret data stored in etcd is ciphertext. However, the Secret data that clients retrieve through the API server is still plaintext.
How to prevent a RAM user or RAM role from enabling or disabling Secret encryption at rest in an existing ACK Pro cluster
To prevent a RAM user or RAM role from enabling or disabling encryption at rest for Secrets in an existing ACK Pro cluster or , attach the following RAM policy to the user or role. For more information, see Use RAM to grant permissions on clusters and cloud resources.
{
"Action": [
"cs:UpdateKMSEncryption"
],
"Effect": "Deny",
"Resource": [
"*"
]
}