In serverless Kubernetes (ASK) Pro clusters, you can use keys that are created in Key Management Service (KMS) to encrypt Kubernetes Secrets. This topic describes how to use a key that is managed by KMS to encrypt Secrets for an ASK Pro cluster.

Prerequisites

  • A customer master key (CMK) is created in the KMS console. For more information, see Create a CMK.
    Note ASK Pro clusters support only CMKs of the Aliyun_AES_256 type.
  • After you enable Secret encryption, do not use the KMS API or the KMS console to disable or delete the CMK that is used to encrypt and decrypt Secrets, or create a schedule to delete the CMK. Otherwise, the API server becomes unavailable and cannot retrieve the Secret and ServiceAccount objects. As a result, service interruptions occur.
  • Your Alibaba Cloud account is authorized to assume the AliyunCSManagedSecurityRole role. Otherwise, the system prompts you to perform the authorization when you enable Secret encryption.
  • If you log on to the Container Service for Kubernetes (ACK) console with a Resource Access Management (RAM) user or RAM role, you must make sure that the RAM user or RAM role is attached with the AliyunKMSCryptoAdminAccess permission. For more information, see Grant permissions to a RAM user or RAM role.
  • You are charged by KMS for key management and API calls (on a per 10,000 calls basis). After Secret encryption is enabled for the ASK Pro cluster, kube-apiserver must call the encryption and decryption API operations of KMS to read and write Secrets. By default, a large number of API calls are made to read and write Secrets when you manage the lifecycle of service accounts in the cluster. If the cluster contains a large number of service accounts and Secrets, the cost of KMS API calls is high. Make sure that your account balance is sufficient. If you have questions about billing or your account balance is insufficient, you can disable Secret encryption. For more information, see Disable Secret encryption. If your account has been overdue for more than seven days, you cannot manage the cluster. For more information about KMS billing, see Billing of KMS.

Background information

Kubernetes Secrets are used to store and manage sensitive data, such as passwords of applications, Transport Layer Security (TLS) certificates, and credentials to download Docker images. Kubernetes stores Secrets in the etcd of a cluster. For more information, see Secrets.

ACK Pro clusters allow you to use a key created in KMS to encrypt Secrets. The KMS provider mechanism of Kubernetes is used during encryption. A KMS provider uses envelope encryption to encrypt and decrypt the keys of Secrets that are stored in etcd. Procedures of Secret encryption and decryption:
  • When you use a Kubernetes Secret to encrypt and store a password, the API server generates a random data encryption key (DEK) to encrypt the Secret. Then, the API server sends the DEK to KMS. KMS uses the specified key to encrypt the DEK and returns the encrypted DEK to the API server. The API server then stores the encrypted Secret and DEK in etcd.
  • When you decrypt the Kubernetes Secret, the system calls the Decrypt operation of KMS to decrypt the data key first. Then, the system uses the plaintext of the data key to decrypt the Kubernetes Secret and returns the decrypted Secret.

For more information, see KMS Encryption Provider mechanism and Use envelope encryption to encrypt and decrypt local data.

Note Secret encryption is supported only by existing ASK Pro clusters. This feature cannot be enabled in newly created ASK Pro clusters.

Enable Secret encryption for an existing ASK Pro cluster

  1. Log on to the ACK console and click Clusters in the left-side navigation pane.
  2. On the Clusters page, click the name of the ASK Pro cluster for which you want to enable Secret encryption.
  3. On the details page of the cluster, click the Basic Information tab. In the Basic Information section, turn on Secret Encryption.
    Note If you log on to the ACK console with a RAM user, make sure that the RAM user is assigned one of the following RBAC roles: the administrator role or O&M engineer role. For more information, see Grant RBAC permissions to RAM users or RAM roles.
  4. In the Secret Encryption dialog box, select an existing key and click OK.
    If no key is available, click create keys to create a key in the KMS console. For more information, see Create a CMK.
    If the status of the cluster changes from Updating to Running, the Secret encryption feature is enabled for the cluster.
Note If the Secret encryption feature is no longer required, you can turn off Secret Encryption in the Basic Information section.

Use automatic key rotation to encrypt Secrets

You can use the automatic key rotation feature provided by KMS to encrypt Secrets. During a key rotation, the system still uses the original key to encrypt existing Secrets. New Secrets are encrypted by using the new key. For more information about automatic key rotation, see Automatic key rotation.

To force the system to use the new key to encrypt existing Secrets, run the following command after the key is rotated:

kubectl get secrets --all-namespaces -o json | kubectl annotate --overwrite -f - encryption-key-rotation-time="TIME"
Note Replace TIME with the string that indicates the time when the key was rotated, for example, 20220101-010101.