Container Service for Kubernetes:Use ack-secret-manager to import secrets from KMS

Security considerations

In most cases, information about the KMS secret is stored in files for applications to retrieve. This situation poses a compatibility issue with directly retrieving secrets from the KMS Secrets Manager. The ack-secret-manager component can resolve this compatibility issue. You can also use this component to synchronize secrets from KMS to Kubernetes Secrets in your ACK cluster. Then, you can mount and use the Kubernetes Secrets as environment variables. Before you use this component, assess the following security risks:

• When secrets in your cluster are accessible through the file system, attackers can exploit Common Vulnerabilities and Exposures (CVE) vulnerabilities in applications to traverse the directories in the cluster. As a result, the secrets may be leaked.

• Improper configuration of debug breakpoints or excessive log access permissions may expose secrets in your cluster. We recommend that you do not reference secrets through environment variables to avoid potential security risks.

• When you enable secret synchronization, strictly follow the least privilege principle to limit access permissions.

If you do not need to persist secrets for your application, we recommend that you refer to Use RRSA to authorize different pods to access different cloud services to grant the least access permissions to pods. Then, obtain the secrets in the application by calling the GetSecretValue operations directly. This reduces the chances of exposing the secrets in the pod file system or Kubernetes Secrets.

Prerequisites

An ACK cluster is created in the same region as the KMS secret that you want to use. For more information, see Create an ACK managed cluster, Create ACK One registered clusters, and Create an ACK Serverless cluster.

Step 1: Install ack-secret-manager

1. Log on to the ACK console. In the navigation pane on the left, click Clusters.

2. On the Clusters page, find the cluster you want and click its name. In the left-side navigation pane, choose Applications > Helm.

3. On the Helm page, click Deploy. In the Chart section of the Deploy panel, find and select ack-secret-manager, use the default values for other parameters, and then click Next.

   In the Confirm message, the displayed information indicates that the chart will be installed in the kube-system namespace by default and the default application name will be used. Click Yes. To use a custom application and a custom namespace, configure the Application Name and Namespace parameters in the Basic Information step.

4. In the Parameters step, select the latest chart version, configure other parameters, and then click OK.

   • To enable Resource Access Management (RAM) Roles for Service Accounts (RRSA) for authentication, set the rrsa.enable parameter to true.

   • To enable scheduled synchronization of secrets, configure the parameters shown in the following figure.

     

     • command.disablePolling: specifies whether to disable round-robin for secrets. A value of false enables round-robin for secrets.

     • command.pollingInterval: the synchronization frequency. A value of 120s specifies that the secrets are synchronized every 2 minutes. You can specify a custom value based on your business requirements.

   • To throttle Secrets in clusters containing many ExternalSecrets, which are KMS secrets pending synchronization, improper configuration may trigger throttling on the KMS or RAM side. Therefore, set the following throttling parameter to avoid such issues:

     

     command.maxConcurrentKmsSecretPulls: The maximum number of KMS secrets that can be synchronized within one second. The default value is 10.

   • To specify a KMS endpoint address, configure the kmsEndpoint parameter.

     

     command.kmsEndpoint: shared gateways and dedicated gateways for KMS. This parameter is a global configuration and currently also supports secret-level configuration.

   After you install the component, you are redirected to the ack-secret-manager page. You can check whether the component is installed. If the resources shown in the following figure are created, the component is installed.

Step 2: Authorize ack-secret-manager to access KMS secrets

You need to create a SecretStore to authorize ack-secret-manager to access KMS secrets. Otherwise, ack-secret-manager cannot import or synchronize secrets from KMS to your ACK cluster. You can use the following methods to authorize ack-secret-manager to access KMS secrets:

• Use RRSA to grant permissions: This method applies to ACK managed clusters and ACK Serverless clusters that run Kubernetes 1.22 or later.

• Grant permissions to the worker RAM role of the cluster: This method applies to ACK managed clusters, ACK dedicated clusters, and registered clusters.

• Specify an AccessKey pair used to assume a RAM role: This method applies to all types of ACK clusters.

Step 3: Specify synchronization details

After you authorize ack-secret-manager to access KMS secrets, you need to create an ExternalSecret to specify the details of the KMS secrets that you want to access. This way, you can import a secret from KMS to a Kubernetes Secret in your cluster.

1. Create an ExternalSecret.

   1. Modify the following code block based on the following description. Then, create a file named external.yaml based on the modified code block.

      Parameter	Description
      {KMS secret name}	Required. Replace the value with the name of the KMS secret that you want to access.
      {Kubernetes secret key}	Required. This is a collection of key-value pairs. The data of a KMS secret is stored as a key-value pair in a Kubernetes Secret. You must replace {Kubernetes secret key} with the key of the corresponding key-value pair.
      {KMS secret version stage}	Optional. Replace the value with the stage label of the KMS secret instead of the version number of the KMS secret. Example: ACSCurrent. RDS secrets, PolarDB secrets, Redis/Tair secrets, RAM secrets, and ECS secrets can only retrieve the secret versions corresponding to the ACSPrevious and ACSCurrent parameters. To specify the version number of the KMS secret for synchronization, you can replace the versionStage field in the following template with versionId, and fill in the version number of the KMS secret. RDS secrets, PolarDB secrets, Redis/Tair secrets, RAM secrets, and ECS secrets do not support specifying the VersionId parameter. The setting of this parameter is ignored. For more information about the secret version and version status, see the secret versions in Secret elements.
      {KMS Service endpoint address}	Optional. If you want to specify the KMS request endpoint, replace this parameter with the corresponding endpoint address. Shared gateways and dedicated gateways for KMS are supported and can be configured based on your business requirements. This parameter is a secret-level configuration, which allows you to configure the endpoint address specifically for KMS secrets. Global configuration is also supported. For more information about the configuration description, see Configure KMS endpoint addresses. After this parameter is set, the global configuration and default configuration are overridden, and the endpoint address for the secret request is the value of this parameter.
      {secret store name}	Optional. Replace the value with the name of the SecretStore you created. The SecretStore is used to authorize ack-secret-manager to access the KMS secret. Note You can leave this parameter empty if the component is authorized by the worker RAM role.
      {secret store namespace}	Optional. Replace the value with the namespace of the SecretStore you created. Note You can leave this parameter empty if the component is authorized by the worker RAM role.

      apiVersion: 'alibabacloud.com/v1alpha1'
      kind: ExternalSecret
      metadata:
        name: esdemo
      spec:
        provider: kms # By default, the type of Alibaba Cloud Services to be synchronized is KMS. When synchronizing KMS secrets, this field can be left empty or assigned the value kms.
        data: # No additional configurations are required for the data source. 
          - key: {KMS secret name}
            name: {Kubernetes secret key}
            versionStage: {KMS secret version stage}
            secretStoreRef: # This parameter is not required when the component is authorized through the worker RAM role.
              name: {secret store name}
              namespace: {secret store namespace}
                  - key: {KMS secret name}
            name: {Kubernetes secret key}
            versionStage: {KMS secret version stage}
            kmsEndpoint: {KMS Service endpoint address}

   2. Run the following command to create the ExternalSecret:

      kubectl apply -f external.yaml

2. Run the following command to check whether a Kubernetes Secret exists:

   kubectl get secret esdemo

   If a Kubernetes Secret exists, the KMS secret is synchronized from KMS to your cluster.

Advanced ack-secret-manager configurations

{"name":"tom","friends":[{"name":"lily"},{"name":"mark"}]}

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: es-json-demo
spec:
  provider: kms
  data: 
    - key: {KMS secret name}
      versionStage: {KMS secret version stage}
      secretStoreRef:
        name: {secret store name}
        namespace: {secret store namespace}
      jmesPath: # Parse some json fields 
        - path: "name"
          objectAlias: "myname"
        - path: "friends[0].name"
          objectAlias: "friendname"

{"/name-invalid":"lily","name-invalid/":[{"name":"mark"}]}

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: extract-secret
spec:
  provider: kms
  dataProcess:
    - extract:
        key: {KMS secret name}
        versionStage: ACSCurrent # The version of the KMS secret.
        secretStoreRef:
          name: {secret store name}
          namespace: {secret store namespace}
      replaceRule: # The rules for replacing keys. 
        - source: "^/.*d$" # Replace the keys that start with a forward slash (/) and end with a lowercase letter d with tom. 
          target: "tom"
        - source: "^n.*/$" # Replace the keys that start with a lowercase letter n and end with a forward slash (/) with mark. 
          target: "mark"

name: tom
friends:
  - name: lily
  - name: mark

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: es-json-demo
spec:
  provider: kms
  data: 
    - key: {KMS secret name}
      versionStage: {KMS secret version stage}
      secretStoreRef:
        name: {secret store name}
        namespace: {secret store namespace}
      jmesPath: # Parse some fields in yaml string
        - path: "name"
          objectAlias: "myname"
        - path: "friends[0].name"
          objectAlias: "friendname"

/name-invalid: lily
name-invalid/:
  - name: mark

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: extract-secret
spec:
  provider: kms
  dataProcess:
    - extract:
        key: {KMS secret name}
        versionStage: ACSCurrent # The version of the KMS secret.
        secretStoreRef:
          name: {secret store name}
          namespace: {secret store namespace}
      replaceRule: # The rules for replacing keys. 
        - source: "^/.*d$" # Replace the keys that start with a forward slash (/) and end with a lowercase letter d with tom. 
          target: "tom"
        - source: "^n.*/$" # Replace the keys that start with a lowercase letter n and end with a forward slash (/) with mark. 
          target: "mark"

apiVersion: "alibabacloud.com/v1alpha1"
kind: ExternalSecret
metadata:
  name: esdemo
spec:
  provider: kms
  data:
    - key: test-hangzhou # Actual endpoint address: Use the global configuration if it exists. Otherwise, use the default configuration: kms-vpc.{region}.aliyuncs.com
      name: hangzhou-vpc
      versionId: v1
    - key: test-hangzhou # Actual endpoint address: The endpoint specified by the kmsEndpoint field is kms.cn-hangzhou.aliyuncs.com
      name: hangzhou-public
      versionId: v1
      kmsEndpoint: kms.cn-hangzhou.aliyuncs.com

References

To secure Kubernetes Secrets that are synchronized from KMS secrets and cached in ACK clusters, we recommend that you enable the Secret encryption feature. For more information, see Use KMS to encrypt Kubernetes Secrets.