All Products
Search
Document Center

Container Service for Kubernetes:Import Alibaba Cloud KMS service credentials by using ack-secret-manager

Last Updated:Mar 26, 2026

ack-secret-manager syncs secrets from Key Management Service (KMS) to your Container Service for Kubernetes (ACK) cluster as Kubernetes Secrets. Applications mount these Secrets through the file system to access up-to-date KMS credentials without managing them directly.

image

How it works

Two custom resource definitions (CRDs) drive the sync process:

  • SecretStore — defines how ack-secret-manager authenticates to KMS

  • ExternalSecret — defines which KMS secrets to sync and where to store them in the cluster

When ack-secret-manager processes an ExternalSecret, it reads the referenced SecretStore for authentication credentials, retrieves the specified KMS secret, and creates or updates a Kubernetes Secret with the same name and namespace as the ExternalSecret.

Security considerations

Before enabling secret synchronization, assess these risks:

  • File system traversal: Secrets mounted via the file system can be exposed if an attacker exploits CVE vulnerabilities to traverse cluster directories.

  • Debug and log exposure: Misconfigured debug breakpoints or overly permissive log access can leak Secrets. Avoid referencing Secrets through environment variables.

  • Excessive permissions: Apply the least privilege principle when granting ack-secret-manager access to KMS.

If your application doesn't need to persist secrets locally, grant pods minimal permissions via RRSA and call GetSecretValue directly from the application. This avoids exposing secrets in the pod file system or Kubernetes Secrets.

Prerequisites

Before you begin, ensure that you have:

Step 1: Install ack-secret-manager

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the target cluster name. In the left navigation pane, choose Applications > Helm.

  3. On the Helm page, click Deploy. In the Chart section of the Deploy panel, find and select ack-secret-manager, then click Next. In the Confirm dialog, the chart installs to the kube-system namespace by default. Click Yes to proceed. To use a custom namespace, configure Application Name and Namespace in the Basic Information step.

  4. In the Parameters step, select the latest chart version and configure the following parameters as needed, then click OK. RRSA authentication (recommended): Set rrsa.enable to true to enable RAM Roles for Service Accounts (RRSA). This allows ack-secret-manager to authenticate using a service account identity without managing AccessKey pairs. Scheduled sync: Configure the following parameters to enable periodic secret synchronization. Throttling (for clusters with many ExternalSecrets): Set command.maxConcurrentKmsSecretPulls to limit concurrent KMS secret pulls per second. The default value is 10. Without this limit, a large number of ExternalSecrets can trigger throttling on the KMS or RAM side. KMS endpoint: Set command.kmsEndpoint to specify a global KMS endpoint for all secret requests. This is a global configuration; you can also configure endpoints at the individual secret level. For details, see Configure KMS endpoint addresses.

    Parameter Description
    command.disablePolling Whether to disable periodic polling. Set to false to enable periodic sync.
    command.pollingInterval Sync frequency. For example, 120s syncs secrets every 2 minutes.

    image

    image

    image

    image

After installation, the ACK console redirects to the ack-secret-manager page. Verify the installation by checking that the expected resources are created.

image.png

Step 2: Authorize ack-secret-manager to access KMS

Create a SecretStore to define how ack-secret-manager authenticates to KMS. Without a SecretStore, ack-secret-manager cannot sync secrets from KMS.

Choose the authorization method based on your cluster type:

Method Applicable cluster types Key advantage
RRSA (recommended) ACK managed clusters, ACK Serverless clusters (Kubernetes 1.22+) No AccessKey pairs to manage
Worker RAM role ACK managed clusters, ACK dedicated clusters, registered clusters Simple setup; no additional credentials
AccessKey pair All cluster types Works universally

Use RRSA to grant permissions

RRSA enables pod-level permission control without AccessKey pairs. It applies to ACK managed clusters and ACK Serverless clusters running Kubernetes 1.22 or later.

  1. Enable RRSA for the ACK cluster in the ACK console. This creates an OpenID Connect (OIDC) identity provider (IdP) for the cluster. For details, see Enable RRSA.

  2. Create a RAM role with the IdP as the trusted entity. Set Principal Type to Identity Provider and configure the following parameters. For details, see Create a RAM role for an OIDC IdP.

    Parameter Value
    Identity provider type Select OIDC
    Identity provider Select ack-rrsa-<cluster_id>, where <cluster_id> is your cluster ID
    Condition — oidc:iss Keep the default
    Condition — oidc:aud Keep the default
    Condition — oidc:sub Add manually: Key = oidc:sub, Operator = StringEquals, Value = system:serviceaccount:<namespace>:<serviceAccountName>. For example: system:serviceaccount:kube-system:ack-secret-manager. If you installed ack-secret-manager in a namespace other than kube-system, use the actual namespace.
  3. Create a custom RAM policy that grants access to KMS secrets, then attach it to the RAM role.

    1. Create a custom policy with the following content. For details, see Create custom policies. ``json { "Action": [ "kms:GetSecretValue", "kms:Decrypt" ], "Resource": [ "*" ], "Effect": "Allow" } ``

    2. Attach the policy to the RAM role. For details, see Grant permissions to a RAM role.

  4. Create a SecretStore using RRSA.

    1. Create a file named secretstore-rrsa.yaml with the following content. Replace the placeholders as described.

      Placeholder Description
      {accountID} ID of the Alibaba Cloud account used to sync KMS secrets
      {clusterID} ID of your ACK cluster
      {roleName} Name of the RAM role created in step 2
      apiVersion: 'alibabacloud.com/v1alpha1'
      kind: SecretStore
      metadata:
        name: scdemo-rrsa
      spec:
        KMS:
          KMSAuth:
            oidcProviderARN: "acs:ram::{accountID}:oidc-provider/ack-rrsa-{clusterID}"
            ramRoleARN: "acs:ram::{accountID}:role/{roleName}"
    2. Apply the SecretStore: ``bash kubectl apply -f secretstore-rrsa.yaml ``

Grant permissions to the worker RAM role

This method applies to ACK managed clusters, ACK dedicated clusters, and registered clusters. It does not apply to ACK Serverless clusters, which have no worker RAM role.

  1. Create a custom RAM policy with the following content. For details, see Create custom policies.

    {
      "Action": [
        "kms:GetSecretValue",
        "kms:Decrypt"
      ],
      "Resource": [
        "*"
      ],
      "Effect": "Allow"
    }
  2. Attach the policy to the worker RAM role of the cluster. For details, see Grant permissions to the worker RAM role.

    When using the worker RAM role, you can omit the secretStoreRef field in the ExternalSecret.

Specify an AccessKey pair to assume a RAM role

This method applies to all ACK cluster types.

  1. Create a RAM role with your Alibaba Cloud account as the trusted entity. For details, see Create a RAM role for a trusted Alibaba Cloud account.

    When selecting the trusted Alibaba Cloud account, select Current Account.
  2. Create a custom RAM policy that grants access to KMS secrets, then attach it to the RAM role.

    1. Create a custom policy with the following content. For details, see Create custom policies. ``json { "Action": [ "kms:GetSecretValue", "kms:Decrypt" ], "Resource": [ "*" ], "Effect": "Allow" } ``

    2. Attach the policy to the RAM role. For details, see Grant permissions to a RAM role.

  3. Create a custom RAM policy that allows a RAM user to assume the RAM role, then attach it to the RAM user.

    1. Create a policy with the following content. Replace <account-id> with the Alibaba Cloud account ID that owns the RAM role, and replace <role-name> with the RAM role name. For details on finding the role ARN, see How do I view the ARN of a RAM role?

      {
          "Statement": [
              {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Resource": "acs:ram:*:<account-id>:role/<role-name>"
              }
          ],
          "Version": "1"
      }
    2. Attach the policy to the RAM user. For details, see Grant permissions to RAM users.

  4. Create a Kubernetes Secret to store the RAM user's AccessKey pair.

    1. Create a file named ramuser.yaml with the following content. Replace the AccessKey ID and AccessKey Secret with their Base64-encoded values.

      apiVersion: v1
      data:
        accessKey: {AccessKey ID encoded in Base64}
        accessKeySecret: {AccessKey secret encoded in Base64}
      kind: Secret
      metadata:
        name: ramuser
        namespace: kube-system
      type: Opaque
    2. Apply the Secret: ``bash kubectl apply -f ramuser.yaml ``

  5. Create a SecretStore using the AccessKey pair.

    1. Create a file named secretstore-ramrole.yaml with the following content. Replace the placeholders as described.

      Placeholder Description
      {accountID} ID of the Alibaba Cloud account used to sync KMS secrets
      {roleName} Name of the RAM role created in step 1
      {secretName} Name of the Kubernetes Secret storing the AccessKey pair
      {secretNamespace} Namespace of the Kubernetes Secret
      {secretKey} Key in the Kubernetes Secret that stores the AccessKey value
      {roleSessionName} Role session name — any custom string
      apiVersion: 'alibabacloud.com/v1alpha1'
      kind: SecretStore
      metadata:
        name: scdemo-ramrole
      spec:
        KMS:
          KMSAuth:
            accessKey:
              name: {secretName}
              namespace: {secretNamespace}
              key: {secretKey}
            accessKeySecret:
              name: {secretName}
              namespace: {secretNamespace}
              key: {secretKey}
            ramRoleARN: "acs:ram::{accountID}:role/{roleName}"
            ramRoleSessionName: {roleSessionName}
    2. Apply the SecretStore: ``bash kubectl apply -f secretstore-ramrole.yaml ``

Step 3: Sync KMS secrets to the cluster

Create an ExternalSecret to specify which KMS secrets to sync. ack-secret-manager creates a Kubernetes Secret with the same name and namespace as the ExternalSecret.

The namespace and name of the resulting Kubernetes Secret match the ExternalSecret exactly.
  1. Create a file named external.yaml with the following content. Replace the placeholders as described.

    Placeholder Description
    {KMS secret name} Required. Name of the KMS secret to sync.
    {Kubernetes secret key} Required. Key name in the Kubernetes Secret where the KMS secret value is stored.
    {KMS secret version stage} Optional. Version stage label of the KMS secret, such as ACSCurrent. To use a version ID instead, replace versionStage with versionId. RDS, PolarDB, Redis/Tair, RAM, and ECS secrets only support ACSPrevious and ACSCurrent and ignore versionId. For details, see Secret elements.
    {KMS Service endpoint address} Optional. KMS request endpoint. Overrides global and default endpoint configurations. For supported endpoint formats, see Configure KMS endpoint addresses.
    {secret store name} Optional. Name of the SecretStore created in step 2. Not required when using the worker RAM role.
    {secret store namespace} Optional. Namespace of the SecretStore. Not required when using the worker RAM role.
    apiVersion: 'alibabacloud.com/v1alpha1'
    kind: ExternalSecret
    metadata:
      name: esdemo
    spec:
      provider: kms # Sync from KMS. Can be omitted or set to kms.
      data:
        - key: {KMS secret name}
          name: {Kubernetes secret key}
          versionStage: {KMS secret version stage}
          kmsEndpoint: {KMS Service endpoint address}
          secretStoreRef: # Omit when using the worker RAM role.
            name: {secret store name}
            namespace: {secret store namespace}
  2. Apply the ExternalSecret:

    kubectl apply -f external.yaml
  3. Run the following command to check whether a Kubernetes Secret exists:

    kubectl get secret esdemo

    If a Kubernetes Secret exists, the KMS secret is synchronized from KMS to your cluster.

Advanced configurations

Sync KMS secrets across Alibaba Cloud accounts

Use this configuration when the KMS instance belongs to one Alibaba Cloud account (Account A) but the ACK cluster belongs to another (Account B). ack-secret-manager uses RRSA to assume a RAM role in Account A via an OIDC provider in Account B, then imports the KMS secret into the cluster.

Configure permissions for Account A

  1. Use Account A to create a RAM role with Account B as the trusted entity. Select Other Account and enter the ID of Account B. For details, see Create a RAM role for a trusted Alibaba Cloud account.

  2. Create a custom RAM policy that grants access to KMS secrets:

    {
        "Action": [
           "kms:GetSecretValue",
           "kms:Decrypt"
        ],
        "Resource": [
            "*"
        ],
        "Effect": "Allow"
    }
  3. Attach the policy to the RAM role. For details, see Grant permissions to a RAM role.

Configure permissions for Account B

  1. Enable RRSA for the ACK cluster in the ACK console. When installing ack-secret-manager, set rrsa.enable to true. For details, see Enable RRSA.

  2. Create a RAM role in Account B with the OIDC IdP as the trusted entity. Set Principal Type to Identity Provider and configure the following parameters. For details, see Create a RAM role for an OIDC IdP.

    Parameter Value
    Identity provider type Select OIDC
    Identity provider Select ack-rrsa-<cluster_id>, where <cluster_id> is your cluster ID
    Condition — oidc:iss Keep the default
    Condition — oidc:aud Keep the default
    Condition — oidc:sub Add manually: Key = oidc:sub, Operator = StringEquals, Value = system:serviceaccount:kube-system:ack-secret-manager
  3. Create a custom RAM policy in Account B that allows assuming the RAM role in Account A, then attach it to the Account B RAM role.

    1. Create a policy with the following content. Replace <account-id> with the ID of Account A and <role-name> with the RAM role created in Account A. For details on finding the ARN, see How do I view the ARN of a RAM role?

      {
        "Statement": [
          {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "acs:ram:*:<account-id>:role/<role-name>"
          }
        ],
        "Version": "1"
      }
    2. Attach the policy to the RAM role created in Account B.

  4. Create a SecretStore for cross-account access.

    1. Create a file named secretstore-ramrole.yaml with the following content. Replace the placeholders as described.

      Placeholder Description
      {ACK-accountID} ID of Account B (the account with the ACK cluster)
      {clusterID} ID of your ACK cluster
      {ACK-roleName} Name of the RAM role created in Account B
      {KMS-accountID} ID of Account A (the account with the KMS instance)
      {KMS-roleName} Name of the RAM role created in Account A
      {roleSessionName} Role session name — any custom string
      apiVersion: 'alibabacloud.com/v1alpha1'
      kind: SecretStore
      metadata:
        name: scdemo-cross-account
      spec:
        KMS:
          KMSAuth:
            oidcProviderARN: "acs:ram::{ACK-accountID}:oidc-provider/ack-rrsa-{clusterID}"
            ramRoleARN: "acs:ram::{ACK-accountID}:role/{ACK-roleName}"
            remoteRamRoleARN: "acs:ram::{KMS-accountID}:role/{KMS-roleName}"
            remoteRamRoleSessionName: {roleSessionName}
  5. Create an ExternalSecret to specify the sync details. For details, see Step 3: Sync KMS secrets to the cluster.

Parse JSON and YAML secrets

Extract specific fields from a JSON secret

Use jmesPath to extract specific key-value pairs from a KMS secret stored in JSON format.

For example, given the following JSON secret stored in KMS Secrets Manager:

{"name":"tom","friends":[{"name":"lily"},{"name":"mark"}]}

Create an ExternalSecret with jmesPath to extract specific fields:

  • path: JMESPath expression to extract the field. Required. See the JMESPath specification.

  • objectAlias: Key name in the resulting Kubernetes Secret.

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: es-json-demo
spec:
  provider: kms
  data:
    - key: {KMS secret name}
      versionStage: {KMS secret version stage}
      secretStoreRef:
        name: {secret store name}
        namespace: {secret store namespace}
      jmesPath:
        - path: "name"              # Extracts: "tom"
          objectAlias: "myname"
        - path: "friends[0].name"   # Extracts: "lily"
          objectAlias: "friendname"

Auto-parse a JSON secret

If the JSON structure is unknown or variable, use dataProcess.extract to automatically parse all key-value pairs from a JSON secret. Use dataProcess.replaceRule to rename keys that don't conform to Kubernetes Secret key naming requirements.

For example, given the following JSON secret with non-conforming key names:

{"/name-invalid":"lily","name-invalid/":[{"name":"mark"}]}
apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: extract-secret
spec:
  provider: kms
  dataProcess:
    - extract:
        key: {KMS secret name}
        versionStage: ACSCurrent
        secretStoreRef:
          name: {secret store name}
          namespace: {secret store namespace}
      replaceRule:
        - source: "^/.*d$"   # Keys starting with / and ending with d -> "tom"
          target: "tom"
        - source: "^n.*/$"   # Keys starting with n and ending with / -> "mark"
          target: "mark"

Extract specific fields from a YAML secret

Use jmesPath to extract specific key-value pairs from a KMS secret stored in YAML format.

For example, given the following YAML secret stored in KMS Secrets Manager:

name: tom
friends:
  - name: lily
  - name: mark
apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: es-yaml-demo
spec:
  provider: kms
  data:
    - key: {KMS secret name}
      versionStage: {KMS secret version stage}
      secretStoreRef:
        name: {secret store name}
        namespace: {secret store namespace}
      jmesPath:
        - path: "name"              # Extracts: "tom"
          objectAlias: "myname"
        - path: "friends[0].name"   # Extracts: "lily"
          objectAlias: "friendname"

Auto-parse a YAML secret

If the YAML structure is unknown or variable, use dataProcess.extract to auto-parse all key-value pairs. Use dataProcess.replaceRule to handle non-conforming key names.

For example, given the following YAML secret:

/name-invalid: lily
name-invalid/:
  - name: mark
apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: extract-secret
spec:
  provider: kms
  dataProcess:
    - extract:
        key: {KMS secret name}
        versionStage: ACSCurrent
        secretStoreRef:
          name: {secret store name}
          namespace: {secret store namespace}
      replaceRule:
        - source: "^/.*d$"   # Keys starting with / and ending with d -> "tom"
          target: "tom"
        - source: "^n.*/$"   # Keys starting with n and ending with / -> "mark"
          target: "mark"

Configure KMS endpoint addresses

ack-secret-manager resolves the KMS endpoint using the following priority order:

Type Field Priority Description
Secret-level ExternalSecret.spec.data.kmsEndpoint Highest Applies to a specific secret; overrides global and default configurations.
Global command.kmsEndpoint (startup parameter) Medium Applies to all KMS requests that don't have a secret-level endpoint.
Default None Lowest Uses kms-vpc.{region}.aliyuncs.com, where {region} is the region of the KMS secret.

The following example shows how secret-level and global endpoints interact:

apiVersion: "alibabacloud.com/v1alpha1"
kind: ExternalSecret
metadata:
  name: esdemo
spec:
  provider: kms
  data:
    - key: test-hangzhou
      name: hangzhou-vpc
      versionId: v1
      # No kmsEndpoint: uses global config if set, otherwise uses kms-vpc.{region}.aliyuncs.com
    - key: test-hangzhou
      name: hangzhou-public
      versionId: v1
      kmsEndpoint: kms.cn-hangzhou.aliyuncs.com  # Overrides global and default

Supported endpoint formats:

Gateway type Domain type Endpoint format Requirements
Dedicated gateway KMS private domain {kms-instance-id}.cryptoservice.kms.aliyuncs.com KMS secret instance and cluster in the same region and VPC. KMS instance version 3.0 or later.
Shared gateway VPC domain kms-vpc.{region}.aliyuncs.com KMS secret and cluster in the same region. This is the default endpoint.
Shared gateway Public domain kms.{region}.aliyuncs.com Cluster can access the public internet.

For more information, see Differences between shared and dedicated gateways for accessing KMS.

What's next

To protect Kubernetes Secrets synced from KMS and stored in etcd, enable Secret encryption. For details, see Use KMS to encrypt Kubernetes Secrets.