Mount KMS Secrets in ACK with CSI Secrets Store - Container Service for Kubernetes

Use the CSI Secrets Store provider for Alibaba Cloud to import KMS credentials

The CSI Secrets Store provider for Alibaba Cloud (csi-secrets-store-provider-alibabacloud) lets you mount secrets from Key Management Service (KMS) directly into pods as files via Container Storage Interface (CSI) inline volumes. It also synchronizes KMS secrets to Kubernetes Secrets and supports automatic secret rotation, so your applications always use fresh credentials without manual updates.

Capabilities:

Mount KMS secrets as read-only files in pods via CSI inline volumes
Sync KMS secrets to Kubernetes Secrets
Automatically rotate secrets on a configurable schedule

Limitations:

ACK Serverless clusters are not supported. Use ack-secret-manager instead.
Cluster and KMS secrets must be in the same region.
Requires Kubernetes 1.20 or later.

Security considerations

Before you deploy this plug-in, understand the following security risks:

File system traversal: When secrets are accessible through the file system, attackers can exploit CVE (Common Vulnerabilities and Exposures) vulnerabilities in applications to traverse pod directories and read secrets.
Debugging and logging exposure: Debugging breakpoints or misconfigured log permissions can expose secrets stored in the pod file system.
Environment variable risks: Do not use environment variables to mount secrets. Environment variables are accessible to all processes in a container and are difficult to rotate securely.
Least privilege for secret synchronization: If you enable secret synchronization to Kubernetes Secrets, apply the principle of least privilege strictly to limit access permissions.

If your application does not need to persist secrets to disk, use RRSA to authorize pods and call the GetSecretValue API directly from application code. This approach avoids exposing secrets in the pod file system or as Kubernetes Secrets.

Prerequisites

Before you begin, make sure that you have:

An ACK managed cluster, ACK dedicated cluster, or registered cluster running Kubernetes 1.20 or later
The cluster and your KMS secrets in the same region
kubectl connected to the cluster

For cluster creation, see Create an ACK managed cluster and Create a registered cluster.

Step 1: Configure authentication

The plug-in needs permissions to read secrets from KMS. Choose an authentication method based on your cluster type and Kubernetes version.

Method	Applicable cluster types	Kubernetes version	Recommendation
RRSA	ACK managed cluster	1.22 or later	Recommended. Isolates permissions at the pod level; no AccessKey credentials needed.
Worker RAM role	ACK managed cluster, ACK dedicated cluster, registered cluster	1.20 or later	Simple to configure; permissions apply to all pods on the node.
AccessKey pair	All cluster types	1.20 or later	Use when RRSA and worker RAM role are not applicable.

Grant permissions using RRSA

RAM Roles for Service Accounts (RRSA) isolates permissions at the pod level and avoids the need to manage AccessKey IDs and AccessKey secrets. Use this method when your cluster is an ACK managed cluster running Kubernetes 1.22 or later.

Enable RRSA in the ACK console. For more information, see Enable RRSA.

Create a RAM role for an OIDC identity provider (IdP) so the plug-in can assume it.

Log on to the RAM console. > Note: Use an Alibaba Cloud account or a RAM user with the AdministratorAccess policy. For more information, see Create a RAM user as an account administrator.
In the left-side navigation pane, choose Identities > Roles.
On the Roles page, click Create Role.
In the upper-right corner of the Create Role page, click Switch to Policy Editor.

Configure the role with the following parameters, then click OK.

Parameter	Value
RAM Role Name	A name of your choice
Note	(Optional) A description
IdP Type	OIDC
Select IdP	`ack-rrsa-<cluster_id>`, where `<cluster_id>` is your cluster ID
oidc:iss	Use the default value
oidc:aud	`sts.aliyuncs.com`
oidc:sub	Set the condition operator to StringEquals and the value to `system:serviceaccount:<namespace>:<serviceAccountName>`. For the plug-in installed in the default namespace, use `system:serviceaccount:kube-system:csi-secrets-store-provider-alibabacloud`.

Note: Install the plug-in in the kube-system namespace. If you install it in a different namespace, replace kube-system in the oidc:sub value accordingly.

Create a custom policy that grants access to KMS secrets, then attach it to the RAM role. The policy requires the following permissions. For more information, see Create custom policies and Grant permissions to a RAM role.
```
{
    "Action": [
       "kms:GetSecretValue",
       "kms:Decrypt"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
}
```
Create a Kubernetes Secret named alibaba-credentials in the kube-system namespace.
1. Create a file named secretstore-rrsa.yaml with the following content. Replace {rolearn} with the Base64-encoded ARN of the RAM role created in step 2, and {oidcproviderarn} with the Base64-encoded provider ARN generated when you enabled RRSA.
```
apiVersion: v1 kind: Secret metadata: name: alibaba-credentials namespace: kube-system type: Opaque data: rolearn: {rolearn}           # Base64-encoded RAM role ARN oidcproviderarn: {oidcproviderarn}  # Base64-encoded OIDC provider ARN
```
2. Apply the Secret:
```
kubectl apply -f secretstore-rrsa.yaml
```

Grant permissions to the worker RAM role

This method attaches KMS permissions directly to the worker node's RAM role. It applies to ACK managed clusters, ACK dedicated clusters, and registered clusters. No additional Kubernetes Secret is required.

Create a custom policy with the following content. For more information, see Create custom policies.

{
  "Action": [
    "kms:GetSecretValue",
    "kms:Decrypt"
  ],
  "Resource": [
    "*"
  ],
  "Effect": "Allow"
}

Attach the policy to the cluster's worker RAM role. For more information, see Grant permissions to the worker RAM role.

Specify an AccessKey pair to assume a RAM role

This method works with all ACK cluster types. It uses a RAM user's AccessKey pair to assume a dedicated RAM role that has KMS access.

Create a RAM role for a trusted Alibaba Cloud account. For more information, see Create a RAM role for a trusted Alibaba Cloud account.
Create a custom policy that grants KMS access and attach it to the RAM role.
```
{
  "Action": [
    "kms:GetSecretValue",
    "kms:Decrypt"
  ],
  "Resource": [
    "*"
  ],
  "Effect": "Allow"
}
```
For more information, see Create custom policies and Grant permissions to a RAM role.
Create a custom policy that lets a RAM user assume the role, then attach it to the RAM user. Replace acs:ram::*:role/** with the ARN of the RAM role created in step 1.
```
{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "acs:ram::***:role/****"
        }
    ],
    "Version": "1"
}
```
For more information, see Grant permissions to RAM users.

Create a Kubernetes Secret named alibaba-credentials in the kube-system namespace.

Create a file named alibaba-credentials.yaml with the following content. Replace each placeholder with the corresponding Base64-encoded value.

Placeholder	Value
`{rolearn}`	Base64-encoded ARN of the RAM role created in step 1
`{ak}`	Base64-encoded AccessKey ID of the RAM user
`{sk}`	Base64-encoded AccessKey secret of the RAM user

apiVersion: v1 kind: Secret metadata: name: alibaba-credentials namespace: kube-system type: Opaque data: id: {ak}          # Base64-encoded AccessKey ID secret: {sk}      # Base64-encoded AccessKey secret rolearn: {rolearn} # Base64-encoded RAM role ARN

Apply the Secret:

kubectl apply -f alibaba-credentials.yaml

Step 2: Install the plug-in

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the target cluster. In the left-side navigation pane, choose Applications > Helm.
On the Helm page, click Deploy. In the Chart section, search for csi-secrets-store-provider-alibabacloud, keep the default settings for other parameters, then click Next. Confirm that the component is deployed in the kube-system namespace under the component name. To use a custom name and namespace, configure them as prompted.

Set Chart Version to the latest version. In the Parameters section, configure the parameters based on the authentication method you chose in step 1, then click OK.

RRSA: Set rrsa.enable to true and configure the following parameters.

envVarsFromSecret: #  ACCESS_KEY_ID:          # Not required for RRSA #    secretKeyRef: alibaba-credentials #    key: id #  SECRET_ACCESS_KEY:      # Not required for RRSA #    secretKeyRef: alibaba-credentials #    key: secret ALICLOUD_ROLE_ARN:        # RAM role ARN for RRSA secretKeyRef: alibaba-credentials key: rolearn #  ALICLOUD_ROLE_SESSION_NAME: #    secretKeyRef: alibaba-credentials #    key: rolesessionname #  ALICLOUD_ROLE_SESSION_EXPIRATION: #    secretKeyRef: alibaba-credentials #    key: rolesessionexpiration ALICLOUD_OIDC_PROVIDER_ARN:  # OIDC provider ARN for RRSA secretKeyRef: alibaba-credentials key: oidcproviderarn

Worker RAM role: Use the default parameter settings. No additional configuration is required.

AccessKey pair: Configure the following parameters.

envVarsFromSecret: ACCESS_KEY_ID:           # RAM user AccessKey ID secretKeyRef: alibaba-credentials key: id SECRET_ACCESS_KEY:       # RAM user AccessKey secret secretKeyRef: alibaba-credentials key: secret ALICLOUD_ROLE_ARN:       # RAM role to assume secretKeyRef: alibaba-credentials key: rolearn #  ALICLOUD_ROLE_SESSION_NAME: #    secretKeyRef: alibaba-credentials #    key: rolesessionname #  ALICLOUD_ROLE_SESSION_EXPIRATION: #    secretKeyRef: alibaba-credentials #    key: rolesessionexpiration #  ALICLOUD_OIDC_PROVIDER_ARN:  # Not required for AccessKey method #    secretKeyRef: alibaba-credentials #    key: oidcproviderarn

Secret rotation (optional): To enable automatic synchronization of updated KMS secrets to your pods, configure the following parameters.

Parameter	Description
`secrets-store-csi-driver.enableSecretRotation`	Set to `true` to enable automatic secret rotation.
`secrets-store-csi-driver.rotationPollInterval`	`secrets-store-csi-driver.rotationPollInterval` How often the driver checks for updated secrets and remounts them. In this example, this parameter is set to `120` seconds, which specifies that the secrets are synchronized every 2 minutes. You can adjust the value as needed.

After installation, verify that the plug-in is running. The ACK console redirects you to the csi-secrets-store-provider-alibabacloud page, where all expected resources should be listed.

Step 3: Configure data synchronization

Use a SecretProviderClass resource to tell the plug-in which KMS secrets to mount and how.

SecretProviderClass parameters

All SecretProviderClass resources use the following base structure:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: <NAME>
spec:
  provider: alibabacloud   # Must be set to alibabacloud.
  parameters:

The parameters section supports the following fields:

Parameter	Required	Description
`objects`	Yes	YAML configuration that defines which secrets to mount. See the subfields below.
`region`	No	Region of the KMS Secrets Manager server. If not set, the plug-in uses the region of the current node. For high pod counts, set this explicitly to reduce performance overhead.
`pathTranslation`	No	Character used to replace path separators in secret names. For example, `My/Path/Secret` becomes `My_Path_Secret`. Default: underscore (`_`). Set to `"False"` to disable replacement.

The objects field supports the following subfields:

Subfield	Required	Description
`objectName`	Yes	Name of the secret in KMS Secrets Manager. See SecretName.
`objectType`	No	Source service. Valid values: `kms`, `oos`. Default: `kms`.
`objectAlias`	No	File name used when mounting the secret to a pod. Defaults to the value of `objectName`.
`objectVersion`	No	Maps to the `VersionId` parameter in KMS Secrets Manager.
`objectVersionLabel`	No	Maps to the `VersionStage` parameter in KMS Secrets Manager.
`jmesPath`	No	Extracts specific key-value pairs from JSON-formatted secrets. Requires the `path` subfield (JMESPath expression) and optionally `objectAlias` (mounted file name).

Example: extracting keys from a JSON secret

If a KMS secret contains:

{
  "username": "testuser",
  "password": "testpassword"
}

Use jmesPath to mount username and password as separate files:

objects: |
  - objectName: "MySecret"
    jmesPath:
      - path: "username"        # JMESPath expression to extract the value
        objectAlias: "username" # File name in the pod mount
      - path: "password"
        objectAlias: "password"

Mount a KMS secret to a pod

This example mounts a KMS secret named test into pods running in an ACK managed cluster in the same region as the secret.

Create secretstore.yaml with the following content:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: test-secrets
spec:
  provider: alibabacloud   # Must be set to alibabacloud.
  parameters:
    objects: |
      - objectName: "test"   # Name of the secret in KMS Secrets Manager
        objectType: "kms"    # Valid values: kms, oos. Default: kms.

Apply the SecretProviderClass:
```
kubectl apply -f secretstore.yaml
```

Create deploy.yaml with the following content. This Deployment mounts the test-secrets SecretProviderClass as a CSI inline volume at /mnt/secrets-store. For more examples, see Deployment examples.

apiVersion: apps/v1  # If the API version is earlier than 1.8.0, use apps/v1beta1.
kind: Deployment
metadata:
  name: nginx-deployment-basic
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "test-secrets"  # Must match the SecretProviderClass name above.
      containers:
        - name: nginx
          image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 # Replace with the actual image.
          ports:
            - containerPort: 80
          resources:
            limits:
              cpu: "500m"
          volumeMounts:
            - name: secrets-store-inline
              mountPath: "/mnt/secrets-store"
              readOnly: true

Apply the Deployment:
```
kubectl apply -f deploy.yaml
```
Verify that the secret is mounted. Run the following commands, replacing <pod-name> with the name of a running pod from the Deployment:
```
# List secrets mounted at the volume path
kubectl exec <pod-name> -- ls /mnt/secrets-store/

# Display the content of a mounted secret file
kubectl exec <pod-name> -- cat /mnt/secrets-store/test
```
The output of cat should display the ciphertext stored in KMS for the test secret.

What's next

To import KMS secrets into ACK Serverless clusters, see Use ack-secret-manager to import secrets from KMS.
To encrypt Kubernetes Secrets cached in your ACK cluster, see Use KMS to encrypt Kubernetes Secrets.