Import Alibaba Cloud KMS service credentials using csi-secrets-store-provider-alibabacloud - Key Management Service

Import KMS credentials as Kubernetes Secrets or mount them via CSI inline volumes with csi-secrets-store-provider-alibabacloud, which also supports automatic secret rotation.

Security risks

Direct interaction between workloads reading secrets from the file system and Secrets Manager can cause compatibility issues. csi-secrets-store-provider-alibabacloud resolves these and supports syncing secrets to Kubernetes Secret objects for use as environment variables. Evaluate the following security risks before proceeding.

If a key is accessible in the file system, vulnerabilities in an application can introduce a directory traversal threat, which can expose sensitive information.
Misconfigured debug endpoints or log permissions can expose keys. Therefore, accessing keys by mounting them as references in environment variables is insecure and not recommended.
When you enable the Secret instance sync feature, you must strictly control access permissions based on the principle of least privilege.

For these reasons, if your application does not require persistent storage for ciphertext, we recommend that you Use RRSA for pod-level access controlconfigure minimum pod-level permissions for the application and GetSecretValueobtain key credentials directly in the application to reduce the risk of key content exposure in the pod's file system or Kubernetes Secrets.

Prerequisites

Create an ACK cluster that meets the following requirements (see Create an ACK managed cluster or Create an ACK One registered cluster).
- The cluster runs Kubernetes 1.20 or later. Supported cluster types include ACK managed cluster, ACK dedicated cluster, ACK Edge cluster, and ACK One registered cluster. ACK Serverless clusters are not supported.
A kubectl client is connected to your cluster.

Step 1: Configure add-on authentication

Configure authentication for csi-secrets-store-provider-alibabacloud to fetch credentials from KMS. Without authentication, csi-secrets-store-provider-alibabacloud cannot import or sync credentials to the cluster. Choose one of the following authorization methods based on your cluster type.

Grant permissions by using RRSA: This method is applicable to ACK managed clusters that run Kubernetes 1.22 or later.
Grant permissions to the worker RAM role of the cluster: This method is applicable to ACK managed clusters, ACK dedicated clusters, and ACK One registered clusters.
Specify an AccessKey ID and secret to assume a specific RAM role: This method is applicable to all ACK clusters.

Grant permissions with RRSA

Compared to other authorization methods, RRSA authorization provides pod-level permission isolation and avoids the credential leakage risk of using AK/SKs directly.

Enable the RRSA feature for your cluster in the ACK console. RRSA uses the identity provider information that was specified when the cluster was created.

To use csi-secrets-store-provider-alibabacloud, create a RAM role with an OIDC identity provider as the trusted entity. The main parameters are shown in the following table.

Parameter	Description
IdP Type	Select OIDC.
Identity Provider	Select ack-rrsa-<cluster_id>. Replace <cluster_id> with the ID of your cluster.
Condition	oidc:iss: Keep the default value. oidc:aud: Keep the default value. oidc:sub: Add this condition. Key: Select oidc:sub. Operator: Select StringEquals. Value: Enter `system:serviceaccount:<namespace>:<serviceAccountName>`. Replace `<namespace>` with the application namespace and `<serviceAccountName>` with the service account name. For this topic, use `system:serviceaccount:kube-system:csi-secrets-store-provider-alibabacloud`. Note We recommend that you install the component in the default `kube-system` namespace. If you install csi-secrets-store-provider-alibabacloud in a different namespace, replace `kube-system` with the name of that namespace.

Create a custom policy and grant permissions to the RAM role from the preceding step.
1. Create the permissions policy for csi-secrets-store-provider-alibabacloud to import KMS credentials (see Create a custom policy). Policy content:
```
{
    "Action": [
       "kms:GetSecretValue",
       "kms:Decrypt"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
}
```
2. Grant permissions to the RAM role from the preceding step. See Manage permissions for a RAM role.
Create a Kubernetes Secret named alibaba-credentials in the cluster. Replace the placeholder values.
1. Create a file named secretstore-rrsa.yaml with the following content. Replace the placeholders with your actual values.
  - {rolearn}: Base64-encoded ARN of the RAM role created in step 2.
  - {oidcproviderarn}: Base64-encoded ARN of the OIDC provider generated after enabling RRSA.
```
apiVersion: v1
data:
  rolearn: {rolearn}
  oidcproviderarn: {oidcproviderarn}
kind: Secret
metadata:
  name: alibaba-credentials
  namespace: kube-system
type: Opaque	                     
```
2. Deploy the Kubernetes Secret:
```
kubectl apply -f secretstore-rrsa.yaml
```

By worker RAM role

Applicable to ACK managed clusters, ACK dedicated clusters, and ACK One registered clusters.

Create a custom policy (see Create a custom policy). Policy content:

{
  "Action": [
    "kms:GetSecretValue",
    "kms:Decrypt"
  ],
  "Resource": [
    "*"
  ],
  "Effect": "Allow"
}

Attach the custom policy to the worker RAM role. See Grant permissions to the worker RAM role.

By AccessKey

Applicable to all ACK clusters.

Create a RAM role for a trusted Alibaba Cloud account. See Create a RAM role for an Alibaba Cloud account.
Note
For the trusted entity, select Current Account.
Create a custom policy and grant permissions to the RAM role from the preceding step.
1. Create the permissions policy for KMS credential access (see Create a custom policy). Policy content:
```
{
  "Action": [
    "kms:GetSecretValue",
    "kms:Decrypt"
  ],
  "Resource": [
    "*"
  ],
  "Effect": "Allow"
}
```
2. Grant permissions to the RAM role from the preceding step. See Manage permissions for a RAM role.
Create a custom policy to assume the preceding role, then attach it to a RAM user.
1. Create a custom policy to assume the role (see Create a custom policy). Policy content:
```
{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "acs:ram:*:<account-id>:role/<role-name>" 
        }
    ],
    "Version": "1"
}
```
  In this custom policy, the Resource element specifies the role ARN, where <ACCOUNT_ID> is the Alibaba Cloud account ID and <ROLE_NAME> is the RAM role name.
2. Granting the custom policy to a RAM user allows you to specify which RAM roles the user can assume. For more information, see Manage RAM user permissions.
Create a Kubernetes Secret named alibaba-credentials in the cluster. Replace the placeholder values.
1. Create a file named alibaba-credentials.yaml with the following content. Replace the placeholders with your actual values.
  - {rolearn}: Base64-encoded ARN of the RAM role from step 1.
  - {ak}: Base64-encoded AccessKey ID of the RAM user.
  - {sk} : Base64-encoded AccessKey secret of the RAM user.
```
apiVersion: v1
data:
  id: {ak}
  secret: {sk}
  rolearn: {rolearn}
kind: Secret
metadata:
  name: alibaba-credentials
  namespace: kube-system
type: Opaque
```
2. Deploy the Kubernetes Secret:
```
kubectl apply -f alibaba-credentials.yaml
```

Step 2: Install the csi-secrets-store-provider-alibabacloud add-on

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Helm.
On the Helm page, click Deploy. Search for and select csi-secrets-store-provider-alibabacloud. Keep default settings and click Next.
Confirm the settings. The add-on installs in the default kube-system namespace with the Helm release named after the add-on. To customize the application name or namespace, configure the parameters as prompted.
Select the latest Chart Version. In the Parameters section, configure parameters based on the authentication method from step 1. Then, click OK.
- If you selected Grant permissions with RRSA, set rrsa.enable to true.
  Other parameters:
```
envVarsFromSecret:
#  ACCESS_KEY_ID:
#    secretKeyRef: alibaba-credentials
#    key: id
#  SECRET_ACCESS_KEY:
#    secretKeyRef: alibaba-credentials
#    key: secret
  ALICLOUD_ROLE_ARN:
    secretKeyRef: alibaba-credentials
    key: rolearn
#  ALICLOUD_ROLE_SESSION_NAME:
#    secretKeyRef: alibaba-credentials
#    key: rolesessionname
#  ALICLOUD_ROLE_SESSION_EXPIRATION:
#    secretKeyRef: alibaba-credentials
#   key: rolesessionexpiration
  ALICLOUD_OIDC_PROVIDER_ARN:
    secretKeyRef: alibaba-credentials
    key: oidcproviderarn
```
- If you selected Grant permissions to the worker RAM role of the cluster, use the default settings.
- If you selected Specify an AccessKey ID and secret to assume a specific RAM role, configure the following parameters:
```
envVarsFromSecret:
  ACCESS_KEY_ID:
    secretKeyRef: alibaba-credentials
    key: id
  SECRET_ACCESS_KEY:
    secretKeyRef: alibaba-credentials
    key: secret
  ALICLOUD_ROLE_ARN:
    secretKeyRef: alibaba-credentials
    key: rolearn
#  ALICLOUD_ROLE_SESSION_NAME:
#    secretKeyRef: alibaba-credentials
#    key: rolesessionname
#  ALICLOUD_ROLE_SESSION_EXPIRATION:
#    secretKeyRef: alibaba-credentials
#    key: rolesessionexpiration
#  ALICLOUD_OIDC_PROVIDER_ARN:
#    secretKeyRef: alibaba-credentials
#    key: oidcproviderarn
```
- To sync secrets to Kubernetes Secret objects, configure the following parameters.
```
syncSecret:
  enabled: true
```
  secrets-store-csi-driver.syncSecret.enabled: Syncs secrets as Kubernetes Secrets. When true, deploys the required RBAC Role and RoleBinding.
- To enable automatic secret rotation, configure the following parameters:
  - secrets-store-csi-driver.enableSecretRotation: Enables automatic secret rotation. Set to true.
  - secrets-store-csi-driver.rotationPollInterval: Secret synchronization interval. Example: 120s syncs every two minutes. Adjust based on your requirements.
After deployment, you are redirected to the csi-secrets-store-provider-alibabacloud page. Verify that all resources are created. On the Resources page of the console, view the Kubernetes resources created by the add-on, including secrets-store-csi-driver and csi-secrets-store-provider-alibabacloud (ServiceAccount), secretproviderclasses-admin-role, secretproviderclasses-viewer-role, and secretproviderclasses-role (ClusterRole), secretproviderclasses-rolebinding (ClusterRoleBinding), secrets-store-csi-driver and csi-secrets-store-provider-alibabacloud (DaemonSet), and secrets-store.csi.k8s.io (CSIDriver).

Step 3: Configure data synchronization

After configuring authentication, use a SecretProviderClass to specify the KMS credentials to import.

Template

A SecretProviderClass uses the following format.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: <NAME>
spec:
  provider: alibabacloud   # This value is fixed to 'alibabacloud'.
  parameters:
    objects: |
      - objectName: <KMS Encryption Parameter Name> # Name of the KMS credential.
        objectType: kms # Fixed to kms when syncing KMS credentials.

The parameters field supports the following mount settings:

Parameter	Required	Description
objects	Yes	Specifies the secrets to mount as a YAML array. Example: `parameters: objects: \| - objectName: "MySecret" objectType: "kms"` The `objects` parameter contains the following sub-fields: `objectName`: Required. The secret name in KMS Secrets Manager. See SecretName. `objectType`: Optional. Alibaba Cloud service type. Valid values: `kms`, `oos`. Default: `kms`. Omit when importing from KMS. `objectAlias`: Optional. Filename for the mounted credential in the Pod. Defaults to `objectName`. `objectVersion`: Optional. Maps to the VersionId parameter in Secrets Manager. Not supported for RDS, PolarDB, Redis/Tair, RAM, and ECS credentials. `objectVersionLabel`: Optional. Maps to the VersionStage parameter in Secrets Manager. For RDS, PolarDB, Redis/Tair, RAM, and ECS credentials, only ACSPrevious and ACSCurrent versions are retrievable. `jmesPath`: Optional. Parses specific key-value pairs from a JSON-formatted credential. Example: the `test` credential contains: `{ "username": "testuser", "password": "testpassword" }` To mount `username` and `password` as separate files, use the following JMESPath configuration. The `jmesPath` field requires two sub-fields: `path`: Required. The JMESPath expression to extract the target value. `objectAlias`: Required. Filename for the mounted key-value pair. `kmsEndpoint`: Optional. KMS request endpoint. Defaults to the standard endpoint. Supports shared or dedicated gateways, configurable per credential. See kmsEndpoint configuration.
region	Optional	The Secrets Manager region. Defaults to the node region, which may add performance overhead in large-scale deployments. Specify a region explicitly.
pathTranslation	Optional	If a secret name contains a file separator, the mounted file uses this character as the separator. Example: `My/Path/Secret` mounts as `My_Path_Secret`. Default separator: underscore (_). Set to `"False"` to preserve path separators in filenames.

Usage example

Use a SecretProviderClass to import a KMS credential named test from the same region as an ACK managed cluster into a workload.

Create a file named secretstore.yaml with the following content.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: test-secrets
spec:
  provider: alibabacloud   # This value is fixed to alibabacloud.
  parameters:
    objects: | # objectType supports oos and kms. The default is kms.
      - objectName: "test-hangzhou"
        objectType: "kms"
        objectAlias: "hangzhou-public"
        kmsEndpoint:  "kms.{region}.aliyuncs.com"

Deploy the SecretProviderClass:
```
kubectl apply -f secretstore.yaml
```

Create a file named deploy.yaml with the following content.

This NGINX Deployment mounts credentials from the preceding SecretProviderClass via a CSI inline volume to /mnt/secrets-store (Deployment examples).

apiVersion: apps/v1  # For versions earlier than 1.8.0, use apps/v1beta1.
kind: Deployment
metadata:
  name: nginx-deployment-basic
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "test-secrets"
      containers:
        - name: nginx
          image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 # Replace with your actual image.
          ports:
            - containerPort: 80
          resources:
            limits:
              cpu: "500m"
          volumeMounts:
            - name: secrets-store-inline
              mountPath: "/mnt/secrets-store"
              readOnly: true

Deploy the application:
```
kubectl apply -f deploy.yaml
```
Verify that the secret is mounted correctly.
Log on to the Pod and verify that a file matching the secret name exists in /mnt/secrets-store with content matching the KMS credential.

Sync KMS credentials to Kubernetes Secrets

The Secrets Store CSI Driver automatically syncs secrets from external services (KMS or OOS) as native Kubernetes Secret objects, letting applications use external secrets without code changes.

Configuration method: `SecretProviderClass`

You can add the optional secretObjects field to the spec of a SecretProviderClass resource to enable this feature.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: <NAME>
spec:
  provider: alibabacloud   # This value is fixed to alibabacloud. Do not modify.
  parameters:
    objects: |
      - objectName: <KMS Encryption Parameter Name>  # Name of the KMS credential.
        objectType: kms     # Fixed to kms when syncing KMS credentials.
  secretObjects:
  - secretName: <Kubernetes Secret Name>            # Name of the Kubernetes Secret.
    type: <Kubernetes Secret Type>                  # Type of the Kubernetes Secret.
    data:
    - objectName: <parameters.objects.objectName>   # The name from parameters.objects.objectName. Use the alias if specified.
      key: <Kubernetes Secret Data Key>             # The key name in the Kubernetes Secret data.

secretObjects typically includes the following three parameters:

Parameter	Type	Description
secretName	Required	The name of the Kubernetes Secret to create.
type	Required	The Secret type. Valid values: `Opaque`, `kubernetes.io/basic-auth`, `bootstrap.kubernetes.io/token`, `kubernetes.io/dockerconfigjson` , `kubernetes.io/dockercfg`, `kubernetes.io/ssh-auth`, `kubernetes.io/service-account-token` , or `kubernetes.io/tls`.
data	Required	Defines how to map an external secret to a `Secret`'s `data` field. Its subfields include: `objectName`: Required. The name of the secret (`objectName`) that is defined in `parameters.objects`. If an alias is set, specify its alias `objectAlias` instead. `key`: Required. Specifies the key for the secret data in the `Secret`'s `data` field.

Synchronization lifecycle

The synchronization and cleanup of a Secret object are dynamically triggered by a Pod that mounts the corresponding SecretProviderClass:

Creation: The Kubernetes Secret is created only when the first Pod using the SecretProviderClass starts and mounts the volume.
Update: When secrets-store-csi-driver.enableSecretRotation is true, the Kubernetes Secret is periodically refreshed at the secrets-store-csi-driver.rotationPollInterval interval. Otherwise, it is not updated.
Deletion: When the last Pod using the SecretProviderClass is deleted, the corresponding Kubernetes Secret is also deleted.

Example: Sync and inject credentials

Sync a KMS credential to a Kubernetes Secret and inject it as an environment variable into an NGINX Pod.

Create a SecretProviderClass.

Create a file named syncSecret.yaml with the following content.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: alibabacloud-sync-secret
spec:
  provider: alibabacloud
  parameters:
    objects: |
      - objectName: test-kms
        objectAlias: secretalias
        objectType: kms
  secretObjects:
  - secretName: test-sync-secret  # Name of the generated Kubernetes Secret.
    type: Opaque
    data:
    - objectName: secretalias     # Corresponds to objectName or objectAlias.
      key: test                   # Populates the data key of the generated Kubernetes Secret.

Deploy the SecretProviderClass.
```
kubectl apply -f syncSecret.yaml
```

Create an application Pod to trigger the synchronization.

Create a pod-sync-secret.yaml file with the following content. This Pod mounts the preceding SecretProviderClass and attempts to use secretKeyRef to reference the Secret named test-sync-secret that will be generated.

kind: Pod
apiVersion: v1
metadata:
  name: pod-sync-secret
spec:
  containers:
    - name: nginx
      image: nginx:latest
      volumeMounts:
      - name: secrets-store-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
      env:  
      - name: SECRET_TEST
        valueFrom:
          secretKeyRef:
            name: test-sync-secret
            key: test
  volumes:
    - name: secrets-store-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "alibabacloud-sync-secret"

Deploy the Pod to trigger synchronization.
```
kubectl apply -f pod-sync-secret.yaml
```
Verify the result.
1. Check that the Kubernetes Secret was created.
```
kubectl get secret test-sync-secret
```
  The command output shows the test-sync-secret Kubernetes Secret.
2. Check that the environment variable was injected into the Pod.
```
kubectl exec -it $(kubectl get pods | awk '/pod-sync-secret/{print $1}' | head -1) -- env
```
  The output includes SECRET_TEST with a value matching the KMS secret.

kmsEndpoint configuration

Configure the KMS endpoint for a dedicated or shared gateway (see Differences between shared and dedicated gateways).

KMS endpoint addresses

Gateway type	Domain type	Endpoint address	Description
Dedicated gateway	KMS private	{kms-instance-id}.cryptoservice.kms.aliyuncs.com	The instance for the KMS credential and the cluster must be in the same region and VPC. Replace `{kms-instance-id}` with the instance ID of the KMS credential. The instance version for the KMS credential must be 3.0 or later.
Shared gateway	VPC	kms-vpc.{region}.aliyuncs.com	The KMS credential and the cluster must be in the same region. Replace `{region}` with the region where the KMS credential is located. This address uses default configurations. No extra configuration is needed.
Shared gateway	Public	kms.{region}.aliyuncs.com	Replace `{region}` with the region where the KMS credential is located. The cluster must have public network access.

KMS endpoint configuration example

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: test
spec:
  provider: alibabacloud # Fixed to 'alibabacloud'
  parameters:
    # Gateway usage examples:
    # hangzhou-public uses the public endpoint of a shared gateway. Replace {region} with the region where the KMS credential is located. This method allows you to access KMS credentials from a different region than the cluster.
    # hangzhou-vpc does not specify a kmsEndpoint field and uses the default VPC endpoint of a shared gateway.
    # hangzhou-cryptoservice uses a dedicated gateway. Replace {kms-instance-id} with the ID of the KMS instance.
    # london-public uses the public endpoint of a shared gateway. Replace {region} with the region where the KMS credential is located. This method allows you to access KMS credentials from a different region than the cluster.
    objects: |
      - objectName: "test-hangzhou"
        objectType: "kms"
        objectAlias: "hangzhou-public"
        kmsEndpoint:  "kms.{region}.aliyuncs.com"
      - objectName: "test-hangzhou"
        objectType: "kms"
        objectAlias: "hangzhou-vpc"
      - objectName: "test-hangzhou"
        objectType: "kms"
        objectAlias: "hangzhou-cryptoservice"
        kmsEndpoint:  "{kms-instance-id}.cryptoservice.kms.aliyuncs.com"
      - objectName: "test-london"
        objectAlias: "london-public"
        kmsEndpoint:  "kms.{region}.aliyuncs.com"

References

For ACK Serverless cluster workloads, see Use ack-secret-manager to import Alibaba Cloud KMS service credentials.
To protect cached KMS secrets, encrypt them at rest with Alibaba Cloud KMS.