The csi-secrets-store-provider-alibabacloud component imports OOS encryption parameters into your cluster as Kubernetes Secrets or mounted files, with automated secret rotation. This resolves compatibility issues between workload applications and Alibaba Cloud OOS encryption parameters.
Security considerations
The csi-secrets-store-provider-alibabacloud component resolves compatibility issues between file-system-based secret readers and OOS encryption parameters. It also syncs secrets as Kubernetes Secret objects for use as environment variables. Before using this component, evaluate the following security risks.
File system traversal: Secrets mounted through the file system can be exposed if flawed applications are exploited to perform directory traversal attacks.
Debug and log exposure: Misconfigured debug breakpoints or overly permissive log access can leak Secrets. Referencing Secrets through environment variables is insecure and not recommended.
Excessive permissions: Apply the least privilege principle when granting
ack-secret-manageraccess to KMS.
If your application does not require persisted secrets, configure Pod-level least-privilege permissions through RAM Roles for Service Accounts (RRSA) to achieve Pod permission isolation. Your application can then fetch encryption parameters directly through the GetSecretParameter API operation, avoiding exposure in the Pod file system or as Kubernetes Secret objects.
Prerequisites
An ACK cluster that meets the following requirements. To create one, see Create an ACK managed cluster or Create an ACK One registered cluster.
The cluster runs Kubernetes version 1.20 or later. ACK managed clusters, ACK dedicated clusters, and ACK One registered clusters are supported. ACK Serverless clusters are not supported.
The cluster and your CloudOps Orchestration Service (OOS) must be in the same region.
An OOS encryption parameter has been created. Create an OOS encryption parameter. Create an encryption parameter.
Step 1: Configure authentication
Configure authentication for csi-secrets-store-provider-alibabacloud to grant permission to retrieve OOS encryption parameters. Without authentication, csi-secrets-store-provider-alibabacloud cannot import or sync them. Choose one of the following authorization methods based on your cluster type.
Authorize using RRSA: This method is applicable to ACK managed clusters that run Kubernetes 1.22 or later.
Grant permissions to the worker RAM role of the cluster: This method is applicable to ACK managed clusters, ACK dedicated clusters, and ACK One registered clusters.
Assume a specified RAM role using an AccessKey pair: This method is applicable to all ACK clusters.
RRSA
RRSA applies to ACK managed clusters running Kubernetes 1.22 or later. It provides Pod-level permission isolation and prevents credential leaks from direct AccessKey pair usage.
Enable RRSA for your cluster in the ACK console to create the identity provider. Enable RRSA.
Create a RAM role that trusts an identity provider for the csi-secrets-store-provider-alibabacloud component. Set the parameters as described in the following table. Create a RAM role for an OIDC identity provider.
Parameter
Description
IdP Type
OIDC.
Identity provider
Select ack-rrsa-<cluster_id>, where <cluster_id> is the ID of your cluster.
Conditions
oidc:iss: Keep the default value.
oidc:aud: Keep the default value.
oidc:sub: Manually add this condition.
Key: Select oidc:sub.
Operator: Select StringEquals.
Condition value: Enter
system:serviceaccount:<namespace>:<serviceAccountName>. In this format,<namespace>specifies the namespace where the application is located, and<serviceAccountName>specifies the name of the service account. For the test application in this topic, entersystem:serviceaccount:kube-system:csi-secrets-store-provider-alibabacloud.NoteWe recommend that you install the component in the default
kube-systemnamespace. If you install csi-secrets-store-provider-alibabacloud in a different namespace, replacekube-systemwith the name of that namespace.
Create a custom policy and attach it to the RAM role that you created in the previous step.
Create the following policy for csi-secrets-store-provider-alibabacloud to import OOS encryption parameters. Create a custom policy.
{ "Action": [ "oos:GetSecretParameter", "kms:GetSecretValue" ], "Resource": [ "*" ], "Effect": "Allow" }Attach the policy to the RAM role created in the previous step. Manage permissions for a RAM role.
Create a Kubernetes Secret named alibaba-credentials in your cluster by using the following template and replacing the placeholders.
Create a file named secretstore-rrsa.yaml with the following content, replacing the placeholders with your information.
{rolearn}: Replace with the Base64-encoded ARN of the RAM role that you created in Step 2.{oidcproviderarn}: Replace with the provider ARN that is generated after you enable RRSA for the cluster. The ARN must be Base64-encoded.
apiVersion: v1 data: rolearn: {rolearn} oidcproviderarn: {oidcproviderarn} kind: Secret metadata: name: alibaba-credentials namespace: kube-system type: OpaqueRun the following command to deploy the Secret:
kubectl apply -f secretstore-rrsa.yaml
Worker RAM role
This method is applicable to ACK managed clusters, ACK dedicated clusters, and ACK One registered clusters.
Create the following custom policy. Create a custom policy.
{ "Action": [ "oos:GetSecretParameter", "kms:GetSecretValue" ], "Resource": [ "*" ], "Effect": "Allow" }Attach the policy to the worker RAM role of your cluster. Grant permissions to the worker RAM role of a cluster.
AccessKey pair
This method is applicable to all ACK clusters.
Create a RAM role that trusts an Alibaba Cloud account for the csi-secrets-store-provider-alibabacloud component. Create a RAM role for an Alibaba Cloud account.
Create a custom policy and attach it to the RAM role that you created in the previous step.
Create the following policy for accessing OOS encryption parameters. Create a custom policy.
{ "Action": [ "oos:GetSecretParameter", "kms:GetSecretValue" ], "Resource": [ "*" ], "Effect": "Allow" }Attach the policy to the RAM role created in the previous step. Manage permissions for a RAM role.
Create a custom policy that allows a RAM user to assume the role, and then attach the policy to the intended RAM user.
Create a custom policy to assume the preceding RAM role. Set
Resourceto the ARN of the RAM role created in Step 1. Create a custom policy.{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Resource": "acs:ram::***:role/****" } ], "Version": "1" }Attach the policy to the intended RAM user. Manage permissions for a RAM user.
Create a Kubernetes Secret named alibaba-credentials in your cluster. Use the following template and replace the placeholders.
Create a file named alibaba-credentials.yaml with the following content. Replace the placeholders with your information.
{rolearn}: The Base64-encoded ARN of the RAM role that you created in Step 1.{ak}: Replace with the AK of the RAM user (must be Base64-encoded).{sk}: The SK of the RAM user. This value must be Base64-encoded.apiVersion: v1 data: id: {ak} secret: {sk} rolearn: {rolearn} kind: Secret metadata: name: alibaba-credentials namespace: kube-system type: Opaque
Run the following command to deploy the Secret:
kubectl apply -f alibaba-credentials.yaml
Step 2: Install the csi-secrets-store-provider-alibabacloud component
-
Log on to the ACK console. In the left navigation pane, click Clusters.
-
On the Clusters page, click the name of your cluster. In the left navigation pane, click .
On the Helm page, click Deploy. In the Chart section, search for and select csi-secrets-store-provider-alibabacloud. Keep the default settings and click Next.
Confirm the settings in the dialog box. The component installs in the kube-system namespace by default. To use a custom application name or namespace, set them as prompted.
Select the latest Chart Version. In the Parameters section, configure the relevant parameters based on the authentication method that you chose in Step 1. Then, click OK.
If you select Authorize using RRSA, you need to set the
rrsa.enableparameter to true to enable the RRSA authentication feature.rrsa: # Specifies whether using rrsa and enalbe sa token volume projection, default is false enable: trueOther related parameters are configured as follows.
envVarsFromSecret: # ACCESS_KEY_ID: # secretKeyRef: alibaba-credentials # key: id # SECRET_ACCESS_KEY: # secretKeyRef: alibaba-credentials # key: secret ALICLOUD_ROLE_ARN: secretKeyRef: alibaba-credentials key: rolearn # ALICLOUD_ROLE_SESSION_NAME: # secretKeyRef: alibaba-credentials # key: rolesessionname # ALICLOUD_ROLE_SESSION_EXPIRATION: # secretKeyRef: alibaba-credentials # key: rolesessionexpiration ALICLOUD_OIDC_PROVIDER_ARN: secretKeyRef: alibaba-credentials key: oidcproviderarnIf you chose to grant permissions to the worker RAM role of your cluster, you can proceed with the installation using the default parameter settings.
If you chose to assume a specified RAM role using an AccessKey pair, you need to configure the following parameters.
envVarsFromSecret: ACCESS_KEY_ID: secretKeyRef: alibaba-credentials key: id SECRET_ACCESS_KEY: secretKeyRef: alibaba-credentials key: secret ALICLOUD_ROLE_ARN: secretKeyRef: alibaba-credentials key: rolearn # ALICLOUD_ROLE_SESSION_NAME: # secretKeyRef: alibaba-credentials # key: rolesessionname # ALICLOUD_ROLE_SESSION_EXPIRATION: # secretKeyRef: alibaba-credentials # key: rolesessionexpiration # ALICLOUD_OIDC_PROVIDER_ARN: # secretKeyRef: alibaba-credentials # key: oidcproviderarnTo sync secrets to Kubernetes Secret objects, configure the following parameters.
syncSecret: enabled: truesecrets-store-csi-driver.syncSecret.enabled: Syncs secrets as Kubernetes Secrets. Whentrue, deploys the required RBAC Role and RoleBinding.To enable periodic synchronization of encryption parameters, configure the following parameters.
enableSecretRotation: true rotationPollInterval: 120ssecrets-store-csi-driver.enableSecretRotation: Enables automatic rotation for encryption parameters. Set to true.secrets-store-csi-driver.rotationPollInterval: The sync interval for encryption parameters. Default 120s (every two minutes). Adjust as needed.
After deployment, verify the installation on the csi-secrets-store-provider-alibabacloud page. The following nine Kubernetes resources should appear on the Resources page:
secrets-store-csi-driver(ServiceAccount),csi-secrets-store-provider-alibabacloud(ServiceAccount),secretproviderclasses-admin-role(ClusterRole),secretproviderclasses-viewer-role(ClusterRole),secretproviderclasses-role(ClusterRole),secretproviderclasses-rolebinding(ClusterRoleBinding),secrets-store-csi-driver(DaemonSet),csi-secrets-store-provider-alibabacloud(DaemonSet), andsecrets-store.csi.k8s.io(CSIDriver).
Step 3: Configure data synchronization
Use a SecretProviderClass to specify which OOS encryption parameters to sync.
Template configuration
The SecretProviderClass template is defined as follows.
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: <NAME>
spec:
provider: alibabacloud # This value is fixed. Do not modify it.
parameters:
objects:
|
- objectName: <OOS Encryption Parameter Name> # The name of the OOS encryption parameter.
objectType: oos # This value must be 'oos' when syncing OOS encryption parameters.The parameter parameters typically contains three fields of a mount request:
Parameter | Type | Description |
objects | Required | The YAML configuration for mounting encryption parameters. Example:
|
region | Optional | The region of the OOS encryption parameter backend. Defaults to the node's region, but explicitly setting this avoids performance overhead in large-scale Pod deployments. |
pathTranslation | Optional |
|
Usage example
This example imports an OOS encryption parameter named test from the same region as an ACK managed cluster into a cluster application.
Create a file named secretstore.yaml with the following content to define a simple SecretProviderClass.
apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: test-secrets spec: provider: alibabacloud # This value is fixed as alibabacloud. parameters: objects: | # objectType supports oos and kms. The default is kms. - objectName: "test" objectType: "oos"Run the following command to deploy the SecretProviderClass:
kubectl apply -f secretstore.yamlCreate a file named deploy.yaml with the following content.
This Nginx Deployment mounts the SecretProviderClass as a CSI inline volume at
/mnt/secrets-store. Deployment examples.apiVersion: apps/v1 # For versions earlier than 1.8.0, use apps/v1beta1. kind: Deployment metadata: name: nginx-deployment-basic labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: volumes: - name: secrets-store-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "test-secrets" containers: - name: nginx image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 # Replace with your actual image. ports: - containerPort: 80 resources: limits: cpu: "500m" volumeMounts: - name: secrets-store-inline mountPath: "/mnt/secrets-store" readOnly: trueRun the following command to deploy the application:
kubectl apply -f deploy.yamlVerify that the secret was mounted correctly.
Log on to the Pod and check
/mnt/secrets-storefor a secret file matching the name in the SecretProviderClass. Its content should match the OOS encryption parameter value.
Sync OOS encryption parameters as Kubernetes Secrets
The Secrets Store CSI Driver automatically syncs secrets from external services (KMS or OOS) as native Kubernetes Secret objects, letting applications use external secrets without code changes.
Configuration method: SecretProviderClass
You can add the optional secretObjects field to the spec of a SecretProviderClass resource to enable this feature.
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: <NAME>
spec:
provider: alibabacloud # This value is fixed. Do not modify it.
parameters:
objects:
|
- objectName: <OOS Encryption Parameter Name> # The name of the OOS encryption parameter.
objectType: oos # This value must be 'oos' when syncing OOS encryption parameters.
secretObjects:
- secretName: <Kubernetes Secret Name> # The name of the Kubernetes Secret.
type: <Kubernetes Secret Type> # The type of the Kubernetes Secret.
data:
- objectName: <parameters.objects.objectName> # The name from parameters.objects.objectName. If an alias is specified, use the alias.
key: <Kubernetes Secret Data Key> # The key in the Kubernetes Secret's data field.secretObjects typically includes the following three parameters:
Parameter | Type | Description |
secretName | Required | The name of the Kubernetes Secret to create. |
type | Required | The Secret type. Valid values: |
data | Required | Defines how to map an external secret to a
|
Synchronization lifecycle
The synchronization and cleanup of a Secret object are dynamically triggered by a Pod that mounts the corresponding SecretProviderClass:
Creation: The Kubernetes Secret is created only when the first Pod using the
SecretProviderClassstarts and mounts the volume.Update: When
secrets-store-csi-driver.enableSecretRotationistrue, the Kubernetes Secret is periodically refreshed at thesecrets-store-csi-driver.rotationPollIntervalinterval. Otherwise, it is not updated.Deletion: When the last Pod using the
SecretProviderClassis deleted, the corresponding Kubernetes Secret is also deleted.
Example: Sync a parameter as an environment variable
This example retrieves an OOS encryption parameter, syncs it as a Kubernetes Secret, and injects it as an environment variable into an NGINX Pod.
Create the SecretProviderClass.
Create a file named sync-secret.yaml with the following content.
apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: alibabacloud-sync-secret spec: provider: alibabacloud parameters: objects: | - objectName: test-oos objectAlias: secretalias objectType: oos secretObjects: - secretName: test-sync-secret # Name of the generated Kubernetes Secret type: Opaque data: - objectName: secretalias # Corresponds to objectName or objectAlias key: test # Populates the data of the generated Kubernetes SecretDeploy the SecretProviderClass.
kubectl apply -f sync-secret.yamlCreate the application Pod to trigger the synchronization.
Create a file named pod-sync-secret.yaml with the following content. This Pod mounts the preceding
SecretProviderClass, and attempts to usesecretKeyRefto reference theSecretnamed test-sync-secret that will be generated.kind: Pod apiVersion: v1 metadata: name: pod-sync-secret spec: containers: - name: nginx image: nginx:latest volumeMounts: - name: secrets-store-inline mountPath: "/mnt/secrets-store" readOnly: true env: - name: SECRET_TEST valueFrom: secretKeyRef: name: test-sync-secret key: test volumes: - name: secrets-store-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "alibabacloud-sync-secret"Deploy the Pod, which triggers the Secret synchronization.
kubectl apply -f pod-sync-secret.yamlVerify the result:
Check if the Kubernetes Secret has been created.
kubectl get secret test-sync-secretThe output shows a secret named test-sync-secret.
Check if the environment variable was successfully injected into the Pod:
kubectl exec -it $(kubectl get pods | awk '/pod-sync-secret/{print $1}' | head -1) -- envThe output includes the
SECRET_TESTenvironment variable with its value from the OOS encryption parameter.
References
For ACK Serverless clusters: Use ack-secret-manager to import Alibaba Cloud OOS encryption parameters.
To protect OOS secrets cached in an ACK cluster, encrypt them at rest. Encrypt secrets at rest by using Alibaba Cloud KMS.