Use csi-secrets-store-provider-alibabacloud to import OOS encryption parameters - Container Service for Kubernetes

csi-secrets-store-provider-alibabacloud allows you to import or synchronize encryption parameters from CloudOps Orchestration Service (OOS) to Kubernetes Secrets in an Container Service for Kubernetes (ACK) cluster to ensure that applications have secure access to sensitive data in your cluster. In addition, the plug-in allows you to directly mount encryption parameters to applications by using CSI inline volumes. This is suitable for applications that obtain sensitive data by calling file system APIs, such as the API to read files. This component allows you to automatically update encryption parameters to reduce the risk of exposing encryption parameters in Kubernetes Secrets. It also helps you resolve compatibility issues between your applications and OOS encryption parameters.

Security notes

By default, reading secrets from volumes involves a compatibility issue with the OOS encryption parameters. The csi-secrets-store-provider-alibabacloud plug-in can resolve this compatibility issue. In addition, you can create a secret as a Kubernetes-native Secret in a cluster when mounting environment variables. Before you use this component, assess the following security risks:

When secrets in your cluster are accessible through the file system, attackers can exploit Common Vulnerabilities and Exposures (CVE) vulnerabilities in applications to traverse the directories in the cluster. As a result, the secrets may be leaked.
Debugging breakpoints or improper log permissions may expose secrets in your cluster. We recommend that you do not use environment variables to mount or use secrets because this method poses security risks.
When you enable secret synchronization, strictly follow the least privilege principle to limit access permissions.

If you do not need to persist secrets for your application, we recommend that you refer to Use RRSA to authorize different pods to access different cloud services to grant the least access permissions to pods. Then, refer to GetSecretParameter to configure your application to call API operations to directly obtain the encryption parameters. This reduces the chances of exposing the encryption parameters in the pod file system or Kubernetes Secrets.

Prerequisites

An ACK cluster that meets the following requirements is created. For more information, see Create an ACK managed cluster and Create a registered cluster.
- The Kubernetes version of the cluster is 1.20 or later. You can create an ACK managed cluster, ACK dedicated cluster, or registered cluster. ACK Serverless clusters are not supported.
- The cluster and your OOS encryption parameters must be in the same region.
An OOS encryption parameter is created. For more information, see Create an encrypted parameter and CreateSecretParameter.
A kubectl client is connected to the cluster.

Step 1: Configure component authentication information

You must configure authentication information for csi-secrets-store-provider-alibabacloud to ensure that the plug-in has the permissions to obtain encryption parameters from OOS. Otherwise, csi-secrets-store-provider-alibabacloud cannot import or synchronize encryption parameters to the cluster. You can use the following methods to authorize ack-secret-manager to access OOS encryption parameters:

Grant permissions by using RRSA: This method is applicable to ACK managed clusters that run Kubernetes 1.22 or later.
Grant permissions to the worker RAM role of the cluster: This method applies to ACK managed clusters, ACK dedicated clusters, and registered clusters.
Specify an AccessKey pair used to assume a RAM role: This method applies to all types of ACK clusters.

Grant permissions by using RRSA

RAM Roles for Service Accounts (RRSA) is applicable to ACK managed clusters that run Kubernetes 1.22 or later. Compared to other authorization methods, RRSA can isolate permissions on pods and avoid the risk of secret leaks caused by the use of AccessKey IDs and AccessKey secrets.

Enable the RRSA feature in the ACK console for creating identity providers. For more information, see Enable RRSA.

Create a RAM role with Identity Provider as the Principal Type so that the RAM role can be used by csi-secrets-store-provider-alibabacloud. The following table describes the key parameters. For detailed steps, see Create a RAM role for an OIDC IdP.

Parameter	Description
Identity Provider Type	Select OIDC.
Identity Provider	Select an identity provider named in the ack-rrsa-<cluster_id>format, where <cluster_id> is the ID of your cluster.
Conditions	oidc:iss: Keep the default value. oidc:aud: Keep the default value. oidc:sub: Manually add the following condition: Key: Select oidc:sub. Operator: Select StringEquals. Value: Enter system:serviceaccount:<namespace>:<serviceAccountName>. Replace `<namespace>` with your application's namespace. Replace `<serviceAccountName>` with the service account name. Example: Based on the test application in this topic, enter `system:serviceaccount:kube-system:csi-secrets-store-provider-alibabacloud`. Note We recommend that you install components in the default `kube-system` namespace. If you want to install csi-secrets-store-provider-alibabacloud in another namespace, replace `kube-system` with the actual namespace name.

Create a custom RAM policy and attach the policy to the RAM role.
1. Create a policy that is required when you use csi-secrets-store-provider-alibabacloud to import OOS encryption parameters.
  The following code block shows the content of the policy. For more information, see Create custom policies.
```
{
    "Action": [
       "oos:GetSecretParameter",
       "kms:GetSecretValue"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
}
```
2. Attach the policy to the RAM role. For more information, see Grant permissions to a RAM role.
Create a Secret named alibaba-credentials in the cluster based on the following configuration template and replace the specified fields.
1. Create a file named secretstore-rrsa.yaml based on the following content and replace the specified fields:
  - {rolearn}: Replace the value with the ARN of the RAM role created in Step 2. The ARN must be a Base64-encoded string.
  - {oidcproviderarn}: Replace the value with the provider ARN generated after RRSA is enabled for the cluster. The ARN must be a Base64-encoded string.
```
apiVersion: v1
data:
  rolearn: {rolearn}
  oidcproviderarn: {oidcproviderarn}
kind: Secret
metadata:
  name: alibaba-credentials
  namespace: kube-system
type: Opaque
	                     
```
2. Run the following command to deploy the Secret:
```
kubectl apply -f secretstore-rrsa.yaml
```

Grant permissions to the worker RAM role of a cluster

This method is applicable to ACK managed clusters, ACK dedicated clusters, and registered clusters.

Create the following custom policy. For more information, see Create custom policies.

{
  "Action": [
    "oos:GetSecretParameter",
    "kms:GetSecretValue"
  ],
  "Resource": [
    "*"
  ],
  "Effect": "Allow"
}

Attach the custom policy created in the previous step to the worker RAM role. For more information, see Grant permissions to the worker RAM role.

Specify an AccessKey pair to assume a specific RAM role

This method is suitable for all types of ACK clusters.

Create a RAM role for a trusted Alibaba Cloud account so that the RAM role can be used by csi-secrets-store-provider-alibabacloud. For more information, see Create a RAM role for a trusted Alibaba Cloud account.
Create a custom policy to grant permissions to the RAM role that is created in the previous step.
1. Create a policy that is required to access OOS encryption parameters.
  The following code block shows the content of the policy. For more information, see Create custom policies.
```
{
  "Action": [
    "oos:GetSecretParameter",
    "kms:GetSecretValue"
  ],
  "Resource": [
    "*"
  ],
  "Effect": "Allow"
}
```
2. Attach the policy to the RAM role you created in the previous step. For more information, see Grant permissions to a RAM role.
Create a custom RAM policy that provides the permissions to assume the RAM role you created and then attach the policy to the RAM user that you want to use.
1. Create a custom RAM policy that contains the permissions to assume the RAM role you created.
  The following code block shows the content of the policy. For more information, see Create custom policies.
```
{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
              "Resource": "acs:ram::***:role/****"  # The Alibaba Cloud Resource Name (ARN) of the RAM role you created in Step 1 in this section. 
        }
    ],
    "Version": "1"
}
```
2. Attach the policy to the RAM user that you want to use. For more information, see Grant permissions to RAM users.
Create a Secret named alibaba-credentials in the cluster based on the following configuration template and replace the specified fields.
1. Create a file named alibaba-credentials.yaml based on the following content and replace the specified fields:
  - {rolearn}: Replace the value with the ARN of the RAM role created in Step 1. The ARN must be a Base64-encoded string.
  - {ak}: Replace the value with the AccessKey ID of the RAM user, which must be a Base64-encoded string.
  - {sk}: Replace the value with the AccessKey secret of the RAM user, which must be a Base64-encoded string.
```
apiVersion: v1
data:
  id: {ak}
  secret: {sk}
  rolearn: {rolearn}
kind: Secret
metadata:
  name: alibaba-credentials
  namespace: kube-system
type: Opaque
```
2. Run the following command to deploy the Secret:
```
kubectl apply -f alibaba-credentials.yaml
```

Step 2: Install the csi-secrets-store-provider-alibabacloud component

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
On the Helm page, click Deploy. In the Chart section, enter csi-secrets-store-provider-alibabacloud. Keep the default settings for other parameters, and then click Next.
In the dialog box that appears, confirm that the component is installed in the default kube-system namespace and the application is released with the component name. If you want to specify a custom application name and namespace, configure the application name and namespace as prompted.
Set the Chart Version parameter to the latest version. In the Parameters section, configure the parameters based on the authentication method that you selected in Step 1 and click OK.
- If you choose to grant permissions by using RRSA, you must set the rrsa.enable parameter to true to enable RRSA.
  The following code block shows other parameters:
```
envVarsFromSecret:
#  ACCESS_KEY_ID:
#    secretKeyRef: alibaba-credentials
#    key: id
#  SECRET_ACCESS_KEY:
#    secretKeyRef: alibaba-credentials
#    key: secret
  ALICLOUD_ROLE_ARN:
    secretKeyRef: alibaba-credentials
    key: rolearn
#  ALICLOUD_ROLE_SESSION_NAME:
#    secretKeyRef: alibaba-credentials
#    key: rolesessionname
#  ALICLOUD_ROLE_SESSION_EXPIRATION:
#    secretKeyRef: alibaba-credentials
#   key: rolesessionexpiration
  ALICLOUD_OIDC_PROVIDER_ARN:
    secretKeyRef: alibaba-credentials
    key: oidcproviderarn
```
- If you choose to grant permissions to the worker RAM role of a cluster, you can use the default parameter settings to directly install csi-secrets-store-provider-alibabacloud.
- If you choose to specify an AccessKey pair to assume a specific RAM role, you must configure the following parameters:
```
envVarsFromSecret:
  ACCESS_KEY_ID:
    secretKeyRef: alibaba-credentials
    key: id
  SECRET_ACCESS_KEY:
    secretKeyRef: alibaba-credentials
    key: secret
  ALICLOUD_ROLE_ARN:
    secretKeyRef: alibaba-credentials
    key: rolearn
#  ALICLOUD_ROLE_SESSION_NAME:
#    secretKeyRef: alibaba-credentials
#    key: rolesessionname
#  ALICLOUD_ROLE_SESSION_EXPIRATION:
#    secretKeyRef: alibaba-credentials
#    key: rolesessionexpiration
#  ALICLOUD_OIDC_PROVIDER_ARN:
#    secretKeyRef: alibaba-credentials
#    key: oidcproviderarn
```
- To enable the scheduled encryption parameters synchronization feature, you must configure the following parameters:
  - secrets-store-csi-driver.enableSecretRotation: Specify whether to enable the automatic rotation feature for encryption parameters. Set the value to true to enable the automatic rotation feature for encryption parameters.
  - secrets-store-csi-driver.rotationPollInterval: Specify the frequency at which the encryption parameters are synchronized. In this example, this parameter is set to 120 seconds, which specifies that the encryption parameters are synchronized every 2 minutes. You can adjust the value as needed.
After the creation, you are redirected to the csi-secrets-store-provider-alibabacloud page of the cluster to check the installation result. If all resources in the following figure are created, the component is installed.

Step 3: Configure data synchronization information

After the authentication information is configured, you must configure OOS encryption parameters by using SecretProviderClass.

Configuration template description

The following code block shows the SecretProviderClass template:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: <NAME>
spec:
  provider: alibabacloud   # Set the value to alibabacloud. 
  parameters:

The parameters section usually contains the following request parameters for mounting Kubernetes Secrets.

Parameter	Type	Description
objects	Required	The YAML configuration for mounting encryption parameters. Example: `parameters: objects: \| - objectName: "MySecret" - objectType: "oos"` The `objects` parameter contains the following subfields: `objectName`: Required. The name of the encryption parameter in the OOS parameter store. For more information, see SecretName. `objectType`: Optional. This parameter can be used to specify the type of Alibaba Cloud service to be synchronized. kms and oos are supported. The default value is kms. When importing OOS encryption parameters, you must set this parameter to oos. `objectAlias`: Optional. You can use this field to specify the name of the file that is mounted to the pod. If you do not specify this field, the value of objectName is used by default. `jmesPath`: Optional. This field is used to parse the specified key-value pairs in JSON-formatted encryption parameters. For example, the following sample test encryption parameter contains a JSON segment: `{ "username": "testuser", "password": "testpassword" }` To mount `username` and `password` as secret files separately, configure the JMESPath field. When you use the JMESPath field, you must specify the following subfields: `path`: Required. This field defines the specified configuration items for path retrieval based on JMESPath. `objectAlias`: Optional. This field specifies the name of the file that mounts the secret to the pod.
region	Optional	Send requests to the OOS encryption parameter server in the specified region. If you do not specify this parameter, the region of the current node is selected. This may increase performance overhead if your application runs in large numbers of pods. Therefore, we recommend that you configure this parameter to specify a region.
pathTranslation	Optional	A substitution path separator to use if the file name of the encryption parameter to mount in Parameter Store contains path separators. For example, `My/Path/Secret` is mounted as My_Path_Secret. If you do not specify this parameter, the path separator for replacement is underscore (_). To disable path separator replacement, set the parameter to `"False"`.

Configuration example

This example shows how to use the SecretProviderClass to import the encryption parameter test from OOS to applications in an ACK managed cluster in the same region.

Create a file named secretstore.yaml and add the following content to the file to define the SecretProviderClass:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: test-secrets
spec:
  provider: alibabacloud   # Set the value to alibabacloud. 
  parameters:
    objects: | # The objectType parameter can be set to oos or kms. Default value: kms.
      - objectName: "test"
      - objectType: "oos"

Run the following command to deploy the SecretProviderClass:
```
kubectl apply -f secretstore.yaml
```

Create a file named deploy.yaml and add the following content to the file:

An NGINX Deployment is included, which declares the SecretProviderClass created in the preceding example by using a CSI Inline volume. The encryption parameters are mounted in the /mnt/secrets-store directory of the pod. For more information about Deployments, see Deployment examples.

apiVersion: apps/v1  # If the API version is earlier than 1.8.0, use apps/v1beta1. 
kind: Deployment
metadata:
  name: nginx-deployment-basic
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "test-secrets"
      containers:
        - name: nginx
          image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 # Replace this value with the actual image. 
          ports:
            - containerPort: 80
          resources:
            limits:
              cpu: "500m"
          volumeMounts:
            - name: secrets-store-inline
              mountPath: "/mnt/secrets-store"
              readOnly: true

Run the following command to deploy the Deployment:
```
kubectl apply -f deploy.yaml
```
Verify that the secret is mounted.
Log on to a pod, check whether the encryption parameter specified in the SecretProviderClass is created in the mount target /mnt/secrets-store, and then check whether the encryption parameter contains the corresponding ciphertext stored in OOS.

References

For more information about how to import encryption parameters from OOS to applications in ACK Serverless clusters, see Use ack-secret-manager to import OOS encryption parameters.
To secure Kubernetes Secrets that are synchronized from OOS encryption parameters and cached in ACK clusters, we recommend that you enable the Secret encryption feature. For more information, see Use KMS to encrypt Kubernetes Secrets.