Container Service for Kubernetes:Use ack-secret-manager to import OOS encryption parameters

Security notes

In most cases, applications directly read secrets from the file system. Compatibility issues may exist between applications and OOS encryption parameters. The ack-secret-manager component can resolve the compatibility issues. You can also use this component to synchronize encryption parameters from OOS to Kubernetes Secrets in your ACK cluster. Then, you can mount and use the Kubernetes Secrets as environment variables. Before you use this component, assess the following security risks:

• When secrets in your cluster are accessible through the file system, attackers can exploit Common Vulnerabilities and Exposures (CVE) vulnerabilities in applications to traverse the directories in the cluster. As a result, the secrets may be leaked.

• Due to potential misconfigurations in debugging breakpoints or log permissions that could lead to credential leaks, using secrets by referencing them through environment variables is insecure and not recommended.

• When you enable secret synchronization, strictly follow the least privilege principle to limit access permissions.

If you do not need to persist secrets for your application, we recommend that you refer to Use RRSA to authorize different pods to access different cloud services to grant the least access permissions to pods. Then, refer to GetSecretParameter to configure your application to call API operations to directly obtain the encryption parameters. This reduces the chances of exposing the encryption parameters in the pod file system or Kubernetes Secrets.

Prerequisites

• An ACK cluster is created in the same region as the OOS service that you want to use. You can create an ACK managed cluster, ACK dedicated cluster, registered cluster, or ACK Serverless cluster. For more information, see Create an ACK managed cluster, Create a registered cluster, and Create an ACK Serverless cluster

• An OOS encryption parameter is created. For more information, see Create an encrypted parameter and CreateSecretParameter.

• The kubeconfig file of the cluster is obtained and kubectl is used to connect to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Step 1: Install ack-secret-manager

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane of the cluster details page, choose Applications > Helm.

3. On the Helm page, click Deploy. In the Chart section of the Deploy panel, find and select ack-secret-manager, use the default values for other parameters, and then click Next.

   In the Confirm message, the displayed information indicates that the component will be installed in the kube-system namespace by default and the default application name will be used as the component name. Click Yes. If you want to use a custom application and a custom namespace, configure the Application Name and Namespace parameters in the Basic Information step.

4. In the Parameters step, select the latest chart version, configure other parameters, and then click OK.

   • If you want to enable RAM Roles for Service Accounts (RRSA) for authentication, set the rrsa.enable parameter to true.

   • If you want to enable scheduled synchronization of encryption parameters, configure the parameters shown in the following figure.

     • command.disablePolling: Specify whether to enable the automatic rotation feature for encryption parameters. Set the value to false to enable the automatic rotation feature for encryption parameters.

     • command.pollingInterval: Specify the frequency at which the encryption parameters are synchronized. In this example, this parameter is set to 120 seconds, which specifies that the encryption parameters are synchronized every 2 minutes. You can adjust the value as needed.

   • Configure throttling parameters: If your cluster has a large number of ExternalSecrets (OOS encryption parameters to be synchronized), improper configuration may cause throttling on OOS or RAM. Therefore, you must configure the following throttling parameters to avoid throttling. command.maxConcurrentOosSecretPulls: Specify the maximum number of OOS encryption parameters that can be synchronized per second. Default value: 10.

   After you install the component, you are redirected to the ack-secret-manager page. You can check whether the component is installed. If the resources shown in the following figure are created, the component is installed.

Step 2: Authorize ack-secret-manager to access OOS encryption parameters

You need to create a SecretStore to authorize ack-secret-manager to access OOS encryption parameters. Otherwise, ack-secret-manager cannot import or synchronize encryption parameters from OOS to your ACK cluster. You can use the following methods to authorize ack-secret-manager to access OOS encryption parameters:

• Use RRSA to grant permissions: This method applies to ACK managed clusters and ACK Serverless clusters that run Kubernetes 1.22 or later.

• Grant permissions to the worker RAM role of the cluster: This method applies to ACK managed clusters, ACK dedicated clusters, and registered clusters.

• Specify an AccessKey pair used to assume a RAM role: This method applies to all types of ACK clusters.

Step 3: Specify synchronization details

After the authentication information is configured, you must configure the OOS encryption parameters to be accessed by using ExternalSecret. This way, you can import the OOS encryption parameters to the Kubernetes Secret.

1. Create an ExternalSecret.

   1. Modify the following code block based on the following description. Then, create a file named external.yaml based on the modified code block.

      Parameter	Description
      {OOS parameter name}	Replace the value with the name of the desired OOS encryption parameter.
      {Kubernetes secret key}	A collection of key-value pairs. A single OOS encryption parameter is stored in a specific key-value pair within the Kubernetes Secret. You must replace the {Kubernetes secret key} with the key of the desired key-value pair.
      {secret store name}	Replace the value with the name of the SecretStore you created. The SecretStore is used to authorize ack-secret-manager to access the OOS encryption parameter. Note You can leave this parameter empty if the component is authorized by worker RAM role.
      {secret store namespace}	Replace the value with the namespace of the SecretStore you created. Note You can leave this parameter empty if the component is authorized by worker RAM role.

      apiVersion: 'alibabacloud.com/v1alpha1'
      kind: ExternalSecret
      metadata:
        name: esdemo
      spec:
        provider: oos # The type of the Alibaba Cloud service to be synchronized. The default value is kms. When you synchronize OOS encryption parameters, you must set the value to oos.
        data: # No additional configurations are required for the data source. 
          - key: {OOS parameter name}
            name: {Kubernetes secret key}
            secretStoreRef:   # You can leave this parameter empty if the component is authorized by worker RAM role. 
              name: {secret store name}
              namespace: {secret store namespace}

   2. Run the following command to create the ExternalSecret:

      kubectl apply -f external.yaml

2. Run the following command to check whether a Kubernetes Secret is created:

   kubectl get secret esdemo

   If a Kubernetes Secret is created, the encryption parameter is synchronized from OOS to your cluster.

Advanced ack-secret-manager configurations

cross-account encryption parameter synchronization

If your OOS instance and cluster do not belong to the same Alibaba Cloud account, you must synchronize OOS encryption parameters across accounts to the cluster. ack-secret-manager supports cross-account encryption parameter synchronization. The following example shows how to use the RRSA feature to import a OOS encryption parameter from Alibaba Cloud Account A to an ACK cluster that belongs to Alibaba Cloud Account B.

Configurations for Alibaba Cloud Account A

1. Use Alibaba Cloud Account A to create a RAM role whose trusted entity is Alibaba Cloud Account B. For more information, see Create a RAM role for a trusted Alibaba Cloud account.

   Important

   When you select the trusted Alibaba Cloud account, select Other Alibaba Cloud Account and then enter the ID of Alibaba Cloud Account B.

2. Create a policy that is required to access OOS encryption parameters.

   The following code block shows the content of the policy. For more information, see Create custom policies.

   {
       "Action": [
          "oos:GetSecretParameter",
          "kms:GetSecretValue"
       ],
       "Resource": [
           "*"
       ],
       "Effect": "Allow"
   }

3. Attach the policy to the RAM role. For more information, see Grant permissions to a RAM role.

Configurations for Alibaba Cloud Account B

1. Enable the RRSA feature for the ACK cluster in the ACK console. For more information, see Enable RRSA.

   Note

   When you install ack-secret-manager, set the rrsa.enable parameter to true to enable RRSA.

2. Create a RAM role whose trusted entity is an identity provider (IdP) for ack-secret-manager. The following table describes the key parameters. For details, see Create a RAM role for an OIDC IdP.

   Parameter	Description
   Identity Provider Type	Select OIDC.
   Identity Provider	Select an identity provider, which is named in the ack-rrsa-<cluster_id> format. <cluster_id> indicates the ID of your cluster.
   Condition	oidc:iss: Use the default value. oidc:aud: Use the default value. oidc:sub: Manually add one or more conditions. Key: Select oidc:sub. Operator: Select StringEquals. Value: Enter system:serviceaccount:<namespace>:<serviceAccountName>. <namespace>: Specify the namespace of the application. Note If you installed ack-secret-manager in another namespace, replace kube-system with the actual namespace name. <serviceAccountName>: Specify the name of the service account. In this example, enter system:serviceaccount:rrsa-demo:demo-sa.

3. Create a custom RAM policy and attach the policy to the RAM role you created by using Alibaba Cloud Account B in the previous step.

   1. Create a policy that is required when you use ack-secret-manager to import OOS encryption parameters.

      The following code block shows the content of the policy. Set Resource to the ARN of the RAM role you created by using Alibaba Cloud Account A. For more information, see Create a custom policy.

      {
        "Statement": [
          {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Resource": "acs:ram:*:<account-id>:role/<role-name>" # The ARN of the RAM role you created by using Alibaba Cloud Account A. 
          }
        ],
        "Version": "1"
      }

      In the custom policy above:

      • Resource field specifies the ARN of the RAM role.

      • <account-id>: The ID of Account A where the Key Management Service (KMS) instance is located.

      • <role-name>: The name of the RAM role created in Account A.

      To view a RAM role's ARN, see How do I view the ARN of a RAM role?

   2. Attach the policy to the RAM role you created by using Alibaba Cloud Account B in the previous step. For more information, see Grant permissions to a RAM role.

4. Create a SecretStore.

   Modify the following code block based on the following description. Then, create a file named secretstore-ramrole.yaml based on the modified code block.

   • {ACK-accountID}: Replace the value with the ID of Alibaba Cloud Account B.

   • {clusterID}: Replace the value with the ID of your cluster.

   • {ACK-roleName}: Replace the value with the name of the RAM role you created by using Alibaba Cloud Account B.

   • {OOS-accountID}: Replace the value with the ID of Alibaba Cloud Account A.

   • {OOS-roleName}: Replace the value with the name of the RAM role you created by using Alibaba Cloud Account A.

   • {roleSessionName}: Replace the value with the name of the role session. You can specify a custom string.

   apiVersion: 'alibabacloud.com/v1alpha1'
   kind: SecretStore
   metadata:
     name: scdemo-cross-account
   spec:
     OOS:
       OOSAuth:
         oidcProviderARN: "acs:ram::{ACK-accountID}:oidc-provider/ack-rrsa-{clusterID}"
         ramRoleARN: "acs:ram::{ACK-accountID}:role/{ACK-roleName}"
         remoteRamRoleARN: "acs:ram::{OOS-accountID}:role/{OOS-roleName}"
         remoteRamRoleSessionName: {roleSessionName}

5. Specify synchronization details. For more information, see Step 3: Specify synchronization details.

Encryption parameter parsing and key replacement

Parse the specified keys in a JSON secret

If you want to parse a OOS secret in JSON format and synchronize the specified key-value pairs from the OOS secret to a Kubernetes Secret, you can configure the jmesPath parameter. For example, the following JSON secret is stored in OOS encryption parameters:

{"name":"tom","friends":[{"name":"lily"},{"name":"mark"}]}

Create an ExternalSecret based on the following sample code. You must specify the following fields in the jmesPath parameter:

• path: parses the specified JSON field based on the JMESPath specification. This parameter is required.

• objectAlias: specifies the key of the key-value pair in the Kubernetes Secret that is synchronized based on the parsed field.

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: es-json-demo
spec:
  provider: oos
  data: 
    - key: {OOS parameter name}
      secretStoreRef:
        name: {secret store name}
        namespace: {secret store namespace}
      jmesPath: # Parse some fields in json string
        - path: "name"
          objectAlias: "myname"
        - path: "friends[0].name"
          objectAlias: "friendname"

Automatic JSON secret parsing

If you do not know the data structure of a JSON encryption parameter but still need to parse the encryption parameter and store the parsed data in a Kubernetes Secret, you can configure the dataProcess.extract parameter to enable automatic JSON parsing. In addition, you can configure the dataProcess.replaceRule parameter to specify rules for replacing keys in the parsed data. This prevents Kubernetes Secret creation failures caused by irregular Secret data keys.

For example, the following JSON secret is stored in OOS encryption parameters:

{"/name-invalid":"lily","name-invalid/":[{"name":"mark"}]}

Create an ExternalSecret based on the following sample code:

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
  name: extract-secret
spec:
  provider: oos
  dataProcess:
    - extract:
        key: {OOS parameter name}
        secretStoreRef:
          name: {secret store name}
          namespace: {secret store namespace}
      replaceRule: # The rules for replacing keys. 
        - source: "^/.*d$" # Replace the keys that start with a forward slash (/) and end with a lowercase letter d with tom. 
          target: "tom"
        - source: "^n.*/$" # Replace the keys that start with a lowercase letter n and end with a forward slash (/) with mark. 
          target: "mark"

References

To secure Kubernetes Secrets that are synchronized from OOS secrets and cached in ACK clusters, we recommend that you enable the Secret encryption feature. For more information, see Use KMS to encrypt Kubernetes Secrets.