Key Management Service:ACK rapidly integrates KMS Agent through ack-kms-agent-webhook-injector component

Overview

Architecture

Alibaba Cloud Container Service for Kubernetes (ACK) provides the ack-kms-agent-webhook-injector component, which injects KMS Agent as a sidecar container into pods through specific annotations (kms-agent-webhook-injector/inject). This allows applications in containers to access credentials from KMS instances through local HTTP interfaces and cache them in memory, avoiding hardcoded sensitive information and ensuring data security. Additionally, the caching mechanism of KMS Agent improves stability when accessing KMS in high concurrency and unstable network scenarios, enhancing the user experience. The following figure shows the architecture:

Authentication methods

Two authentication methods are supported: RRSA and Worker RAM role.

• (Recommended) Method 1: RRSA method

  This method can achieve pod-level permission isolation. It is applicable to ACK managed clusters and ACK serverless clusters with version 1.22 and above.

  Important

  To achieve pod-level permission isolation, the version of ack-kms-agent-webhook-injector must be greater than or equal to v0.2.0.

  For example, through the RRSA method, you can allow applications deployed in app1-dev to use the app1-rrsa role to access credentials in KMS with the secret:app1 tag. Applications deployed in app2-dev use the app2-rrsa role to access credentials in KMS with the secret:app2 tag.

• Method 2: Worker RAM role method

  This method can achieve worker-level permission isolation. Since ACK serverless clusters do not support binding Worker RAM roles, this method is only applicable to ACK managed clusters and ACK dedicated clusters.

  For example, you can use Worker RAM roles to set up applications in app1-dev and app2-dev to access credentials managed in KMS. Applications in the same worker use the same Worker RAM role, meaning you cannot set different permissions for app1-dev and app2-dev.

Prerequisites

• A KMS instance has been created, and credentials have been hosted in the KMS instance. For more information, see Purchase and enable a KMS instance, Credential management quick start.

• An ACK cluster has been created, and the cluster is in the same region as your KMS instance. See Create an ACK managed cluster, Create an ACK serverless cluster.

Step 1: Configure authentication information to allow KMS Agent to access specific credentials

Step 2: Create namespaces and service accounts in the ACK cluster

Namespaces divide the ACK cluster into logically isolated virtual spaces to separate development, testing, production, and other environments. Applications in different namespaces cannot access each other's resources by default, providing a physical boundary for RAM role binding.

Create two namespaces, app1-dev and app2-dev. The corresponding serviceAccountNames are app1-service and app2-service.

1. Create a namespace named app1-dev using a YAML file.

   1. Create a YAML file named app1-namespace.yaml.

      apiVersion: v1
      kind: Namespace
      metadata:
        name: app1-dev

   2. Run the following command to create a namespace named app1-dev in the ACK cluster.

      kubectl apply -f app1-namespace.yaml

   3. Check if the namespace has been created successfully.

      kubectl get namespaces

      If the output includes app1-dev, the creation is successful.

2. Create a service account named app1-service using a YAML file.

   1. Create a YAML file named app1-serviceaccount.yaml.

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: app1-service
        namespace: app1-dev

   2. Run the following command to create a service account named app1-service in app1-dev.

      kubectl apply -f app1-serviceaccount.yaml

   3. Check if the service account has been created successfully.

      kubectl get serviceaccount -n app1-dev

      If the output includes app1-service, the creation is successful.

3. Repeat the above steps to create the app2-dev namespace and app2-service service account.

Step 3: Install the ack-kms-agent-webhook-injector component in the ACK cluster

1. Log on to the Container Service console and click Clusters in the navigation pane on the left.

2. On the Clusters page, click the name of the target cluster. In the navigation pane on the left, choose Applications > Helm.

3. On the Helm page, click Create, configure Basic Information, and click Next.

   Parameter	Description
   Application Name	Enter your application name. We recommend that you use the default application name. You do not need to enter it here. After you click Next, you will be prompted to use the default application name ack-kms-agent-webhook-injector. Select Yes.
   Namespace	Select the namespace where the application is located. We recommend that you use the default namespace. You do not need to select it here. After you click Next, you will be prompted to use the default namespace kube-system. Select Yes. Installing the component in one namespace in an ACK cluster is sufficient. There is no need to install it repeatedly.
   Source	The default is App Catalog, which cannot be modified.
   Chart	Search for and select ack-kms-agent-webhook-injector.

4. In the dialog box that appears, confirm the information and click Yes.

   Selecting Yes means using the default installation path. The component is installed in the kube-system namespace by default and published as an application with the component name.

5. On the Parameters page, complete the configuration and click OK.

   Parameter	Description
   Chart Version	We recommend that you select the latest version.
   Parameters	Configuration instructions for agent.auth.roleArn and agent.auth.roleArnMapping parameters: When using RRSA authorization: agent.auth.roleArn is empty, and the format of agent.auth.roleArnMapping is <NameSpace>:<serviceAccountName>:<RAM Role ARN>. agent: auth: roleArn: roleArnMapping: app1-dev:app1-service: acs:ram::190325303126****:role/app1-rrsa app2-dev:app2-service: acs:ram::190325303126****:role/app2-rrsa When using Worker RAM role authorization: both agent.auth.roleArn and agent.auth.roleArnMapping are set to empty.

6. Wait for about 30 seconds, enter the namespace (default is kube-system), and check the component status in the Deployments workload to confirm whether the component is ready.

Step 4: Inject KMS Agent into workloads

Inject KMS Agent into pods by configuring annotations. The specific operation varies depending on whether you are configuring for a new workload's pod or an existing workload's pod.

Take the stateless workload Deployment as an example. In this example, perform the following operations for app1-dev and app2-dev respectively.

1. Log on to the Container Service console.

2. Go to the target cluster and choose Workloads > Deployments in the navigation pane on the left.

3. When creating or modifying a workload, add the following pod annotation: Name is kms-agent-webhook-injector/inject, Value is true (values 1, T, t, True, and TRUE are also considered valid).

   Scenario 1: Create a new workload using an image

   Click Create From Image, complete the configuration, and click Create.

   When filling in Advanced, in the Labels And Annotations section, add the following pod annotation: Name is kms-agent-webhook-injector/inject, Value is true.

   Scenario 2: Create a new workload using YAML

   Click Create From YAML, complete the configuration, and click Create.

   Edit the spec.template.metadata.annotations section in the YAML file (if it does not exist, you need to create it), and add the key-value pair kms-agent-webhook-injector/inject: "true". For information about how to configure other parameters, see Workload YAML examples.

   Scenario 3: Existing workloads

   1. Locate the target workload and click Details in the Actions column.

   2. On the workload details page, click the YAML button in the upper right corner, find the spec.template.metadata.annotations section (if it does not exist, you need to create it), and add the key-value pair kms-agent-webhook-injector/inject: "true".

   3. Click Update and wait for the workload to be ready again.

4. Return to the Deployments page, enter the workload, and you can observe in the Pods tab that KMS Agent has been injected into your workload as a sidecar in the Image column.

   Important

   When viewing images in the Pods tab, you may notice that the KMS Agent image has been injected twice into the pod. This is because we use an initialization container (initContainer) to perform necessary initialization work. This initialization container terminates (Terminated) after initialization is complete and does not negatively impact your application or continuously occupy your computing resources.

   

5. If you use RRSA authorization, modify the YAML configuration file to change the serviceAccountName parameter to the service account authorized in Step 1.

   In this example, they are app1-service and app2-service. After modification, the application and KMS Agent can be properly authorized to access KMS credentials.

   Note

   When using Worker RAM role authorization, there is no need to modify the configuration file. The service account can automatically assume the Worker RAM role to obtain permissions.

Step 5: Obtain KMS credentials in the application container

After KMS Agent is injected into the workload, you can send requests to KMS Agent through the HTTP protocol in the application container to obtain credentials stored in KMS.

Example:

• The local host HTTP port number takes the default value 2025. If you set it to another port number, please replace 2025 in the example with the actual port number you are using.

• The token path takes the default path file:///var/run/kmstoken/token. If you set another path, such as file:///var/run/path1/path2, please replace /var/run/kmstoken/token in the example with /var/run/path1/path2.

 # Read token from file
 curl -v -H "X-KMS-Token:$(</var/run/kmstoken/token)" 'http://localhost:2025/secretsmanager/get?secretId=<SecretId>'
 
 # Write token directly
 curl -v -H "X-KMS-Token:<token>" 'http://localhost:2025/secretsmanager/get?secretId=<SecretId>'

 # Read token from file
 curl -v -H "X-KMS-Token:$(</var/run/kmstoken/token)" 'http://localhost:2025/secretsmanager/get?secretId=<SecretId>&versionId=0a7513ee719da740807b15b77500****'
 
 # Write token directly
 curl -v -H "X-KMS-Token:<token>" 'http://localhost:2025/secretsmanager/get?secretId=<SecretId>&versionId=0a7513ee719da740807b15b77500****'

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
)

func main() {
	
	//Support specifying versionStage or versionId to obtain specific credential values.
	//For example, to obtain credential values with a specified versionId, url := fmt.Sprintf("http://localhost:2025/secretsmanager/get?secretId=%s&versionId=%s", "agent-test", "version-id").
	url := fmt.Sprintf("http://localhost:2025/secretsmanager/get?secretId=%s", "agent-test")

	token, err := ioutil.ReadFile("/var/run/kmstoken/token")
	if err != nil {
		fmt.Printf("error reading token file: %v\n", err)
	}

	req, err := http.NewRequest("GET", url, nil)
	if err != nil {
		fmt.Printf("error creating request: %v\n", err)
	}

	req.Header.Add("X-KMS-Token", string(token))

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		fmt.Printf("error sending request: %v \n", err)
	}
	defer resp.Body.Close()

	body, _ := ioutil.ReadAll(resp.Body)
	fmt.Printf("status code %d - %s \n",resp.StatusCode, string(body))
}