Key Management Service:Deploy KMS Agent in ACK container environment to retrieve secrets

Step 1: Build the KMS Agent executable file

1. Install the Golang environment. For detailed operations, see Go installation guide.

2. Download the source code and dependencies.

   1. Visit the Git website to download and install the Git tool.

   2. Execute the following command to download the source code and dependencies.

      git clone https://github.com/aliyun/alibabacloud-kms-agent
      go mod download

3. In the project root directory, execute the go build . command to compile the executable file. The default file name is alibabacloud-kms-agent, saved in the project root directory.

   If the compilation environment and deployment environment are the same, simply execute the go build . command. If the compilation environment and deployment environment are different, refer to the following commands for cross-platform compilation to generate a 64-bit executable file.

   Compilation environment	Deployment environment is Mac	Deployment environment is Linux	Deployment environment is Windows
   Mac	go build .	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build .	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build .
   Linux	CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build .	go build .	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build .
   Windows	SET CGO_ENABLED=0 SET GOOS=darwin SET GOARCH=amd64 go build .	SET CGO_ENABLED=0 SET GOOS=linux SET GOARCH=amd64 go build .	go build .

4. In the project root directory, check if the executable file alibabacloud-kms-agent exists.

Step 2: Enable ACK RRSA and authorize it to access specific secrets

1. Enable the ACK RRSA feature.

   Enable during cluster creation

   When you create an ACK managed cluster or ACK Edge cluster, click Enable next to RRSA OIDC in the Advanced Options (Optional) section of the Cluster Configurations step.

   

   Enable after cluster creation

   1. Log on to the ACK console. In the left navigation pane, click Clusters.

   2. On the Clusters page, find the target cluster and click its name. In the navigation pane on the left, click Cluster Information.

   3. In the Security and Auditing section of the Basic Information tab, click Enable next to RRSA OIDC.

   4. In the Enable RRSA dialog box, click Confirm.

      In the Basic Information section, if the cluster status changes from Updating to Running, the RRSA feature is enabled for the cluster.

2. Open the cluster details page. On the Basic Information tab, in the Security And Auditing section, hover over the Enabled status to the right of RRSA OIDC to view the provider's URL link and ARN information.

3. Create a RAM role with a trusted entity of Identity Provider.

   1. Log on to the RAM console using your Alibaba Cloud account.

   2. In the left-side navigation pane, choose Identities > > Roles. On the Roles page, click Create Role.

   3. In the Create Role panel, select Identity Provider as the trusted entity type, and click Switch to Policy Editor

   4. On the Create Role page, in the Visual editor section, configure the following role information and click OK.

      Configuration item	Description
      Effect	Select Allow.
      Principal	Select the identity provider. Identity provider type: Select OIDC. Identity provider: After enabling RRSA, the ACK cluster creates a default identity provider with the naming format ack-rrsa-<cluster_id>. Specify the ID of your cluster in <cluster_id>.
      Action	Keep the default. That is, select sts:AssumeRole.
      Condition	Add a new condition based on the default oidc:iss and oidc:aud conditions: Condition key: Select oidc:sub. Operator: Select StringEquals. Condition value: system:serviceaccount:<namespace>:<serviceAccountName>. <namespace>: Specify the namespace of the application. <serviceAccountName>: the name of the service account. This example uses system:serviceaccount:rrsa-dev:dev-sa, where rrsa-dev is the Kubernetes namespace that needs to be created in the next step, and dev-sa is the Kubernetes service account that needs to be created in the next step.

   5. In the Create Role dialog box, set the role name and click OK. In this example, the role name is dev-role-for-rrsa.

   6. View the trust policy of the dev-role-for-rrsa RAM role.

      This policy allows a specific service account to assume a RAM role through Alibaba Cloud RRSA (RAM Roles for Service Accounts) after meeting OIDC authentication conditions.

      {
        "Statement": [
          {
            "Action": "sts:AssumeRole",
            "Condition": {
              "StringEquals": {
                "oidc:aud": [
                  "sts.aliyuncs.com"
                ],
                "oidc:iss": [
                  "https://oidc-ack-cn-hongkong.oss-cn-hongkong.aliyuncs.com/cf01******"
                ],
                "oidc:sub": [
                  "system:serviceaccount:rrsa-dev:dev-sa"
                ]
              }
            },
            "Effect": "Allow",
            "Principal": {
              "Federated": [
                "acs:ram::5269************:oidc-provider/ack-rrsa-cf01******"
              ]
            }
          }
        ],
        "Version": "1"
      }

4. Create a permission policy and grant it to the RAM role.

   1. For this example, the permission policy name is dev-role-for-rrsa-kms-policy, and the policy content only allows access to secrets with the env:dev tag.

      The permission policy content is as follows:

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "kms:Decrypt",
                      "kms:GetSecretValue"
                  ],
                  "Resource": "*",
                  "Condition": {
                      "StringEqualsIgnoreCase": {
                          "kms:tag/env": [
                              "dev"
                          ]
                      }
                  }
              }
          ]
      }

   2. Grant the dev-role-for-rrsa-kms-policy permission policy to the dev-role-for-rrsa role.

Step 3: Create Namespace and Service Account in ACK

Namespaces divide the ACK cluster into logically isolated virtual spaces, used to separate development, testing, production, and other environments. Applications in different namespaces cannot access each other's resources by default, providing a physical boundary for RAM role binding. Service Accounts provide identity for Pods and dynamically bind with RAM roles through the RRSA mechanism.

1. Create a Namespace using a YAML file.

   For this example, the Namespace name is rrsa-dev, and the YAML file name is namespace.yaml.

   apiVersion: v1
   kind: Namespace
   metadata:
     name: rrsa-dev

2. Execute the following command to create a Namespace named rrsa-dev in the ACK cluster.

   kubectl apply -f namespace.yaml

   Check if the Namespace was created successfully:

   kubectl get namespaces

   If the output includes rrsa-dev, the creation was successful.

3. Create a Service Account using a YAML file.

   For this example, the Service Account name is dev-sa, and the YAML file name is serviceaccount.yaml.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: dev-sa
     namespace: rrsa-dev

4. Execute the following command to create a ServiceAccount named dev-sa in rrsa-dev.

   kubectl apply -f serviceaccount.yaml

   Check if the ServiceAccount was created successfully:

   kubectl get serviceaccount -n rrsa-dev

   If the output includes dev-sa, the creation was successful.

Step 4: Create a sidecar container image

Deploy KMS Agent as a sidecar container alongside the application container in Alibaba Cloud's ACK container service.

1. Configure KMS Agent runtime parameters in config.toml.

   Note

   config.toml is the Agent's configuration file. You can find this file in the configs directory of the alibabacloud-kms-agent source code retrieved in Step 1.

   [Server]
   HttpPort = 2025
   
   [KMS]
   Region = "cn-hangzhou"
   
   [Cache]
   CacheType = "InMemory"
   CacheSize = 1000
   TtlSeconds = 300
   
   [Log]
   LogLevel = "Debug"
   LogPath = "./logs/"
   MaxSize = 100
   MaxBackups = 2

2. Create a Dockerfile.

   For example, if the Agent is deployed in the deploy/ack/agent path, create a Dockerfile in that path. For the file content, refer to agent_Dockerfile.

   # Use the centos image as the base
   FROM centos:centos7
   
   # Set the working directory inside the container
   WORKDIR /usr/local/alibabacloudkmsagent
   
   # Copy the binary and config to the container
   COPY alibabacloud-kms-agent .
   COPY config.toml .
   
   # Set the entry point to run the kms agent
   ENTRYPOINT ["./alibabacloud-kms-agent", "agent", "./config.toml"]

3. Copy the files created in steps 1 and 2 to the deploy/ack/agent directory.

   deploy/ack/agent
   ├── alibabacloud-kms-agent
   ├── config.toml
   ├── Dockerfile.agent

4. Build the sidecar container image and upload it to Alibaba Cloud Container Registry.

   # Build
   docker build -t registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:kmsagent-v1.0 .
   # Upload
   docker push registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:kmsagent-v1.0

Step 5: Create a business container image

1. For example, if the business application is deployed in the deploy/ack/app path, write a Dockerfile for the application container image in that path.

   # Use the centos image as the base
   FROM centos:centos7
   
   # Set the working directory inside the container
   WORKDIR /
   
   # Set the entry point to run the kms agent
   ENTRYPOINT ["sleep", "360000"]

2. Build the application container image and upload it to Alibaba Cloud Container Registry.

   # Build
   docker build -t registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:app-v1.0 .
   # Upload
   docker push registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:app-v1.0

Step 6: Deploy the application

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: rrsa-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      serviceAccountName: dev-sa
      containers:
      - name: kmsagent
        image: registry-vpc.cn-hangzhou.aliyuncs.com/<ns>/<repo>:kmsagent-v1.0
        env:
        - name: ALIBABA_CLOUD_ROLE_ARN
          value: acs:ram::<uid>:role/dev-role-for-rrsa
        - name: ALIBABA_CLOUD_OIDC_PROVIDER_ARN
          value: acs:ram::<uid>:oidc-provider/ack-rrsa-<ackClusterId>
        - name: ALIBABA_CLOUD_OIDC_TOKEN_FILE
          value: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token
        - name: KMS_TOKEN
          value: file:///var/run/kmstoken/token
        volumeMounts:
          - name: shared-volume
            mountPath: /var/run/kmstoken
          - name: rrsa-oidc-token
            mountPath: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens
            readOnly: true
        command: ["/bin/sh", "-c", "./alibabacloud-kms-agent token /var/run/kmstoken/token && ./alibabacloud-kms-agent agent config.toml"]
      - name: my-app
        image: registry-vpc.cn-hangzhou.aliyuncs.com/<ns/<repo>:app-v1.0
        volumeMounts:
          - name: shared-volume
            mountPath: /var/run/kmstoken
      volumes:
        - name: shared-volume
          emptyDir: {}
        - name: rrsa-oidc-token
          projected:
            defaultMode: 420
            sources:
            - serviceAccountToken:
               audience: sts.aliyuncs.com
               expirationSeconds: 3600
               path: token

Step 7: Create and retrieve secret values

1. Create secrets in KMS.

   In this example, when creating secrets, you need to add the tag env:dev. The application container can only retrieve secrets with this tag and does not have permission to retrieve other secrets.

   1. Log on to the KMS console. In the top navigation bar, select a region. In the navigation pane on the left, choose Resource > Secrets.

   2. Click the Generic Secrets tab, select Instance ID, and then click Create Secret. Configure the parameters and click OK.

      Parameter	Description
      Secret Name	The name of the secret. The secret name is unique within the current region.
      Secret Value	The type of sensitive data that you want to manage. Valid values: Secret Key/Value and Plain Text. The value cannot exceed 30,720 bytes in length, which is equivalent to 30 KB in size.
      Initial Version	The initial version of the secret. Default value: v1. You can also specify a custom version number.
      CMK	The key that is used to encrypt the current value of the secret. Important Your key and secret must belong to the same KMS instance. The key must be a symmetric key. For more information about the symmetric keys supported by KMS, see Key specifications for symmetric and asymmetric encryption. If you are a RAM user or a RAM role, you must have the permissions to call the GenerateDataKey operation by using a key.
      Tag	The tag that you want to add to the secret. You can use tags to classify and manage secrets. A tag consists of a key-value pair. Note A tag key or a tag value can be up to 128 characters in length and can contain letters, digits, forward slashes (/), backslashes (\), underscores (_), hyphens (-), periods (.), plus signs (+), equal sign (=), colons (:), at signs (@), and spaces. A tag key cannot start with aliyun or acs:. You can configure up to 20 key-value pairs for each secret.
      Description	The description of the secret.
      Advanced Settings > Policy Settings	The policy settings of the secret. For more information, see Overview. You can use the default policy and then modify the policy based on your business requirements after you create the secret.

2. Retrieve secret values.

   • Using the curl command.

     curl -v -H "X-Vault-Token:$(</var/run/kmstoken/token)" 'http://localhost:2025/secretsmanager/get?secretId=app/dev/secret-1'

   • Using an HTTP GET request in business code to retrieve secrets.

     package main
     
     import (
     	"fmt"
     	"io/ioutil"
     	"net/http"
     )
     
     func main() {
     	url := fmt.Sprintf("http://localhost:2025/secretsmanager/get?secretId=%s", "app/dev/secret-1")
     
     	token, err := ioutil.ReadFile("/var/run/kmstoken/token")
     	if err != nil {
     		fmt.Printf("error reading token file: %v\n", err)
     	}
     
     	req, err := http.NewRequest("GET", url, nil)
     	if err != nil {
     		fmt.Printf("error creating request: %v\n", err)
     	}
     
     	req.Header.Add("X-KMS-Token", string(token))
     
     	client := &http.Client{}
     	resp, err := client.Do(req)
     	if err != nil {
     		fmt.Printf("error sending request: %v \n", err)
     	}
     	defer resp.Body.Close()
     
     	body, _ := ioutil.ReadAll(resp.Body)
     	fmt.Printf("status code %d - %s \n", resp.StatusCode, string(body))
     }