Key Management Service:Deploy KMS Agent in ACK to retrieve secrets

Step 1: Build the KMS Agent executable

1. Install Go. For instructions, see the Go installation guide.

2. Download the source code and dependencies.

   1. Download and install Git from the official Git website.

   2. Run the following commands to clone the repository and download dependencies:

      git clone https://github.com/aliyun/alibabacloud-kms-agent
      go mod download

3. In the project root directory, run go build . to compile the executable. The default executable name is alibabacloud-kms-agent, saved in the project root directory.

   If the build environment and deployment environment are the same, run go build . directly. For cross-platform builds, use the following commands to generate a 64-bit executable:

   Build environment	Deploy to macOS	Deploy to Linux	Deploy to Windows
   macOS	go build .	CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build .	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build .
   Linux	CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build .	go build .	CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build .
   Windows	SET CGO_ENABLED=0 && SET GOOS=darwin && SET GOARCH=amd64 && go build .	SET CGO_ENABLED=0 && SET GOOS=linux && SET GOARCH=amd64 && go build .	go build .

4. In the project root directory, confirm that the alibabacloud-kms-agent executable exists.

   

Step 2: Enable RRSA and authorize access to secrets

1. Enable RRSA for your ACK cluster.

   Enable during cluster creation

   When creating an ACK managed cluster or ACK Edge cluster, go to the Cluster Configurations step, expand Advanced Options (Optional), and click Enable next to RRSA OIDC.

   

   Enable for an existing cluster

   1. Log on to the ACK console. In the left navigation pane, click Clusters.

   2. Click the name of your cluster. In the left navigation pane, click Cluster Information.

   3. On the Basic Information tab, scroll to the Security and Auditing section and click Enable next to RRSA OIDC.

      

   4. In the Enable RRSA dialog box, click Confirm. Wait for the cluster status to change from Updating to Running. RRSA is now enabled.

2. Go to the cluster details page. On the Basic Information tab, in the Security and Auditing section, hover over the Enabled status next to RRSA OIDC to view the provider URL and ARN.

   

3. Create a RAM role with an identity provider as the trusted entity.

   1. Log on to the RAM console as a RAM administrator. In the left-side navigation pane, choose Identities > Roles.

   2. Click Create Role. On the create page, click Switch to Policy Editor in the upper-right corner.

   3. In the editor, specify the OIDC identity provider. Select Visual Editor mode and configure the following settings:

      Note

      For more information about other modes, see Create a RAM role for an OIDC identity provider.

      1. In the Principal section, select IdP, and then click Edit.

      2. In the Add Principal dialog box, configure the following parameters:

         • IdP Type: Select OIDC.

         • IdP: After RRSA is enabled, ACK creates a default identity provider named ack-rrsa-<cluster_id>, where <cluster_id> is your cluster ID.

         • Action: Keep the default sts:AssumeRole.

         • Condition: Add a new condition alongside the default oidc:iss and oidc:aud conditions:

           • Condition key: Select oidc:sub.

           • Operator: Select StringEquals.

           • Condition value: Enter system:serviceaccount:<namespace>:<serviceAccountName>.

             • <namespace>: The namespace where the application resides.

             • <serviceAccountName>: The name of the service account.

             This example uses system:serviceaccount:rrsa-dev:dev-sa, where rrsa-dev is the Kubernetes namespace and dev-sa is the service account you will create in Step 3.

   4. In the Create Role dialog box, set a role name and click OK. This example uses dev-role-for-rrsa.

   5. Review the trust policy for dev-role-for-rrsa. The policy allows a specific service account to assume the RAM role through RRSA after passing OIDC authentication.

      {
        "Statement": [
          {
            "Action": "sts:AssumeRole",
            "Condition": {
              "StringEquals": {
                "oidc:aud": [
                  "sts.aliyuncs.com"
                ],
                "oidc:iss": [
                  "https://oidc-ack-cn-hongkong.oss-cn-hongkong.aliyuncs.com/cf01******"
                ],
                "oidc:sub": [
                  "system:serviceaccount:rrsa-dev:dev-sa"
                ]
              }
            },
            "Effect": "Allow",
            "Principal": {
              "Federated": [
                "acs:ram::5269************:oidc-provider/ack-rrsa-cf01******"
              ]
            }
          }
        ],
        "Version": "1"
      }

4. Create a permission policy and attach it to the RAM role.

   1. Create a permission policy. This example uses the policy name dev-role-for-rrsa-kms-policy and allows access only to secrets tagged with env:dev.

      

      The policy content is as follows:

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "kms:Decrypt",
                      "kms:GetSecretValue"
                  ],
                  "Resource": "*",
                  "Condition": {
                      "StringEqualsIgnoreCase": {
                          "kms:tag/env": [
                              "dev"
                          ]
                      }
                  }
              }
          ]
      }

   2. Attach the dev-role-for-rrsa-kms-policy policy to the dev-role-for-rrsa role.

      

Step 3: Create a namespace and service account

A namespace divides an ACK cluster into logically isolated virtual spaces for different environments such as development, testing, and production. By default, applications in different namespaces cannot access each other's resources. A service account provides an identity for a Pod and dynamically binds to a RAM role through RRSA.

1. Create a namespace YAML file.

   Create a file named namespace.yaml with the following content. This example creates a namespace named rrsa-dev.

   apiVersion: v1
   kind: Namespace
   metadata:
     name: rrsa-dev

2. Run the following command to create the namespace:

   kubectl apply -f namespace.yaml

   Verify the namespace was created:

   kubectl get namespaces

   If the output includes rrsa-dev, the namespace is created successfully.

3. Create a service account YAML file.

   Create a file named serviceaccount.yaml with the following content. This example creates a service account named dev-sa in the rrsa-dev namespace.

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: dev-sa
     namespace: rrsa-dev

4. Run the following command to create the service account:

   kubectl apply -f serviceaccount.yaml

   Verify the service account was created:

   kubectl get serviceaccount -n rrsa-dev

   If the output includes dev-sa, the service account is created successfully.

Step 4: Build the sidecar container image

Package KMS Agent as a sidecar container to deploy alongside your application in ACK.

1. Configure the KMS Agent runtime parameters in config.toml.

   Note

   The config.toml file is located in the configs directory of the alibabacloud-kms-agent source code obtained in Step 1.

   [Server]
   HttpPort = 2025
   
   [KMS]
   Region = "cn-hangzhou"
   
   [Cache]
   CacheType = "InMemory"
   CacheSize = 1000
   TtlSeconds = 300
   
   [Log]
   LogLevel = "Debug"
   LogPath = "./logs/"
   MaxSize = 100
   MaxBackups = 2

2. Create a Dockerfile.

   For example, create a Dockerfile in deploy/ack/agent to deploy the agent. For the file content, see agent_Dockerfile.

   # Use the centos image as the base
   FROM centos:centos7
   
   # Set the working directory inside the container
   WORKDIR /usr/local/alibabacloudkmsagent
   
   # Copy the binary and config to the container
   COPY alibabacloud-kms-agent .
   COPY config.toml .
   
   # Set the entry point to run the kms agent
   ENTRYPOINT ["./alibabacloud-kms-agent", "agent", "./config.toml"]

3. Copy the executable and configuration to the deploy/ack/agent directory:

   deploy/ack/agent
   ├── alibabacloud-kms-agent
   ├── config.toml
   ├── Dockerfile.agent

4. Build the sidecar image and push it to Container Registry:

   # Build the image.
   docker build -t registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:kmsagent-v1.0 .
   # Push the image.
   docker push registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:kmsagent-v1.0

Step 5: Build the application container image

1. For an application deployed in deploy/ack/app, create a Dockerfile in that directory:

   # Use the centos image as the base
   FROM centos:centos7
   
   # Set the working directory inside the container
   WORKDIR /
   
   # Set the entry point to run the kms agent
   ENTRYPOINT ["sleep", "360000"]

2. Build the application image and push it to Container Registry:

   # Build the image.
   docker build -t registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:app-v1.0 .
   # Push the image.
   docker push registry.cn-hangzhou.aliyuncs.com/<ns>/<repo>:app-v1.0

Step 6: Deploy the application

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: rrsa-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      serviceAccountName: dev-sa
      containers:
      - name: kmsagent
        image: registry-vpc.cn-hangzhou.aliyuncs.com/<ns>/<repo>:kmsagent-v1.0
        env:
        - name: ALIBABA_CLOUD_ROLE_ARN
          value: acs:ram::<uid>:role/dev-role-for-rrsa
        - name: ALIBABA_CLOUD_OIDC_PROVIDER_ARN
          value: acs:ram::<uid>:oidc-provider/ack-rrsa-<ackClusterId>
        - name: ALIBABA_CLOUD_OIDC_TOKEN_FILE
          value: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token
        - name: KMS_TOKEN
          value: file:///var/run/kmstoken/token
        volumeMounts:
          - name: shared-volume
            mountPath: /var/run/kmstoken
          - name: rrsa-oidc-token
            mountPath: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens
            readOnly: true
        command: ["/bin/sh", "-c", "./alibabacloud-kms-agent token /var/run/kmstoken/token && ./alibabacloud-kms-agent agent config.toml"]
      - name: my-app
        image: registry-vpc.cn-hangzhou.aliyuncs.com/<ns>/<repo>:app-v1.0
        volumeMounts:
          - name: shared-volume
            mountPath: /var/run/kmstoken
      volumes:
        - name: shared-volume
          emptyDir: {}
        - name: rrsa-oidc-token
          projected:
            defaultMode: 420
            sources:
            - serviceAccountToken:
               audience: sts.aliyuncs.com
               expirationSeconds: 3600
               path: token

Step 7: Create and retrieve a secret

1. Create a secret in KMS.

   Add the tag env:dev when you create the secret. The application container can only access secrets with this tag.

   1. Log on to the Key Management Service console. In the top navigation bar, select a region. In the left-side navigation pane, choose Resource > Secrets.

   2. Click the Generic Secrets tab, select a Instance ID, and then click Create Secret. Configure the parameters and click OK.

      Parameter	Description
      Secret Name	A custom credential name. Must be unique within the current region.
      Secret Value	Select Secret Key/Value or Plain Text and enter the sensitive data to store. The value must be up to 30,720 bytes (30 KB) in length.
      Initial Version	The initial version number. Default value: v1. Custom values are supported.
      CMK	Select the symmetric key used to encrypt the credential value. The key and the credential must belong to the same KMS instance. Important The key and the credential must belong to the same KMS instance, and the key must be a symmetric key. For more information about the symmetric keys supported by KMS, see Key types and specifications. If a RAM user or RAM role is used, it must have permissions to perform the GenerateDataKey operation using the encryption master key.
      Tags	Add tags to the credential for classification. Each tag consists of a key-value pair. Note Format of a tag key and a tag value: A tag key or a tag value can be up to 128 characters in length and can contain letters, digits, forward slashes (/), backslashes (\), underscores (_), hyphens (-), periods (.), plus signs (+), equal signs (=), colons (:), at signs (@), and spaces. A tag key cannot start with aliyun or acs:. Each credential supports up to 20 tag key-value pairs.
      Description	An optional description of the credential for identification and management.
      Advanced Settings > Policy Settings	Set the access policy for the credential to control which RAM users or roles can read or modify it. You can select the default policy first and modify it later as needed.

2. Retrieve the secret value.

   • Use the curl command:

     curl -v -H "X-KMS-Token:$(</var/run/kmstoken/token)" 'http://localhost:2025/secretsmanager/get?secretId=app/dev/secret-1'

   • Retrieve the secret in your application code via an HTTP GET request:

     package main
     
     import (
     	"fmt"
     	"io/ioutil"
     	"net/http"
     )
     
     func main() {
     	url := fmt.Sprintf("http://localhost:2025/secretsmanager/get?secretId=%s", "app/dev/secret-1")
     
     	token, err := ioutil.ReadFile("/var/run/kmstoken/token")
     	if err != nil {
     		fmt.Printf("error reading token file: %v\n", err)
     	}
     
     	req, err := http.NewRequest("GET", url, nil)
     	if err != nil {
     		fmt.Printf("error creating request: %v\n", err)
     	}
     
     	req.Header.Add("X-KMS-Token", string(token))
     
     	client := &http.Client{}
     	resp, err := client.Do(req)
     	if err != nil {
     		fmt.Printf("error sending request: %v \n", err)
     	}
     	defer resp.Body.Close()
     
     	body, _ := ioutil.ReadAll(resp.Body)
     	fmt.Printf("status code %d - %s \n", resp.StatusCode, string(body))
     }