Container Service for Kubernetes:Use RRSA to authorize different pods to access different cloud services

Table of contents

• Background information

• Limits

• Enable RRSA

• Work with RRSA

• Alibaba Cloud SDKs that support the OIDC token authentication of RRSA

• References

Background information

Elastic container instances run on Elastic Compute Service (ECS) instances. ECS instance metadata contains information about ECS instances on Alibaba Cloud. You can log on to a running ECS instance and view its metadata, and then configure or manage the ECS instance based on the metadata. Applications deployed in an ACK cluster can use ECS instance metadata to obtain Security Token Service (STS) tokens that are used to assume different Resource Access Management (RAM) roles. This way, the applications can call the APIs of different cloud services. For more information, see Overview of ECS instance metadata.

In multi-tenant scenarios, ECS instance metadata cannot meet the requirements for controlling the pods of different applications to access different cloud services. In scenarios where ACK Serverless clusters or elastic container instances are used, ECS instance metadata cannot meet the requirements for using STS tokens because RAM roles cannot be assigned to elastic container instances. To address these issues, ACK releases the RRSA feature which is developed based on RAM. By using the RRSA feature, you can allow different applications in an ACK cluster to assume different RAM roles. Applications can obtain STS tokens, use the tokens to assume specific RAM roles, and then access relevant cloud services. This enforces the principle of least privilege

and implements fine-grained access control on the pods of different applications in multi-tenant scenarios. This also allows you to use STS tokens as temporary access tokens when ACK Serverless clusters and elastic container instances are used.

Limits

RRSA supports only Kubernetes 1.22 and later. The cluster types that support RRSA include ACK standard cluster, ACK Pro cluster, ACK Serverless standard cluster, and ACK Serverless Pro cluster.

Enable RRSA

1. Log on to the ACK console.

2. In the left-side navigation pane of the ACK console, click Clusters.

3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.

4. On the cluster details page, click the Basic Information tab. In the Cluster Information section, click Enable RRSA for RRSA OIDC.

5. In the Enable RRSA message, click Confirm.

   In the Basic Information section, if the cluster status changes from Updating to Running, the RRSA feature is enabled for the cluster and the URL and Alibaba Cloud Resource Name (ARN) of the OIDC provider are displayed next to RRSA OIDC.

ACK performs the following operations in the background to enable RRSA:

• Automatically creates an OIDC issuer that is dedicated to the cluster. The OIDC issuer is managed by ACK. For more information, see OIDC Issuer.

• Enables service account token volume projection for your cluster. For more information, see Enable service account token volume projection.

  Note

  If your cluster already has this feature enabled, ACK overwrites the value of service-account-issuer with the setting of the OIDC issuer created in the preceding step.

• Creates a RAM role identity provider within your account. The identity provider uses the created OIDC issuer for single sign-on (SSO). The identity provider is named ack-rrsa-<cluster_id>. <cluster_id> indicates the ID of your cluster. For more information, see Manage an OIDC IdP.

Work with RRSA

After you enable RRSA for your cluster, perform the following steps to enable the applications in the cluster to obtain STS tokens through RRSA. The STS tokens are used to call the APIs of specific cloud services.

Example

In this example, an application is created and RRSA is enabled for the cluster to allow the application to perform the following operations: assuming a specified RAM role and then calling an API operation to query clusters that belong to the current Alibaba Cloud account.

Sample configurations

• Namespace: rrsa-demo

• Service account: demo-sa

• RAM role: demo-role-for-rrsa

Procedure

• If you want to enable RRSA without installing the ack-pod-identity-webhook component, you can manually specify the required OIDC token file as environment variables in the application template. For more information, see Manually modify the application template to enable RRSA.

• If you want to use an existing RAM role, you must grant the required permissions to the RAM role. For more information, see Use an existing RAM role and grant the required permissions to the RAM role.

1. Install the ack-pod-identity-webhook component.

   1. On the Clusters page, click the name of your cluster. In the left-side navigation pane, choose Operations > Add-ons.

   2. On the Add-ons page, click the Security tab, find ack-pod-identity-webhook, and then click Install in the lower-right part of the card.

   3. In the message that appears, confirm the information and click OK.

2. Create a RAM role named demo-role-for-rrsa.

   1. Log on to the RAM console with your Alibaba Cloud account.

   2. In the left-side navigation pane, choose Identities > Roles. On the Roles page, click Create Role.

   3. In the Create Role panel, select IdP for Select Trusted Entity and click Next.

   4. On the Configure Role wizard page, set the following parameters and click OK.

      The following table describes the parameters.

      Parameter	Description
      RAM Role Name	Set the value to demo-role-for-rrsa.
      Note	Enter the description of the RAM role. This parameter is optional.
      IdP Type	Select OIDC.
      Select IdP	Select ack-rrsa-<cluster_id>. <cluster_id> indicates the ID of your cluster.
      Conditions	oidc:iss: Use the default value. oidc:aud: Select sts.aliyuncs.com. oidc:sub: Set the condition operator to StringEquals and the value to system:serviceaccount:<namespace>:<serviceAccountName>. <namespace>: Specify the namespace of the application. <serviceAccountName>: Specify the name of the service account. In this example, enter system:serviceaccount:rrsa-demo:demo-sa.

3. Attach the AliyunCSReadOnlyAccess policy to the RAM role created in Step 2 to grant the required permissions to the application. For more information, see Grant permissions to a RAM role.

4. Deploy an application. For more information about the Alibaba Cloud SDKs that support the OIDC token authentication of RRSA, see Alibaba Cloud SDKs that support the OIDC token authentication of RRSA.

   1. Create a file named demo.yaml based on the following requirements:

      In the following YAML content, the pod-identity.alibabacloud.com/injection: 'on' label is added to the namespace, the pod-identity.alibabacloud.com/role-name: demo-role-for-rrsa annotation is added to the service account, and the auto injection feature of ack-pod-identity-webhook is enabled. For more information about how to configure ack-pod-identity-webhook, see ack-pod-identity-webhook.

      Sample YAML template

      ---
      apiVersion: v1
      kind: Namespace
      metadata:
        name: rrsa-demo
        labels:
          pod-identity.alibabacloud.com/injection: 'on'
      
      ---
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: demo-sa
        namespace: rrsa-demo
        annotations:
          pod-identity.alibabacloud.com/role-name: demo-role-for-rrsa
      
      ---
      apiVersion: v1
      kind: Pod
      metadata:
        name: demo
        namespace: rrsa-demo
      spec:
        serviceAccountName: demo-sa
        containers:
          - image: registry.cn-hangzhou.aliyuncs.com/acs/ack-ram-tool:1.0.0
            imagePullPolicy: "Always"
            args:
              - rrsa
              - demo
            name: demo
        restartPolicy: OnFailure

   2. Run the following command to deploy the application:

      kubectl apply -f demo.yaml

5. Run the following command to check whether ack-pod-identity-webhook has injected the required configurations into the pod that is created for the application:

   kubectl -n rrsa-demo get pod demo -o yaml

   Expected output

   apiVersion: v1
   kind: Pod
   metadata:
     name: demo
     namespace: rrsa-demo
   spec:
     containers:
     - args:
       - rrsa
       - demo
       env:
       - name: ALIBABA_CLOUD_ROLE_ARN
         value: acs:ram::1***:role/demo-role-for-rrsa
       - name: ALIBABA_CLOUD_OIDC_PROVIDER_ARN
         value: acs:ram::1***:oidc-provider/ack-rrsa-c***
       - name: ALIBABA_CLOUD_OIDC_TOKEN_FILE
         value: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token
       image: registry.cn-hangzhou.aliyuncs.com/acs/ack-ram-tool:1.0.0
       imagePullPolicy: Always
       name: demo
       volumeMounts:
       - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
         name: kube-api-access-4bwdg
         readOnly: true
       - mountPath: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens
         name: rrsa-oidc-token
         readOnly: true
     restartPolicy: OnFailure
     serviceAccount: demo-sa
     serviceAccountName: demo-sa
     volumes:
     - name: kube-api-access-4bwdg
       projected:
         defaultMode: 420
         sources:
         - serviceAccountToken:
             expirationSeconds: 3607
             path: token
         - configMap:
             items:
             - key: ca.crt
               path: ca.crt
             name: kube-root-ca.crt
         - downwardAPI:
             items:
             - fieldRef:
                 apiVersion: v1
                 fieldPath: metadata.namespace
               path: namespace
     - name: rrsa-oidc-token
       projected:
         defaultMode: 420
         sources:
         - serviceAccountToken:
             audience: sts.aliyuncs.com
             expirationSeconds: 3600
             path: token

   The output indicates that ack-pod-identity-webhook has injected the following configurations into the pod.

   Category	Configuration item	Description
   Environment variables	ALIBABA_CLOUD_ROLE_ARN	The ARN of the RAM role to be assumed.
   	ALIBABA_CLOUD_OIDC_PROVIDER_ARN	The ARN of the OIDC IdP.
   	ALIBABA_CLOUD_OIDC_TOKEN_FILE	The path of the OIDC token file.
   VolumeMount	rrsa-oidc-token	The configuration for mounting the OIDC token.
   Volume	rrsa-oidc-token	The configuration for mounting the OIDC token.

6. Run the following command to print the log of the application:

   kubectl -n rrsa-demo logs demo

   A list of clusters are displayed in the output:

   cluster id: cf***, cluster name: foo*
   cluster id: c8***, cluster name: bar*
   cluster id: c4***, cluster name: foob*

7. Optional: Detach the AliyunCSReadOnlyAccess system policy from the RAM role. For more information, see Remove permissions from a RAM role.

   Wait 30 seconds and run the following command to print the log of the application again:

   kubectl -n rrsa-demo logs demo

   The following error message indicates that the application does not have the required permissions:

      StatusCode: 403
      Code: StatusForbidden
      Message: code: 403, STSToken policy Forbidden for action cs:DescribeClusters request id: E78A2E2D-***
      Data: {"accessDeniedDetail":{"AuthAction":"cs:DescribeClusters","AuthPrincipalDisplayName":"demo-role-for-rrsa:ack-ram-tool","AuthPrincipalOwnerId":"11***","AuthPrincipalType":"AssumedRoleUser","NoPermissionType":"ImplicitDeny","PolicyType":"ResourceGroupLevelIdentityBasedPolicy"},"code":"StatusForbidden","message":"STSToken policy Forbidden for action cs:DescribeClusters","requestId":"E78A2E2D-***","status":403,"statusCode":403}

Manually modify the application template to enable RRSA

You can manually specify the required OIDC token file as environment variables in the application template. This way, you can enable RRSA without installing ack-pod-identity-webhook.

The following template provides an example.

apiVersion: v1
kind: Pod
metadata:
  name: demo
  namespace: rrsa-demo
spec:
  containers:
  - args:
    - rrsa
    - demo
    env:
    - name: ALIBABA_CLOUD_ROLE_ARN
      value: <role_arn>
    - name: ALIBABA_CLOUD_OIDC_PROVIDER_ARN
      value: <oid_provider_arn>
    - name: ALIBABA_CLOUD_OIDC_TOKEN_FILE
      value: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token
    image: registry.cn-hangzhou.aliyuncs.com/acs/ack-ram-tool:1.0.0
    imagePullPolicy: Always
    name: demo
    volumeMounts:
    - mountPath: /var/run/secrets/ack.alibabacloud.com/rrsa-tokens
      name: rrsa-oidc-token
      readOnly: true
  restartPolicy: OnFailure
  serviceAccount: demo-sa
  serviceAccountName: demo-sa
  volumes:
  - name: rrsa-oidc-token
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          audience: sts.aliyuncs.com
          expirationSeconds: 3600
          path: token

After you redeploy the application based on the modified template, the application can use the OIDC token file, the RAM role ARN, and the OIDC provider ARN that you specified in the template to call the AssumeRoleWithOIDC operation of STS and obtain the STS token that is used to assume a specific RAM role. This way, the application can call the APIs of different cloud services. The OIDC token file is specified in the ALIBABA_CLOUD_OIDC_TOKEN_FILE environment variable, the RAM role ARN is specified in the ALIBABA_CLOUD_ROLE_ARN environment variable, and the OIDC provider ARN is specified in the ALIBABA_CLOUD_OIDC_PROVIDER_ARN environment variable. For more information about the RRSA SDK demos used by the application, see Alibaba Cloud SDKs that support the OIDC token authentication of RRSA. For more information, see AssumeRoleWithOIDC.

Use an existing RAM role and grant the required permissions to the RAM role

If you want the application to use an existing RAM role, you must modify the trust policy of the RAM role based on the following template. This way, the application can use the service account to assume the RAM role and obtain an STS token. For more information, see Edit the trust policy of a RAM role.

Example of the Statement configurations:

{
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringEquals": {
      "oidc:aud": "sts.aliyuncs.com",
      "oidc:iss": "<oidc_issuer_url>",
      "oidc:sub": "system:serviceaccount:<namespace>:<service_account>"
    }
  },
  "Effect": "Allow",
  "Principal": {
    "Federated": [
      "<oidc_provider_arn>"
    ]
  }
}

You can also use the ack-ram-tool tool to automate the modification of the trust policy. Sample commands:

ack-ram-tool rrsa associate-role --cluster-id <cluster_id> \
    --namespace <namespace> --service-account <service_account> \
    --role-name <role_name> --create-role-if-not-exist

Alibaba Cloud SDKs that support the OIDC token authentication of RRSA

Some Alibaba Cloud SDKs allow applications to call the APIs of specific cloud services by using the OIDC tokens of RRSA. The following table describes the supported SDK versions.

References

• ack-pod-identity-webhook

• ServiceAccount token volume projection

• OIDC introduction

• Overview of OIDC-based SSO

• AssumeRoleWithOIDC - Obtains an STS token to assume a RAM role during SSO by using OIDC

• ack-ram-tool