Use a secret-free component to pull images across accounts - Container Registry

In an Alibaba Cloud multi-account architecture, an ACK cluster and a Container Registry (ACR) Enterprise Edition instance may be managed by different accounts that belong to different business organizations. To allow the ACK cluster to pull container images from the ACR instance to deploy workloads, you must establish network connectivity and grant the required permissions. This topic describes how to use the aliyun-acr-credential-helper secret-free component to pull images across accounts.

Selection guide

First, select a network connectivity method and an account authorization method that meet your business requirements. Then, select a secret-free component that supports the chosen authorization method.

For more information, see Configure network connectivity, Configure account authorization and the secret-free component, and Verify cross-account image pulling.

Note

In this topic, cross-account refers to different Alibaba Cloud accounts, not RAM users.

Prerequisites

The RAM role that is used by the secret-free component is granted the required permissions.
The ACK cluster supports the secret-free component.
- aliyun-acr-credential-helper (managed)
  - ACK Pro clusters that run Kubernetes 1.22 or later.
  - ACK Serverless clusters that run Kubernetes 1.22 or later.
  - ACK Edge clusters that run Kubernetes 1.22 or later.
- aliyun-acr-credential-helper (self-managed)
  - ACK Basic clusters that run Kubernetes 1.20.0 or later.
  - ACK Pro clusters that run Kubernetes 1.20.0 or later.
  - ACK dedicated clusters that run Kubernetes 1.20.0 or later.
The Container Registry (ACR) instance is an Enterprise instance.
Important
- Only Container Registry Personal Edition instances that were created on September 8, 2024 or earlier and Container Registry Enterprise Edition instances support aliyun-acr-credential-helper.
- Container Registry Personal Edition instances created on or after September 9, 2024 do not support aliyun-acr-credential-helper. If you want to pull images from Container Registry Personal Edition instances that were created later than September 8, 2024, we recommend that you store the username and token that are used to log on to the Personal Edition instance in a secret and then reference the secret in the imagePullSecrets parameter in the YAML file of the workload.

Procedure

Step 1: Configure network connectivity

When you pull images across accounts, the ACK cluster and the ACR Enterprise Edition instance are in different VPCs that belong to different accounts and may be in different regions. Before you can pull images, you must ensure that the network is connected and the related domain names can be parsed. The following methods are available:

Internet connection: Configure a public endpoint for the ACR Enterprise Edition instance and enable Internet access for the ACK cluster. Images are transferred over the Internet. However, transferring data over the Internet is less secure and incurs EIP and data transfer fees.
VPC peering connection: You can use a VPC peering connection to connect the two VPCs. This allows the ACK cluster to access the ACR Enterprise Edition instance. VPC peering connections are free of charge if the VPCs are in the same region, but are charged if the VPCs are in different regions. This method requires that the two VPCs have non-overlapping CIDR blocks. This may reduce the number of available CIDR blocks in the VPCs. In addition, if the CIDR blocks of the two VPCs overlap, you must modify the existing network architecture.
CEN connection: A CEN instance can contain one or more transit routers. The transit routers can be connected using inter-region connections to implement cross-region and cross-account VPC-to-VPC connections.

For a comparison between VPC peering connections and CEN connections, see What are the differences between CEN and virtual private cloud (VPC) peering connections?

Comparison item	Internet connection	VPC peering connection	CEN connection
Network type	Internet	Private	Private
Billing	Fees are charged based on the billing methods of Elastic IP Address.	No fees are charged if the VPCs of the ACK cluster and the ACR Enterprise Edition instance are in the same region. If the VPCs are in different regions, Cloud Data Transfer (CDT) charges data transfer fees for outbound data transfer.	Fees are charged based on the billing rules of CEN.
Key features	You do not need to modify the existing network architecture. For security, you need to consider inbound rules and access control.	Images are transferred over an internal network, which provides high security. No fees are charged if the VPCs are in the same region. This feature requires the two VPCs to share CIDR blocks, which reduces the number of available CIDR blocks in the VPCs. If many used CIDR blocks in the two VPCs overlap, you need to modify the existing network architecture.	Images are transferred over an internal network, which provides high security. CEN is designed for enterprise production environments that have high security requirements.
Configuration procedure	Configure a public endpoint for the ACR instance By configuring an access control policy for the Internet, you can remotely and securely manage and access the ACR Enterprise Edition instance. Enable Internet access for the ACK cluster Configure the ACK cluster to access external resources over the Internet to pull images from the ACR Enterprise Edition instance.	Configure internal DNS resolution for the ACR instance After you connect the ACR Enterprise Edition instance to the VPC, you can access the Enterprise Edition instance from the VPC using an internal endpoint. After the configuration is complete, you need to obtain the VPC ID and the internal IP address of the ACR Enterprise Edition instance. Obtain the domain names and IP addresses related to the ACR instance Obtain the domain name and IP address of the authentication service and the domain name and IP address of the associated OSS bucket that are used to access the ACR Enterprise Edition instance. Create a VPC peering connection and configure a route table You need to add a route that points to the peer VPC at each end of the VPC peering connection to enable private connectivity between the VPC of the ACK cluster and the VPC of the ACR Enterprise Edition instance. At the VPC peering connection end of the ACK cluster, you also need to configure routes for the IP address of the authentication service and the IP address of the associated OSS bucket. Resolve the domain name of the ACR instance for the ACK cluster You can use methods such as adding an internal DNS resolution record or using a custom data script for a node pool to modify the `/etc/hosts` file in batches. This resolves the domain name of the ACR instance to the internal IP address of the ACR instance for the ACK cluster. This way, traffic is forwarded to the VPC of the ACR instance through the route of the VPC peering connection.	Configure internal DNS resolution for the ACR instance After you connect the ACR Enterprise Edition instance to the VPC, you can access the Enterprise Edition instance from the VPC using an internal endpoint. After the configuration is complete, you need to obtain the VPC ID and the internal IP address of the ACR Enterprise Edition instance. Obtain the domain names and IP addresses related to the ACR instance Obtain the domain name and IP address of the authentication service and the domain name and IP address of the associated OSS bucket that are used to access the ACR Enterprise Edition instance. Configure a cross-account VPC-to-VPC connection using CEN Connect the VPC of the ACR Enterprise Edition instance to the transit router in the region of the account. Connect the VPC of the ACK cluster to the transit router in the region of the account. Then, use an inter-region connection to connect the transit routers in the two regions. Configure the route tables of the VPC and the transit router For the VPC and transit router of the ACK cluster, you also need to configure routes for the IP address of the authentication service and the IP address of the associated OSS bucket. Resolve the domain name of the ACR instance for the ACK cluster You can use methods such as adding an internal DNS resolution record or using a custom data script for a node pool to modify the `/etc/hosts` file in batches. This resolves the domain name of the ACR instance to the internal IP address of the ACR instance for the ACK cluster. This way, traffic is forwarded to the VPC of the ACR instance through the CEN route.

Step 2: Configure account authorization and the secret-free component

The following three authorization methods are available for pulling images across accounts without credentials. You can select the most suitable configuration based on your scenario.

Comparison item	Use RRSA	Use Worker RAM role assumption	Use the AccessKey pair of a RAM user
Cluster type	Supported in ACK managed cluster Basic Edition, ACK managed cluster Pro Edition, ACK Edge cluster, and ACK Serverless cluster Pro Edition of version 1.22 and later.	Supported in ACK managed cluster Basic Edition, ACK managed cluster Pro Edition, and ACK dedicated cluster of version 1.20 and later.	Supported in ACK managed cluster Basic Edition, ACK managed cluster Pro Edition, and ACK dedicated cluster of version 1.20 and later.
Supported components	aliyun-acr-credential-helper managed component aliyun-acr-credential-helper component You must upgrade the component to v23.02.06.1-74e2172-aliyun or later. For more information about the differences between the components, see Comparison of secret-free components.	aliyun-acr-credential-helper component	aliyun-acr-credential-helper component
Permission granularity	Pod-level (fine-grained)	Cluster-level (medium-grained)	Account-level (coarse-grained)
Security	High. This method provides fine-grained permission control and isolation. It uses STS temporary credentials without hard-coded AccessKey pairs.	Medium. All pods share permissions, which may cause excessive permission risks.	Low. The risk of AccessKey pair leakage is high.
Scenarios	This method is suitable for security-sensitive services and production environments that require strict permission control.	This method is suitable for scenarios that require unified permissions, and development and test environments that require a certain level of permission control.	This method is suitable for quick deployments or demo environments.

Use RRSA

In the ACK cluster of Account A, configure a specific ServiceAccount to assume a RAM role of Account B that has the permissions to pull images without credentials. This allows the ACK cluster to access and pull private images from Account B.

Note

To enable the RRSA feature of the secret-free component, you must first enable RRSA in the cluster and then configure RRSA for the secret-free component. If you perform these steps in the wrong order, you must delete the pod of the secret-free plug-in to activate the RRSA feature.

In Account A, enable the RRSA feature for the ACK cluster and create a RAM role that has the permissions to assume roles.
1. Enable the RRSA feature for the ACK cluster.
  1. Log on to the ACK console. In the navigation pane on the left, click Clusters.
  2. On the Clusters page, find the target cluster and click its name. In the navigation pane on the left, click Cluster Information.
  3. In the Security and Auditing section of the Basic Information tab, click Enable next to RRSA OIDC.
  4. In the Enable RRSA dialog box, click Confirm.
    In the Basic Information section, if the cluster status changes from Updating to Running, the RRSA feature is enabled for the cluster.
2. Use the script editor to edit the trust policy and create a RAM role for an OIDC IdP.
  - Replace <oidc_issuer_url> in the example with the URL of the OIDC IdP of the current cluster that you obtained in the preceding step.
  - Replace <oidc_provider_arn> in the example with the ARN of the OIDC IdP of the current cluster that you obtained in the preceding step.
```
{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "oidc:aud": "sts.aliyuncs.com",
          "oidc:iss": "<oidc_issuer_url>",
          "oidc:sub": "system:serviceaccount:kube-system:aliyun-acr-credential-helper"
        }
      },
      "Effect": "Allow",
      "Principal": {
        "Federated": [
          "<oidc_provider_arn>"
        ]
      }
    }
  ],
  "Version": "1"
}
```
3. Attach the AliyunSTSAssumeRoleAccess policy to the role to grant it the permissions to assume roles, and record its ARN. For more information, see Grant permissions to a RAM role.
  1. On the role details page, click the Permission Management tab, and then click Grant Permission.
  2. In the Grant Permission panel, in the Access Policy section, find and select the AliyunSTSAssumeRoleAccess policy, and then click Confirm Grant Permission.
  3. On the role details page, find and record the ARN of the RAM role in the Basic Information section. For more information, see How do I view the ARN of a RAM role?
In Account B, create a RAM role, grant it the permissions to pull private images, and allow the RAM role of Account A to assume this role.
1. Use the script editor to edit the trust policy and create a RAM role for a trusted Alibaba Cloud account to allow Account A to assume the role.
```
{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "RAM": [
          "<ARN of the role created in Account A>"
        ]
      }
    }
  ],
  "Version": "1"
}
```
2. Create a custom policy with the following content and grant the policy to the RAM role to grant the role the permissions to obtain instance information and pull images.
```
{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "cr:GetAuthorizationToken",
                "cr:ListInstanceEndpoint",
                "cr:PullRepository"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}
```
3. Set the maximum session duration for the RAM role to a value between 3,600 seconds and 43,200 seconds. The default value is 3,600 seconds.
  Ensure that this value is the same as the value of the expireDuration parameter that you configure later. The value of expireDuration cannot exceed the maximum session duration.
4. On the role details page, find and record the ARN of the RAM role in the Basic Information section.

In Account A, install the secret-free component for the ACK cluster and modify its configuration items.

For more information about the differences between the components, see Comparison of secret-free components.

aliyun-acr-credential-helper managed component

Log on to the ACK console. In the navigation pane on the left, click Clusters.
On the Clusters page, find the cluster you want and click its name. In the left-side pane, choose Operations > Add-ons.
On the Add-ons page, click the Security tab, find the aliyun-acr-credential-helper (Managed) card in the Security section, and then click Install in the lower-right corner of the card.

In the Aliyun-acr-credential-helper Parameter Configuration dialog box, select Enable RRSA, click Add on the right, enter the following parameters, and then click OK.

791184c653f445f62c3387d88ad8bdcf

Configuration of the associated ACR Enterprise Edition instance:

Parameter	Description	Example
instanceId	The ID of the ACR instance. To specify multiple IDs, separate them with commas (,).	cri-XXXXX
regionId	The region ID of the ACR instance.	cn-hangzhou
domains	The domain names used by the ACR instance. Enter all endpoints (Internet and VPC) of the ACR instance. To specify individual domain names, separate them with commas (,).	XXXXX-registry.cn-hangzhou.cr.aliyuncs.com
assumeRoleARN	The ARN of the RAM role of the ACR instance owner. Enter the ARN of the RAM role created in Account B.	acs:ram::100XXXXXXXX9630:role/XXXX
expireDuration	The validity period of the temporary credentials in a cross-account scenario. Enter the maximum session duration of the RAM role created in Account B.	3600
rrsaRoleARN	The ARN of the RAM role of the ACK cluster owner. Enter the ARN of the RAM role created in Account A.	acs:ram::128XXXXXXXXXX09011:role/XXXX
rrsaOIDCProviderRoleARN	The ARN of the OIDC IdP of the ACK cluster. Enter the ARN of the RRSA OIDC IdP of the ACK cluster in Account A.	acs:ram::128XXXXXXXXXX09011:oidc-provider/ack-rrsa-c8864XXXXXXXXXXXXXXXXXX99356a636

For more information about other parameters, see Component configuration.

aliyun-acr-credential-helper component

Log on to the ACK console. In the navigation pane on the left, click Clusters.
On the Clusters page, find the cluster you want and click its name. In the left-side pane, choose Operations > Add-ons.
On the Component Management page, click the Security tab, find the aliyun-acr-credential-helper component, and then click Install. In the dialog box that appears, set the tokenMode option to auto and click OK.

Modify the ConfigMap configuration items of the secret-free component.

In the navigation pane on the left, choose Configuration Management > ConfigMaps.

On the Configuration Items page, select kube-system from the Namespace drop-down list. Then, click YAML Edit in the Actions column for acr-configuration to modify the configuration as shown in the following examples.

data:
  service-account: "default"
  watch-namespace: "all"
  expiring-threshold: "15m"
  notify-email: "c*@example.com"
  acr-registry-info: |
    - instanceId: "cri-xxx"                                   # The ID of the ACR instance.
      regionId: "cn-hangzhou"                                 # The region ID of the ACR instance.
      domains: "xxxxx-registry.cn-hangzhou.cr.aliyuncs.com"   # The endpoint of the ACR instance.
      rrsaRoleARN: "<ARN of the role created in Account A>"
      rrsaOIDCProviderRoleARN: "<The ARN of the OIDC IdP in the basic information of the ACK cluster in the console for Account A.>"
      assumeRoleARN: "<ARN of the role created in Account B>"
      expireDuration: 3600                                    # The maximum session duration of the RAM role in Account B. The default value is 3600.             
  rrsa: |
    enable: true                                              # Enable the RRSA feature for the secret-free component.

Use Worker RAM role assumption

In the ACK cluster of Account A, configure the default Worker RAM role of the cluster to assume a RAM role of Account B that has the permissions to pull images without credentials. This allows the ACK cluster to access and pull private images from Account B.

In Account A, view the Worker RAM role of the cluster and grant it the permissions to assume roles.
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, find the target cluster and click its name. In the navigation pane on the left, click Cluster Information.
3. On the Cluster Information page, click the Basic Information tab. In the Cluster Resources section, click the link to the right of Worker RAM Role.
4. Attach the AliyunSTSAssumeRoleAccess policy to the role to grant it the permissions to assume roles, and record its ARN. For more information, see Grant permissions to a RAM role.
  1. On the role details page, click the Permission Management tab, and then click Grant Permission.
  2. In the Grant Permission panel, in the Access Policy section, find and select the AliyunSTSAssumeRoleAccess policy, and then click Confirm Grant Permission.
  3. On the role details page, find and record the ARN of the RAM role in the Basic Information section. For more information, see How do I view the ARN of a RAM role?
In Account B, create a RAM role, grant it the permissions to pull private images, and allow the Worker RAM role of the ACK cluster in Account A to assume this role.
1. Create a RAM role for a trusted Alibaba Cloud account.
2. Create a custom policy with the following content and grant the policy to the RAM role to grant the role the permissions to obtain instance information and pull images.
```
{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "cr:GetAuthorizationToken",
                "cr:ListInstanceEndpoint",
                "cr:PullRepository"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}
```
3. On the basic information page of the RAM role, click the Trust Policy tab and click Edit Trust Policy to modify the policy with the following content. This allows the Worker RAM role of the ACK cluster in Account A to assume the RAM role of Account B.
```
{
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "RAM": [
          "<ARN of the role created in Account A>"
        ]
      }
    }
  ],
  "Version": "1"
}
```
4. On the role details page, find and record the ARN of the RAM role in the Basic Information section. For more information, see How do I view the ARN of a RAM role?
In Account A, install the secret-free component for the ACK cluster and modify its configuration items.
1. Log on to the ACK console. In the navigation pane on the left, click Clusters.
2. On the Clusters page, find the cluster you want and click its name. In the left-side pane, choose Operations > Add-ons.
3. On the Component Management page, click the Security tab, find the aliyun-acr-credential-helper component, and then click Install. In the dialog box that appears, set the tokenMode option to workerRole and click OK.
4. Modify the ConfigMap configuration items of the secret-free component.
  1. In the navigation pane on the left, choose Configuration Management > ConfigMaps.
  2. On the Configuration Items page, select kube-system from the Namespace drop-down list. Then, click YAML Edit in the Actions column for acr-configuration to modify the configuration as shown in the following examples.
```
data:
    service-account: "default"
    watch-namespace: "all"
    expiring-threshold: "15m"
    notify-email:"c*@example.com"
    acr-registry-info: |
      - instanceId: "cri-xxx"                                    # The ID of the ACR Enterprise Edition instance.
        regionId: "cn-hangzhou"                                  # The region ID of the ACR Enterprise Edition instance.
        domains: "xxxxx-registry.cn-hangzhou.cr.aliyuncs.com"    # The endpoint of the ACR Enterprise Edition instance.
        assumeRoleARN: "<ARN of the role created in Account B>"
        expireDuration: 3600                                     # The maximum session duration of the RAM role in Account B. The default value is 3600.
```

Use the AccessKey pair of a RAM user

In the ACK cluster of Account A, the secret-free component saves the AccessKey ID and AccessKey secret of a RAM user from Account B to pull private images from Account B. Although this method is simple to configure, the AccessKey ID and AccessKey secret are stored in plaintext, which poses a security risk.

In Account B, create a RAM user and ensure that the RAM user has the cr.* permissions.
1. Create a RAM user.
2. Create a custom policy with the following content and grant the policy to the RAM user to grant the user the permissions to obtain instance information and pull images.
```
{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "cr:GetAuthorizationToken",
                "cr:ListInstanceEndpoint",
                "cr:PullRepository"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}
```
3. View the AccessKey pair information of the RAM user and record it.
In Account A, install the secret-free component for the ACK cluster and modify its configuration items.
1. Log on to the ACK console. In the navigation pane on the left, click Clusters.
2. On the Clusters page, find the cluster you want and click its name. In the left-side pane, click Cluster Information.
3. On the Clusters page, find the cluster you want and click its name. In the left-side pane, choose Operations > Add-ons.
4. On the Component Management page, click the Security tab, find the aliyun-acr-credential-helper component, and then click Install. In the dialog box that appears, set the tokenMode option to auto and click OK.
5. Modify the ConfigMap configuration items of the secret-free component.
  1. In the navigation pane on the left, choose Configuration Management > ConfigMaps.
  2. On the Configuration Items page, select kube-system from the Namespace drop-down list. Then, click YAML Edit in the Actions column for acr-configuration to modify the configuration as shown in the following examples.

Step 3: Verify cross-account image pulling

Note

The verification procedure is for demonstration purposes only. For more information, see Build images and Create a workload.

In the ACR Enterprise Edition instance of Account B, obtain the Public Endpoint or VPC address of the container image as needed.

In the ACK cluster of Account A, choose Workloads > Deployments and use the container image to create a workload.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment-basic
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: ******.cn-hangzhou.cr.aliyuncs.com/instance/instancetest:v1 # Specify the address of the ACR image in Account B. 
        ports:
        - containerPort: 80

If the message Successfully pulled image "XXX" in XXXs (XXXs including waiting). Image size: XXX bytes. appears in the pod events of the workload, it indicates that the image was successfully pulled from a different account using the secret-free component.

FAQ

How do I resolve IP address conflicts in the 100.0.X.X/8 CIDR block?

When you configure the routing rule, the domain names of the authentication service and the OSS bucket are mapped to IP addresses that belong to the 100 CIDR block. If the 100 CIDR block is assigned to a VPC in which the Enterprise Edition instance resides, domain name conflicts may occur when you access the Enterprise Edition instance. To prevent the conflicts from occurring, you can use the following solutions:

Authentication domain name CIDR block conflict

You can enable the feature that allows an Enterprise Edition instance to take over the domain name of the authentication service and only need to access the domain name of the Enterprise Edition instance. This solves the conflict with the CIDR block that is mapped to the domain name of the authentication service.

Log on to the Container Registry console.
In the top navigation bar, select a region.
On the Instances page, click the Enterprise Edition instance that you want to manage.
In the left-side navigation pane of the management page of the Enterprise Edition instance, choose Repository > Domain. On the Domain page, turn on Instance Taking over Authentication Domain Name.
Important
If you want to use the feature that allows the Enterprise Edition instance to take over the authentication domain name, submit a ticket to request to add the Enterprise Edition instance to the whitelist.
In the Confirm to Enable Instance Taking over Authentication Domain Name message, click OK.

OSS domain name CIDR block conflict

To prevent the conflict with the CIDR block that is mapped to the domain name of the OSS bucket from occurring, you can use PrivateLink to access the OSS bucket and add a CNAME record to point the domain name of the OSS bucket to the domain name of the PrivateLink connection. For more information, see Access OSS by using PrivateLink.