You can use the P2P acceleration feature in managed, dedicated, and serverless Kubernetes clusters of Container Service for Kubernetes (ACK) to accelerate image pulling and reduce the time used to deploy applications. This topic describes how to use the P2P acceleration feature in managed, dedicated, and serverless Kubernetes clusters of ACK.

Prerequisites

  • A Container Registry Enterprise Edition instance is created. The Container Registry Enterprise Edition instance must be of the standard or advanced edition. For more information, see Create a Container Registry Enterprise Edition instance.
  • Access control is configured. If the ACK cluster needs to access a Container Registry Enterprise Edition instance over a virtual private cloud (VPC), configure access to the instance over the VPC where the ACK cluster resides. For more information, see Configure access over VPCs. If the ACK cluster needs to access a Container Registry Enterprise Edition instance over the Internet, enable access over the Internet and configure the whitelist for the instance. For more information, see Configure access over the Internet.

Step 1: Grant read permissions on Container Registry resources

If the ACK cluster is a serverless Kubernetes cluster, you must grant read permissions on Container Registry resources to the P2P component.
Note If the ACK cluster is a managed or dedicated Kubernetes cluster, you do not need to grant read permissions on Container Registry resources to the P2P component. The permissions have been granted by default.
  1. Create a RAM role.
    When you create the RAM role, set the Role Type parameter to Normal Service Role and the trusted service to Elastic Compute Service (ECS). For more information, see Create a RAM role for a trusted Alibaba Cloud service.
  2. Configure the permissions of the RAM role.
    Attach the AliyunContainerRegistryReadOnlyAccess policy to the RAM role. For more information, see Grant permissions to a RAM role.

Step 2: Obtain the ID of the Container Registry instance

  1. Log on to the Container Registry console.
  2. In the top navigation bar, select a region.
  3. In the left-side navigation pane, click Instances.
  4. On the Instances page, click the required Container Registry Enterprise Edition instance.
  5. View the ID of the Container Registry instance in the upper-left corner of the Overview page.

Step 3: Install the P2P component

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, choose Marketplace > App Catalog.
  3. On the App Catalog page, search for the ack-acr-acceleration-p2p component. After the ack-acr-acceleration-p2p component is found, click the card of the component.
  4. On the Parameters tab of the page that appears, set the registryInstances parameter to the Container Registry instance ID that you obtained in Step 2.
    If multiple Container Registry instances are used, separate the instance IDs with commas (,) when you set the registryInstances parameter.
    Note By default, the P2P component uses port 65001 on nodes. If port 65001 has been used by another component, change the port used by the P2P component based on your business requirements.
    p2p:
      ...
      # Port of P2P Agent in host network
      port: 65001
    
      # Id of ACR registry instances, support multi, e.g. "cri-xxx,cri-yyy"
      registryInstances: <ACR instance Id>
  5. Optional:Set the controller.ramRole parameter to the name of the RAM role that you created in Step 1.
    Note The controller.ramRole parameter is required only for serverless Kubernetes clusters. You can skip this step for other clusters.
    controller:
      ...
      # Ask cluster setting, in order to accessing ACR OpenAPI(Get*, List*) for ECS
      ramRole: <your ram role name>
  6. Optional:If your cluster and the Container Registry Enterprise Edition instance are in different regions or VPCs, the cluster can access the Container Registry Enterprise Edition instance over the Internet. In this case, you must set the nested parameter enable under the plusMode parameter to true and specify the region of the Container Registry Enterprise Edition instance.
    # Your cluster and the Container Registry Enterprise Edition instance are in different regions. Specify the region of the Container Registry Enterprise Edition instance.
    region: ""
    
    p2p:
      plusMode:
        # If you enable the plus mode, the scheduler and registry-mirror components are deployed to your cluster. 
        enable: true
        
        # Valid values: p2p and source. 
        # A value of p2p indicates that images are downloaded in P2P mode. 
        # A value of source indicates that images are downloaded from the registry-mirror component or a registry in the cloud. If you set the parameter to source, the scheduler component is not deployed to your cluster. 
        downloadPattern: "p2p"
        
        # The retention period of the data cached by the registry-mirror component. 
        cacheTTL: 24h
        
        # The path to which the data is cached by the registry-mirror component on the host. 
        # If this parameter is not specified, the cache will be lost when the registry-mirror component restarts. 
        cacheHostPath: ""    
        
        # If you set the parameter to true, the transfer acceleration feature of Object Storage Service (OSS) is enabled during the pull from origin over the Internet. This feature is applicable to scenarios where the Internet is unstable. For example, you can enable this feature when images are pulled across regions over the Internet. 
        # Before the configuration, make sure that the transfer acceleration feature is enabled in the OSS console. 
        ossInternetAccelerate: false    
       
        # If you set the parameter to true, the scheduler and registry-mirror components are deployed on a specified node. 
        # Before the configuration, you must add the following label to the node: k8s.aliyun.com/p2p-plus-node=true. 
        nodeSelectorEnable: false
  7. Optional:If you want to specify the upper limit of the total bandwidth for uploading and downloading by the P2P component, set the ratelimit parameter. The default value is 512 MB/s. You can specify a value based on the bandwidth of nodes.
    p2p:
      # Total net rate limit (MBytes/s) for uploading and downloading
      ratelimit: "512M"
  8. Optional:To accelerate the pull from origin over the Internet, you can enable the transfer acceleration feature for the OSS bucket that is used by the Container Registry Enterprise Edition instance.
    1. Enable the transfer acceleration feature in the OSS console. For more information, see Enable transfer acceleration.
    2. On the Parameters tab of the ack-acr-acceleration-p2p component details page, change the value of the ossInternetAccelerate parameter to true.
      ossInternetAccelerate: true
  9. In the Deploy pane, select a cluster and click Create.

Step 4: Enable P2P acceleration

You can add the P2P acceleration label to workloads such as pods and Deployments to enable P2P acceleration for these workloads. You can also add the P2P acceleration label to a namespace in your ACK cluster. This way, P2P acceleration is enabled for all workloads that meet acceleration conditions in this namespace. This way, you do not need to modify the YAML files of specific workloads to enable P2P acceleration. Select a method to add the P2P acceleration label based on your business requirements.
Note The name of the P2P acceleration label is k8s.aliyun.com/image-accelerate-mode and the value is p2p.
  • Add the P2P acceleration label to a workload
    In this example, add the P2P acceleration label to a Deployment. Run the following command to edit the YAML file of the Deployment:
    kubectl edit deploy <Deployment name>
    Add the label k8s.aliyun.com/image-accelerate-mode: p2p to the YAML file of the Deployment.
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: test
      labels:
        app: nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            # enable P2P
            k8s.aliyun.com/image-accelerate-mode: p2p
            app: nginx
        spec:
          # your ACR instacne image pull secret
          imagePullSecrets:
          - name: test-registry
          containers:
          # your ACR instacne image
          - image: test-registry-vpc.cn-hangzhou.cr.aliyuncs.com/docker-builder/nginx:latest
            name: test
            command: ["sleep", "3600"]
  • Add the P2P acceleration label to a namespace
    • Add the P2P acceleration label to a namespace in the ACK console
      1. Log on to the ACK console.
      2. In the left-side navigation pane, click Clusters.
      3. On the Clusters page, find the cluster that you want to manage and click the cluster name or click Details in the Actions column.
      4. In the left-side navigation pane, click Namespaces and Quotas.
      5. On the Namespace page, find the namespace that you want to configure and click Edit in the Actions column.
      6. In the Label section of the Edit Namespace dialog box, set the Variable Key parameter to k8s.aliyun.com/image-accelerate-mode and the Variable Value parameter to p2p, and click OK.
    • Add the P2P acceleration label to a namespace by using kubectl
      kubectl label namespaces <your-namespace> k8s.aliyun.com/image-accelerate-mode=p2p

Verify P2P acceleration

After P2P acceleration is enabled for a pod, the P2P component automatically adds P2P-related information to the YAML file of the pod. The information includes P2P-related annotations, the address of the P2P-accelerated image, and the Secret for pulling the P2P-accelerated image.
Notice

The Secret for pulling a P2P-accelerated image and the Secret for pulling the original image are different only in the domain name of the image repository. Other configurations of the two Secrets are the same. If the user information is invalid in the Secret for pulling the original image, the P2P-accelerated image also fails to be pulled.

Run the following command to view the YAML file of the pod:

kubectl get po <Pod name> -oyaml

Expected output:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    # inject p2p-annotations automatically
    k8s.aliyun.com/image-accelerate-mode: p2p
    k8s.aliyun.com/p2p-config: '...'
spec:
  containers:
   # inject image to p2p endpoint
   - image: test-registry-vpc.distributed.cn-hangzhou.cr.aliyuncs.com:65001/docker-builder/nginx:latest
  imagePullSecrets:
  - name: test-registry
  # inject image pull secret for p2p endpoint
  - name: acr-credential-test-registry-p2p

If P2P-related annotations, the address of the P2P-accelerated image, and the Secret for pulling the P2P-accelerated image exist in the YAML file, P2P acceleration is enabled.