The P2P acceleration feature uses the internal bandwidth resources of compute nodes to distribute images among the nodes. This feature can accelerate image pulling to reduce the time required to deploy applications. If a large number of nodes in a container cluster need to pull an image at the same time, you can use the P2P acceleration feature to accelerate the pulling. This topic describes how to use the P2P acceleration feature in a Container Service for Kubernetes (ACK) cluster and the Docker environment.

Prerequisites

  • A Container Registry Enterprise Edition instance is created. The Container Registry Enterprise Edition instance must be of the standard or advanced edition. For more information, see Create a Container Registry Enterprise Edition instance.
  • The Container Registry Enterprise Edition instance is configured to allow access over the VPC where your ACK cluster resides. For more information, see Configure access over VPCs.

Background information

Images are stored in Object Storage Service (OSS). When a large number of nodes in a container cluster pull an image at the same time, the network bandwidth of the OSS service becomes the performance bottleneck. As a result, it takes a long period of time to pull the image to all the nodes. The P2P acceleration feature uses the internal bandwidth resources of compute nodes to distribute images among the nodes. This reduces the pressure on OSS that stores the images and significantly increases the image pulling speed. The time required to deploy applications is greatly reduced. A test was performed in which 1,000 nodes pulled an image of 1 GB in size at the same time in P2P mode and then in regular mode where the OSS bandwidth is 10 Gbit/s. The test result shows that the P2P mode reduces the image pulling time by about 95% or more, compared with the regular mode.

Use P2P acceleration in an ACK cluster

Step 1: Grant read permissions on Container Registry resources

If the ACK cluster is a serverless Kubernetes cluster, you must grant read permissions on Container Registry resources to the P2P component.
Note If the ACK cluster is a managed or dedicated Kubernetes cluster, you do not need to grant read permissions on Container Registry resources to the P2P component. The permissions have been granted by default.
  1. Create a RAM role.
    When you create the RAM role, set Role Type to Normal Service Role and the trusted service to Elastic Compute Service (ECS). For more information, see Create a RAM role for a trusted Alibaba Cloud service.
  2. Configure the permissions of the RAM role.
    Attach the AliyunContainerRegistryReadOnlyAccess policy to the RAM role. For more information, see Grant permissions to a RAM role.

Step 2: Obtain the ID of the Container Registry instance

  1. Log on to the Container Registry console.
  2. In the top navigation bar, select a region.
  3. In the left-side navigation pane, click Instances.
  4. On the Instances page, click the required Container Registry Enterprise Edition instance.
  5. View the ID of the Container Registry instance in the upper-left corner of the Overview page.

Step 3: Install the P2P component

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, choose Marketplace > App Catalog.
  3. On the App Catalog page, search for the ack-acr-acceleration-p2p component. After ack-acr-acceleration-p2p is found, click the card of the component.
  4. On the Parameters tab of the page that appears, set the registryInstances parameter to the Container Registry instance ID that you obtained in Step 5.
    If multiple Container Registry instances are used, separate the instance IDs with commas (,) when you set the registryInstances parameter.
    Note By default, the P2P component uses port 65001 on nodes. If port 65001 has been used by another component, change the port used by the P2P component based on your business requirements.
    p2p:
      ...
      # Port of P2P Agent in host network
      port: 65001
    
      # Id of ACR registry instances, support multi, e.g. "cri-xxx,cri-yyy"
      registryInstances: <ACR instance Id>
  5. Optional:Set the controller.ramRole parameter to the name of the RAM role that you created in Step 1.
    Note You need to set the controller.ramRole parameter only for serverless Kubernetes clusters. You can skip this step for other clusters.
    controller:
      ...
      # Ask cluster setting, in order to accessing ACR OpenAPI(Get*, List*) for ECS
      ramRole: <your ram role name>
  6. In the Deploy pane, select a cluster and click Create.

Step 4: Enable P2P acceleration

You can attach the P2P acceleration label to objects such as pods and Deployments to enable P2P acceleration for these objects. You can also attach the P2P acceleration label to a namespace in your ACK cluster. This way, P2P acceleration is enabled for all objects that meet acceleration conditions in this namespace. This way, you do not need to modify the YAML files of specific objects to enable P2P acceleration. Select a method to add the P2P acceleration label based on your business requirements.
Note The name of the P2P acceleration label is k8s.aliyun.com/image-accelerate-mode and the value is p2p.
  • Attach the P2P acceleration label to an object
    The following example shows how to attach the label to a pod. Run the following command to go to the editing page of the pod:
    kubectl edit pod <Pod name>
    Add the label k8s.aliyun.com/image-accelerate-mode=p2p to the YAML file of the pod.
    apiVersion: v1
    kind: Pod
    metadata:
      labels:
        # enable P2P
        k8s.aliyun.com/image-accelerate-mode: p2p
      name: test
    spec:
      containers:
       # your ACR instacne image
       - image: test-registry-vpc.cn-hangzhou.cr.aliyuncs.com/docker-builder/nginx:latest
         name: test
         command: ["sleep", "3600"]
      # your ACR instacne image pull secret
      imagePullSecrets:
      - name: test-registry
  • Attach the P2P acceleration label to a namespace
    • Attach the P2P acceleration label to a namespace in the ACK console
      1. Log on to the ACK console.
      2. In the left-side navigation pane, click Clusters.
      3. On the Clusters page, find the cluster that you want to manage and click the cluster name or click Details in the Actions column. The cluster details page appears.
      4. In the left-side navigation pane, click Namespaces.
      5. On the Namespaces page, find the namespace that you want to configure and click Edit in the Actions column.
      6. In the Label section of the Edit Namespace dialog box, set Variable Key to k8s.aliyun.com/image-accelerate-mode and Variable Value to p2p, and click OK.
    • Attach the P2P acceleration label to a namespace by using kubectl
      kubectl label namespaces <your-namespace> k8s.aliyun.com/image-accelerate-mode=p2p

Verify P2P acceleration

After P2P acceleration is enabled for a pod, the P2P component automatically adds P2P-related information to the YAML file of the pod. The information includes P2P-related annotations, the address of the P2P-accelerated image, and the Secret for pulling the P2P-accelerated image.
Notice
  • If no image from a Container Registry Enterprise Edition instance is specified in the original YAML file of the pod, the P2P component does not add the preceding information to the file. If no Secrete for pulling an image from a Container Registry Enterprise Edition instance in regular mode is configured, the P2P component also does not add the preceding information.
  • The Secret for pulling a P2P-acclerated image and the Secret for pulling the original image are different only in the domain name of the image repository. Other configurations of the two Secrets are the same. If the user information is invalid in the Secret for pulling the original image, the P2P-acclerated image also fails to be pulled.

Run the following command to view the YAML file of the pod:

kubectl get po <Pod name> -oyaml

Expected output:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    # inject p2p-annotations automatically
    k8s.aliyun.com/image-accelerate-mode: p2p
    k8s.aliyun.com/p2p-config: '...'
spec:
  containers:
   # inject image to p2p endpoint
   - image: test-registry-vpc.distributed.cn-hangzhou.cr.aliyuncs.com:65001/docker-builder/nginx:latest
  imagePullSecrets:
  - name: test-registry
  # inject image pull secret for p2p endpoint
  - name: acr-credential-test-registry-p2p

If P2P-related annotations, the address of the P2P-accelerated image, and the Secret for pulling the P2P-accelerated image exist in the YAML file, P2P acceleration is enabled.

Use P2P acceleration in the Docker environment

  1. Log on to an ECS instance. For more information, see Connect to a Linux instance by using an SSH key pair.
  2. Download the installation package of the P2P component.
    docker run --rm -v /var/lib/aliyun-acr/p2p:/var/lib/aliyun-acr/p2p registry.cn-hangzhou.aliyuncs.com/acr-toolkit/p2p-installer-manual:v1.0.6-b6b9f5f9-aliyun
  3. Configure the P2P component.
    /var/lib/aliyun-acr/p2p/scripts/01-init.sh --ak <aliyun-ak> --sk <aliyun-sk> --port 65001 --instance <acr-ee-instance-id>
    Replace the AccessKey ID, AccessKey secret, ID of the Container Registry Enterprise Edition instance, and the port used by the P2P component in the preceding command as required. By default, the P2P component uses port 65001 on nodes.
    Note The AccessKey ID and AccessKey secret are used only to obtain information about the Container Registry Enterprise Edition instance during configuration initialization.
    After the P2P component is configured, the /var/lib/aliyun-acr/p2p directory is generated.
  4. Start the P2P component.
    Note To install the P2P component on other ECS instances, copy the directory that was generated in Step 3 to these ECS instances. Then, run the command to start the P2P component on these ECS instances.
    /var/lib/aliyun-acr/p2p/scripts/02-run.sh
  5. Log on to an image repository by using the P2P-accelerated domain name.
    docker login <P2P-accelerated domain name of the Container Registry Enterprise Edition instance>
    The P2P-accelerated domain name is in the following format: <Name of the Container Registry Enterprise Edition instance>-registry-vpc.distributed.<Region where the Container Registry Enterprise Edition instance resides>.cr.aliyuncs.com:<Port used by the P2P component>.
  6. Pull an image by using the P2P-accelerated domain name.
    docker pull <P2P-accelerated domain name of the Container Registry Enterprise Edition instance>/test/busybox:latest
  7. Optional:Uninstall the P2P component.
    /var/lib/aliyun-acr/p2p/scripts/03-uninstall.sh