Function Compute:Build an image and push it to an image repository

Limits on the default execution environment

• The disk size cannot exceed 10 GB.

• The time spent on building a task cannot exceed 15 minutes.

• If you use GitHub as the code source, build a task in the Singapore region on the International site (alibabacloud.com). If you use GitLab or Codeup as the code source, build the task in the China (Hangzhou) region on the China site (aliyun.com).

• You cannot use fixed IP addresses, CIDR blocks, and an IP address whitelist to access specific websites.

If the default execution environment cannot meet your business requirements, you can use a custom execution environment to build an image. For more information, see Use a custom execution environment to build an image.

Prerequisites

• If you are using a private Container Registry image repository, you must grant the AliyunFCServerlessDevsRole role the write permissions on Container Registry, such as the AliyunContainerRegistryFullAccess permission or permissions in custom policies. For more information, see Attach a custom policy to a RAM user.

• If you use an image repository of Container Registry Enterprise Edition, you must use the custom execution environment to execute pipelines and build tasks in a virtual private cloud (VPC). For more information, see Use a custom execution environment to build an image.

• If you use an image building task, you must delete the component: fc build --use-docker hook in the s.yaml file to prevent image building during task deployment.

Use a default execution environment to build an image

You can use the default execution environment to build an image. Log on to the Function Compute console. Find the application that you want to manage and click the Pipeline Management tab. In the Pipeline Details section, use a template to build an image. The following figure shows the procedure. On the Task Template (1) tab, choose Create (2) > Image Build (3). Then update the .yaml file (4) on the right side of the page.

The following code provides an example .yaml file of a pipeline for image building.

kind: Pipeline
name: "image-build-pipeline"
description: 'demo pipeline'
labels:
  myLabel: image-build
spec:
  templateSpec:
    tasks:
      - name: build-push
        context:
          data:
            # The display name of the task, which is displayed in the console. 
            displayName: image build
            # Specifies whether to enable task execution. By default, task execution is disabled. 
            enable: true
            # The build information. 
            build: 
              # Specify the path to the location where the Dockerfile file is stored. The relative path in the code repository is used. By default, if you do not specify this parameter, ./Dockerfile is used. 
              dockerfile: code/Dockerfile
              # Specify the name of the image that you want to build. The image must be a Container Registry image. 
              image: registry-vpc.cn-hangzhou.aliyuncs.com/pipeline-demo/demo-repo:latest
              # Specify the build context directory. If you do not specify a directory, the directory in which the Dockerfile file resides is used. 
              context: ./context
        # The build-image-task template, a built-in template, is used as the task template. 
        taskTemplate: build-image-task
        runAfters: []

Use a custom execution environment to build an image

If the default execution environment cannot meet your business requirements, you can use a custom execution environment to run pipelines within your account. You can specify the region, network, resource specifications, and timeout interval in a custom execution environment. For information of custom execution environments for pipelines, see Manage pipelines.

You can configure an execution environment of a pipeline by graphically editing the pipeline or editing a .yaml file of the pipeline.

• For more information about how to graphically edit a pipeline, see Manage pipelines.

• For more information about how to use YAML to edit the pipeline execution environment, see Built-in custom task templates.

Use NAS for caching to build an image

If the size of a base image in the Dockerfile file is too large, you can use Apsara File Storage NAS (NAS) to cache the base image and resolve issues. For example, you use models or CUDA execution environments, which increases the size of the base image. When the base image is pulled during image build, the build speed slows down or the disk space become insufficient.

Procedure

1. Use a custom execution environment for a pipeline and bind a VPC and a NAS file system to the pipeline.

2. Use image preheating to cache the base image to the NAS file system.

3. Use the cache in the NAS file system to build an image.

Sample .yaml file

kind: Pipeline
name: "image-build-use-cache-pipeline"
description: 'demo pipeline'
labels:
  myLabel: image-build-use-cache
spec:
  templateSpec:
    context:
      data:
        # Specify a custom execution environment. 
        runnerConfig:
          regionId: cn-hangzhou
          logConfig:
            project: demo-project
            logstore: demo-logstore
          # The network configuration must be the same as that of the VPC in which the NAS file system resides. 
          vpcConfig:
            securityGroupId: sg-xxxx
            vSwitchIds:
              - vsw-xxx
            vpcId: vpc-xxxx
          # The configurations of a NAS file system. 
          nasConfig:
            groupId: 0
            userId: 0
            mountPoints:
              - enableTLS: false
                # The local directory in which cache is stored. 
                mountDir: /mnt/image-cache
                # The remote directory in which the cache is stored. 
                serverAddr: xxxx.cn-hangzhou.nas.aliyuncs.com:/image-cache
          # We recommend that you use an instance that is configured with 8 vCPUs, 32 GB of memory, and 10 GB of disk size. 
          cpu: 8
          memory: 32768
          diskSize: 10240
          # The build timeout interval. 
          timeout: 1800
    tasks:
      - name: build-push
        context:
          data:
            # The display name of the task, which is displayed in the console. 
            displayName: image build
            # Specifies whether to enable task execution. By default, task execution is disabled.
            enable: true
            # Prefetch the base image and cache the image. 
            warm: 
              # You can prefetch multiple base images. 
              images:
              - registry-vpc.cn-hangzhou.aliyuncs.com/pipeline-demo/base-image-1:debug
              - registry-vpc.cn-hangzhou.aliyuncs.com/pipeline-demo/base-image-2:debug
              # The path of the base image that you want to cache. 
              cacheDir: "/mnt/image-cache" 
            # The build information. 
            build: 
              # Specify the path to the location where the Dockerfile file is stored. The relative path in the code repository is used. By default, if you do not specify this parameter, ./Dockerfile is used. 
              dockerfile: code/Dockerfile
              # Specify the name of the image that you want to build. The image must be a Container Registry image. 
              image: registry-vpc.cn-hangzhou.aliyuncs.com/pipeline***/demo***:latest
              # Use the cache path of the base image. 
              cacheDir: "/mnt/image-cache" 
        # The build-image-task template, a built-in template, is used as the task template. 
        taskTemplate: build-image-task
        runAfters: []

After the cache is used and a cache hit occurs, the "Found manifest at ..." message is printed.

You can also separately execute image build task and image prefetching task. Example:

- name: only-warm
  context:
    data:
      enable: true
      # Prefetch the base image and cache the image. 
      warm: 
        # You can prefetch multiple base images. 
        images:
        - registry-vpc.cn-hangzhou.aliyuncs.com/pipeline-demo/base-image-1:debug
        - registry-vpc.cn-hangzhou.aliyuncs.com/pipeline-demo/base-image-2:debug
        # The path of the base image that you want to cache. 
        cacheDir: "/mnt/image-cache"
        # Specify a Container Registry Enterprise Edition instance. 
        instanceID: cri-xxxx
  # The build-image-task template, a built-in template, is used as the task template. 
  taskTemplate: build-image-task
  runAfters: []
- name: only-build
  context:
    data:
      enable: true
      # The build information. 
      build: 
        # Specify the path to the location where the Dockerfile file is stored. The relative path in the code repository is used. By default, if you do not specify this parameter, ./Dockerfile is used. 
        dockerfile: code/Dockerfile
        # Specify the name of the image that you want to build. The image must be a Container Registry image. 
        image: registry-vpc.cn-hangzhou.aliyuncs.com/pipeline-demo/demo-repo:latest
        # Use the cache path of the base image. 
        cacheDir: "/mnt/image-cache" 
  # The build-image-task template, a built-in template, is used as the task template. 
  taskTemplate: build-image-task
  runAfters: [only-warm]

Build a Container Registry Enterprise Edition image

If you use a Container Registry Enterprise Edition instance, you must configure Internet-based access.

• By default, Internet-based access is not supported. If you want to enable Internet-based access to the Container Registry Enterprise Edition instance, you must configure a public IP address whitelist. For more information, see Configure access over the Internet.

• You must configure an access control list (ACL) before you establish a connection between an Elastic Compute Service (ECS) instance and a Container Registry Enterprise Edition instance that reside in the same virtual private network (VPC). For more information, see Configure a VPC ACL.

The default execution environment of a pipeline does not support fixed public IP addresses and internal CIDR blocks. Therefore, the default execution environment cannot access Container Registry Enterprise Edition instances. You must use a custom execution environment of the pipeline to access your Container Registry Enterprise Edition instances.

Procedure

1. Use a custom execution environment of the pipeline and bind a VPC. The VPC must be connected to your Container Registry Enterprise Edition instance. We recommend that you use the VPC and the vSwitch in which the Container Registry Enterprise Edition instance resides. You can also specify another VPC and complete the ACL configurations for the VPC. For more information, see Configure a VPC ACL.

2. Specify the Container Registry Enterprise Edition instance to build an image.

3. (Recommended) Enable image acceleration for image repositories and append _accelerated to image tags. The image build tasks in pipelines automatically push original images and wait until accelerated images are converted. For more information, see Load resources of a container image on demand.

Sample .yaml file

kind: Pipeline
name: "image-build-use-cache-pipeline"
description: 'demo pipeline'
labels:
  myLabel: image-build-use-cache
spec:
  templateSpec:
    context:
      data:
        # Specify a custom execution environment. 
        runnerConfig:
          regionId: cn-hangzhou
          logConfig:
            project: demo-project
            logstore: demo-logstore
          # Configure the network. The network must be connected to the Container Registry Enterprise Edition instance. 
          vpcConfig:
            securityGroupId: sg-xxxx
            vSwitchIds:
              - vsw-xxx
            vpcId: vpc-xxxx
          // Specify the instance specifications. 
          cpu: 8
          memory: 32768
          diskSize: 10240
          # The build timeout interval. 
          timeout: 1800
    tasks:
      - name: build-push
        context:
          data:
            # The display name of the task, which is displayed in the console. 
            displayName: image build
            # Specifies whether to enable task execution. By default, task execution is disabled. 
            enable: true
            # The build information. 
            build:
              # Specify the path to the location where the Dockerfile file is stored. The relative path in the code repository is used. By default, if you do not specify this parameter, ./Dockerfile is used. 
              dockerfile: code/Dockerfile
              # Specify the image name. We recommend that you enable image acceleration and append _accelerated to the name. 
              image: demo-registry-vpc.cn-hangzhou.aliyuncs.com/pipeline***/demo***:debug_accelerated
              # Specify a Container Registry Enterprise Edition instance. 
              instanceID: cri-xxxx
            
        # The build-image-task template, a built-in template, is used as the task template. 
        taskTemplate: build-image-task
        runAfters: []

Complete CI/CD process with image build and deployment capabilities

We recommend that you use image build task templates and custom task templates to build and deploy automated CI/CD processes.

• We recommend that you add a tag to the image in each CI process, and the tag must correspond to the current commit.

• During each CD process, the new tag is used for deployment.

The following example shows the preceding procedure.

• For build tasks, the preset variable <% .git.shortCommitId %> in the .yaml file of a pipeline is appended as the image tag to the name of an image. The variable indicates the first seven characters of the commit ID in HEAD Commit of a Git operation. Example: image: registry-vpc.cn-hangzhou.aliyuncs.com/demo/demo-image:<% .git.shortCommitId %> For more information about built-in YAML variables, see Use a .yaml file to describe a pipeline.

• For a deployment task, you need to inject an environment variable GIT_COMMIT and assign its value to <% .git.shortCommitId %>.

• Then, you can modify environment variables in the s.yaml to replace the tag of the custom image with ${env(GIT_COMMIT)}, which indicates that the environment variable is referenced. Variable assignment allows the mag tag in the s.yaml file to be dynamically modified.

The following code shows a sample .yaml file:

• Sample .yaml file of a pipeline

  # Commit a pipeline for execution. 
  kind: Pipeline
  # The name of the pipeline that you want to execute. The name must be globally unique. If a pipeline template exists, the pipeline execution cannot be successfully committed. 
  # We recommend that you specify a name that includes the commit ID and timestamp. 
  name: "p-<% .git.shortCommitId %>-<% .currentTimestampMs %>"
  # Arbitrary description of the pipeline. 
  description: 'demo pipeline'
  # You can add labels as filter conditions to query the pipeline. 
  # Serverless Application Center also adds other labels when you commit pipeline execution. 
  labels:
    myLabel: my-cicd-example
  spec:
    context:
      data:
        appName: <% .appName %>
    templateSpec:
      context:
        data:
          envName: <% .envName %>
          deployFile: s.yaml
      tasks:
        # Build an image. 
        - name: build-image
          context:
            data:
              displayName: image build
              enable: true
              build: # Build the image. 
                dockerfile: code/Dockerfile
                # Use <% .git.shortCommitId %> to associate the tag of the image with the current commit. 
                image: registry-vpc.cn-hangzhou.aliyuncs.com/demo/demo-image:<% .git.shortCommitId %>
          taskTemplate: build-image-task
          runAfters: []
      	# Deploy. 
        - name: build-and-deploy
          context:
            data:
              displayName: deployment
              enable: true
              # Dynamically inject the environment variables of this deployment. 
              envVars:
                GIT_COMMIT: <% .git.shortCommitId %>
              steps:
                - plugin: "@serverless-cd/checkout"
                - plugin: "@serverless-cd/s-setup"
                - plugin: "@serverless-cd/s-deploy"
          taskTemplate: serverless-runner-task
          runAfters:
            - name: build-push

• s.yaml

  edition: 3.0.0
  name: demo
  access: 'default'  #  The alias of the key. 
  
  resources:
    function: 
      component: fc3
      props:
        region: 'cn-hangzhou'
        functionName: demo
        runtime: custom-container
        customContainerConfig:
          # Reference the environment variables that are injected into the deployment task. 
          image: registry-vpc.cn-hangzhou.aliyuncs.com/demo***/demo-ima***:${env(GIT_COMMIT)}
          port: 9000
        memorySize: 4096
        timeout: 3600