Accelerate CI Pipelines with Argo Workflows and BuildKit Cache - Container Service for Kubernetes

Argo Workflows lets you define a complete continuous integration (CI) pipeline as a YAML workflow, running jobs in parallel on your cluster and scaling compute resources dynamically. This guide walks you through the predefined ci-go-v1 ClusterWorkflowTemplate, which clones a Git repository, runs Go tests, and builds and pushes a container image — all within your ACK cluster.

How it works

The ci-go-v1 ClusterWorkflowTemplate runs three steps in sequence:

Git clone and checkout — clones the repository, checks out the target branch, and records the short commit ID for use as an image tag suffix.
Run Go test — runs all Go test cases in the repository. The Go mod cache is stored on a NAS volume so that subsequent runs skip re-downloading dependencies.
Build and push image — uses BuildKit to build and push the container image. BuildKit Cache (registry type) is imported and exported on each run to skip unchanged build layers. During image pushing, the latest version of the image is pushed to overwrite the current version.

Prerequisites

Before you begin, make sure you have:

A Container Service for Kubernetes (ACK) cluster with Argo Workflows installed
A Container Registry Enterprise Edition (ACR EE) instance with an image repository
An Apsara File Storage NAS (NAS) file system accessible from the cluster

Important

The Secret that stores registry credentials and the NAS volume must belong to the same namespace where you submit the pipeline.

This guide uses a public Git repository as an example. For private repositories, you need to first clone the private repository in the CI pipeline. See Clone a private Git repository in a CI pipeline.

The `ci-go-v1` ClusterWorkflowTemplate

You can use the predefined template directly or customize it for your project.

Template YAML

All three steps share a common NAS volume (pvc-nas) for code checkout and the Go mod cache. The docker-config Secret mounts registry credentials into the BuildKit step.

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: ci-go-v1
spec:
  entrypoint: main
  volumes:
  - name: run-test
    emptyDir: {}
  - name: workdir
    persistentVolumeClaim:
      claimName: pvc-nas
  - name: docker-config
    secret:
      secretName: docker-config
  arguments:
    parameters:
    - name: repo_url
      value: ""
    - name: repo_name
      value: ""
    - name: target_branch
      value: "main"
    - name: container_image
      value: ""
    - name: container_tag
      value: "v1.0.0"
    - name: dockerfile
      value: "./Dockerfile"
    - name: enable_suffix_commitid
      value: "true"
    - name: enable_test
      value: "true"
  templates:
    - name: main
      dag:
        tasks:
          - name: git-checkout-pr
            inline:
              container:
                image: mirrors-ssl.aliyuncs.com/alpine:latest
                command:
                  - sh
                  - -c
                  - |
                    set -eu

                    apk --update add git

                    cd /workdir
                    echo "Start to Clone "{{workflow.parameters.repo_url}}
                    git -C "{{workflow.parameters.repo_name}}" pull || git clone {{workflow.parameters.repo_url}}
                    cd {{workflow.parameters.repo_name}}

                    echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
                    git checkout --track origin/{{workflow.parameters.target_branch}} || git checkout {{workflow.parameters.target_branch}}
                    git pull

                    echo "Get commit id"
                    git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
                    commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
                    echo "Commit id is got: "$commitId

                    echo "Git Clone and Checkout Complete."
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
                resources:
                  requests:
                    memory: 1Gi
                    cpu: 1
                activeDeadlineSeconds: 1200
          - name: run-test
            when: "{{workflow.parameters.enable_test}} == true"
            inline:
              container:
                image: mirrors-ssl.aliyuncs.com/golang:alpine3.21
                command:
                  - sh
                  - -c
                  - |
                    set -eu

                    if [ !  -d "/workdir/pkg/mod" ]; then
                      mkdir -p /workdir/pkg/mod
                      echo "GOMODCACHE Directory /pkg/mod is created"
                    fi

                    export GOMODCACHE=/workdir/pkg/mod

                    cp -R /workdir/{{workflow.parameters.repo_name}} /test/{{workflow.parameters.repo_name}}
                    echo "Start Go Test..."

                    cd /test/{{workflow.parameters.repo_name}}
                    go test -v ./...

                    echo "Go Test Complete."
                volumeMounts:
                - name: "workdir"
                  mountPath: /workdir
                - name: run-test
                  mountPath: /test
                resources:
                  requests:
                    memory: 4Gi
                    cpu: 2
              activeDeadlineSeconds: 1200
            depends: git-checkout-pr
          - name: build-push-image
            inline:
              container:
                image: mirrors-ssl.aliyuncs.com/moby/buildkit:v0.13.0-rootless
                command:
                  - sh
                  - -c
                  - |
                    set -eu

                    tag={{workflow.parameters.container_tag}}
                    if [ {{workflow.parameters.enable_suffix_commitid}} == "true" ]
                    then
                      commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
                      tag={{workflow.parameters.container_tag}}-$commitId
                    fi

                    echo "Image Tag is: "$tag
                    echo "Start to Build And Push Container Image"

                    cd /workdir/{{workflow.parameters.repo_name}}

                    buildctl-daemonless.sh build \
                    --frontend \
                    dockerfile.v0 \
                    --local \
                    context=. \
                    --local \
                    dockerfile=. \
                    --opt filename={{workflow.parameters.dockerfile}} \
                    build-arg:GOPROXY=http://goproxy.cn,direct \
                    --output \
                    type=image,\"name={{workflow.parameters.container_image}}:${tag},{{workflow.parameters.container_image}}:latest\",push=true,registry.insecure=true \
                    --export-cache mode=max,type=registry,ref={{workflow.parameters.container_image}}:buildcache \
                    --import-cache type=registry,ref={{workflow.parameters.container_image}}:buildcache

                    echo "Build And Push Container Image {{workflow.parameters.container_image}}:${tag} and {{workflow.parameters.container_image}}:latest Complete."
                env:
                  - name: BUILDKITD_FLAGS
                    value: --oci-worker-no-process-sandbox
                  - name: DOCKER_CONFIG
                    value: /.docker
                volumeMounts:
                  - name: workdir
                    mountPath: /workdir
                  - name: docker-config
                    mountPath: /.docker
                securityContext:
                  seccompProfile:
                    type: Unconfined
                  runAsUser: 1000
                  runAsGroup: 1000
                resources:
                  requests:
                    memory: 4Gi
                    cpu: 2
              activeDeadlineSeconds: 1200
            depends: run-test

Deploy the template to your cluster:

kubectl apply -f cluster-workflow-template.yaml

Template parameters

Parameter	Description	Default	Example
`repo_url`	URL of the Git repository	—	`https://github.com/ivan-cai/echo-server.git`
`repo_name`	Name of the repository	—	`echo-server`
`target_branch`	Branch to check out	`main`	`main`
`container_image`	Image to build. Format: `<ACR EE Domain>/<ACR EE Namespace>/<Repository Name>`	—	`test-registry.cn-hongkong.cr.aliyuncs.com/acs/echo-server`
`container_tag`	Base image tag	`v1.0.0`	`v1.0.0`
`dockerfile`	Relative path to the Dockerfile from the repository root	`./Dockerfile`	`./Dockerfile`
`enable_suffix_commitid`	Append the short commit ID to the image tag. When `true`, the tag format is `{container_tag}-{commit_id}`. The `latest` tag is always pushed.	`true`	`true`
`enable_test`	Run the Go test step	`true`	`true`

Create a CI pipeline

Step 1: Create registry credentials

The credentials are used to pull images from BuildKit. You create a Kubernetes Secret in the pipeline namespace to store these credentials.

Configure access credentials for your Container Registry Enterprise Edition instance. See Configure access credentials for a Container Registry Enterprise Edition instance.

Create a Secret in the cluster to store the credentials:

Placeholder	Description
`$repositoryDomain`	Domain name of the Container Registry image repository
`$username`	Username for the Container Registry image repository
`$password`	Password for the Container Registry image repository

kubectl create secret -n argo generic docker-config --from-literal="config.json={\"auths\": {\"$repositoryDomain\": {\"auth\": \"$(echo -n $username:$password|base64)\"}}}"

Replace the following placeholders:

Step 2: Mount a NAS volume

The NAS volume serves two purposes: it shares the cloned repository between the git-checkout-pr step and subsequent steps, and it persists the Go mod cache across pipeline runs to speed up go test and go build.

Mount a NAS volume to the cluster. See Use volumes.

Step 3: Submit the pipeline

Submit the pipeline using the Argo console or the Argo CLI.

Use the console

Log on to the Argo console. In the left-side navigation pane, click Cluster Workflow Templates and click ci-go-v1.
On the template details page, click + SUBMIT in the upper-right corner. Configure the parameters based on the template parameters table, then click + SUBMIT.
View the pipeline status on the Workflows details page.

Use the Argo CLI

Create a file named workflow.yaml with the following content. Set the parameter values based on the template parameters table.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: ci-go-v1-
  labels:
    workflows.argoproj.io/workflow-template: ackone-ci
  namespace: argo
spec:
  arguments:
    parameters:
    - name: repo_url
      value: https://github.com/ivan-cai/echo-server.git
    - name: repo_name
      value: echo-server
    - name: target_branch
      value: main
    - name: container_image
      value: "test-registry.cn-hongkong.cr.aliyuncs.com/acs/echo-server"
    - name: container_tag
      value: "v1.0.0"
    - name: dockerfile
      value: ./Dockerfile
    - name: enable_suffix_commitid
      value: "true"
    - name: enable_test
      value: "true"
  workflowTemplateRef:
    name: ci-go-v1
    clusterScope: true

Submit the pipeline:
```
argo submit workflow.yaml
```

Contact us

If you have suggestions or questions about this product, join the DingTalk group 35688562 to contact us.