Alibaba Cloud DevOps:Artifacts

Generate Artifacts

Flow pipeline steps such as image building and artifact uploads currently generate the following outputs:

• Docker image：After the image building job is completed, the Docker image is uploaded to ACR or a private image repository. The artifact information structure is as follows:

{
  "artifact": "yunxiao-demo",		                                        # Image name
  "type": "dockerImage",    				                        # Artifact type
  "dockerUrl": "registry.cn-beijing.aliyuncs.com/yunxiao/yunxiao-demo:1.0",	# Image address
  "dockerTag": "1.0"								# Image tag
}

• Flow Public Storage Artifacts：After the build job is completed, the build artifact is uploaded to the Flow Public Storage. The artifact information structure is as follows:

{
  "artifact": "yunxiao-demo",		# Atifact name
  "type": "flowPublic",			# Artifact type
  "downloadUrl": "https://sdfdff",	# Atifact download address 
  "md5": "dffff"			# Artifact MD5
}

Reference Artifacts

Flow pipeline offers several methods to reference artifacts:

• Cross-Job or cross-Stage artifact referencing within the same pipeline: Typically suited for scenarios within the same pipeline where preceding tasks build and subsequent tasks deploy, or for integrated CI/CD pipeline scenarios.

• Cross-pipeline artifact referencing: Commonly applied when the source pipeline generates artifacts that target pipelines then deploy (e.g., a production pipeline directly referencing an image validated in pre-release), especially in scenarios where CI and CD are separated.

• Artifact referencing from pipeline artifact sources: Frequently employed in artifact promotion scenarios (e.g., after pre-release validation, the artifact is uploaded to an artifact repository, and the production environment selects artifacts directly from the repository based on version numbers).

Example 1: Cross-Job or Cross-Stage Artifact Reference within the Same Pipeline

Below is an example of artifact production and utilization across Jobs within the same pipeline, with the artifact type being a Docker image:

sources:
  my_repo:
    ……
stages:
  my_stage:
    name: Build And Deploy
    jobs:
      build_job:
        name: Image Build Job
        runsOn:
          group: public/ap-southeast-1
          container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
        steps:
          private_registry_docker_setup:
            name: Login Docker Registry
            step: DockerLogin
            with:
              dockerRepository: registry.cn-hangzhou.aliyuncs.com
              serviceConnection: <your-service-connection-id>
          docker_build_push_step:              
            name: Image Build and Push
            step: DockerBuildPush              # DockerBuildPush step will generate a docker image artifact.
            with:
              artifact: my_image
              image: registry.cn-hangzhou.aliyuncs.com/ns/demo:1.0
              dockerfilePath: Dockerfile
              cacheType: remote
      deploy_job:
        name: Kubernetes Deploy Job
        steps:
          kubectl_apply:
            step: KubectlApply
            name: Kubectl Deploy
            with:
              kubernetesCluster: <your-kubernetes-id>
              kubectlVersion: "1.16.4"
              namespace: default
              yamlPath: app-configs/manifest-app
              variables:
                - key: image
                  value: $[jobs.build_job.docker_build_push_step.artifacts.my_image]			   # Reference the artifact generated by the docker_build_step step, which is my-image.

Below is an example of artifact production and utilization across stages within the same pipeline, where the artifact type is Flow Public Storage Artifacts:

sources:
  my_repo:
    ……
stages:
  build_stage:
    name: Build Stage
    jobs:
      build_job:
        name: Build Job
        runsOn:
          group: public/ap-southeast-1
          container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
        steps:
          setup_java_step:
            name: Setup Java SDK
            step: SetupJava
            with:
              jdkVersion: "1.8"
              mavenVersion: "3.5.2"
          command_step:
            name: Execute Command
            step: Command
            with:
              run: |
                mvn -B clean package -Dmaven.test.skip=true -Dautoconfig.skip
          upload_artifact_step:
            name: Artifact Upload
            step: ArtifactUpload                 # ArtifactUpload step will generate a artifact.
            with:
              uploadType: flowPublic
              artifact: "Artifacts_${PIPELINE_ID}"
              filePath:
                - target/
  deploy_stage:
    name: Deploy Stage
    jobs:
      deploy_job:
        name: VM Deploy Job
        component: VMDeploy
        with:
          artifact: $[stages.build_stage.build_job.upload_artifact_step.artifacts.default]	 # Reference the artifact generated by the upload_step step, which is default.
          machineGroup: <your-machine-group-id>
          artifactDownloadPath: /home/admin/app/package.tgz
          executeUser: root
          run: |
            mkdir -p /home/admin/application/
            tar zxvf /home/admin/app/package.tgz -C /home/admin/application/
            sh /home/admin/application/deploy.sh restart

Example 2: Cross-Pipeline Artifact Reference

Below is an example of pipeline A generating artifacts and referencing artifacts in pipeline B. Cross-pipeline artifact referencing is commonly used in scenarios where, after a pre-release stage image has been validated, the production pipeline directly deploys using the pre-release image.

• Source Pipeline A

sources:
  my_repo:
    type: gitSample
    name: Java Sample Code Source
    endpoint: https://atomgit.com/flow-example/spring-boot.git
    branch: master
stages:
  build_stage:
    name: Build Stage
    jobs:
      build_job:
        name: Java Image Build Job
        steps:
          private_registry_docker_setup:
            name: Login Docker Registry
            step: DockerLogin
            with:
              dockerRepository: registry.cn-hangzhou.aliyuncs.com
              serviceConnection: <your-service-connection-id>
          docker_build_push_step:
            name: Image Build and Push
            step: DockerBuildPush
            with:
              artifact: my_image         # Source Pipeline A build job will generate a docker image.
              image: registry.cn-hangzhou.aliyuncs.com/ns/demo:1.0
              dockerfilePath: Dockerfile
              cacheType: remote

• Target Pipeline B

sources:
  pipeline_a:
    type: flowPipeline
    name: Source Pipeline A
    flowPipeline: n86bmctmphs84icj       # Source Pipeline A ID 
    build: lastSuccessfulBuild
    triggerEvents: buildSuccess
stages:
  kubectl_apply_stage:
    name: Deploy Stage
    jobs:
      deploy_job:
        name: "Kubectl Apply"
        runsOn:
          group: public/ap-southeast-1
          container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
        steps:
          kubectl_apply_step:
            name: "Kubectl Apply"
            step: "KubectlApply"
            with:
              kubernetesCluster: <your-kubernetes-id>
              kubectlVersion: "1.27.9"
              namespace: default
              yamlPath: app-configs/manifest-app
              variables:
                - key: image
                  value: $[sources.pipeline_a.stages.build_stage.build_job.docker_build_push_step.artifacts.my_image]   # Reference the image generated by Source Pipeline A, which is my_image.

Detailed Explanation