Pipeline jobs - Alibaba Cloud DevOps - Alibaba Cloud Documentation Center

A pipeline job is a collection of steps that run sequentially in a shared workspace. Alternatively, a job can be a single call to a component, which supports advanced operations such as retries and skips.

Examples

Configure a job with multiple steps:

stages:
  build_stage:
    name: Build Stage
    jobs:
      build_job: 
        name: Build Job
        runsOn: public/cn-beijing
        steps:                                # Configure the steps for the job
          build_step:                        
            step: JavaBuild                   
            name: Java Build                     
            with:                            
              ......
          upload_step:
            step: ArtifactUpload
            name: Artifact Upload
            with:
              ......

stages:
  build_stage:
    name: Build Stage
    jobs:
      build_job: 
        name: Build Job
        runsOn:
          group: public/ap-southeast-1
          container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
        steps:                                # Configure the steps for the job
          setup_java_step:
            name: "Set up Java"
            step: SetupJava
            with:
              jdkVersion: "1.8"
              mavenVersion: "3.5.2"
          command_step:
            name: "Command"
            step: Command
            with:
              run: |
                mvn -B clean package -Dmaven.test.skip=true -Dautoconfig.skip
          upload_artifact_step:
            name: "Artifact Upload"
            step: ArtifactUpload
            with:
              uploadType: flowPublic
              artifact: "Artifacts_${PIPELINE_ID}"
              filePath:
                - target/

Configure a job with a component:

stages:
  build_stage:
    name: Build Stage
    jobs:
      deploy_job:
        name: VM Deploy Job
        component: VMDeploy                # Configure the job by using a component
        with:                              
          artifact: $[stages.build_stage.build_job.upload_artifact_step.artifacts.default]
          ......

Syntax

stages.<stage_id>.jobs

The collection of jobs for a stage. A job can be a combination of multiple steps or a call to a component.

stages.<stage_id>.jobs.<job_id>

Required. A unique identifier for the job. The job_id must start with a letter and can only contain letters, numbers, and underscores (_). The maximum length is 64 characters.

stages.<stage_id>.jobs.<job_id>.name

Optional. The display name of the job. If omitted, the job_id is used. The maximum length is 64 characters.

stages.<stage_id>.jobs.<job_id>.runsOn

Optional. The execution environment for the job. You can use the public K8s cluster from Alibaba Cloud DevOps or your own self-hosted build cluster. Supported environments: Specified Container Environment and Default VM Environment.

Specified Container Environment: Runs builds in an isolated container on the build machine. Syntax:

jobs:
  my_job:
    name: My Job
    runsOn:
      group: public/ap-southeast-1 // The Specified Container Environment currently supports only public build clusters from Alibaba Cloud DevOps.
      container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest // A public image address. For official system images from Alibaba Cloud DevOps, see https://atomgit.com/flow-steps/system_images/blob/main/README_INTL.md.

Build cluster	YAML identifier	Description
Alibaba Cloud DevOps Singapore public build cluster	group: public/ap-southeast-1	A public K8s cluster provided by Alibaba Cloud DevOps in the Singapore region. This is the default if `runsOn` is not specified.
Self-hosted build cluster	group: private/<your_self-hosted_build_cluster_id>	A private cluster that your organization registers as a self-hosted build cluster.

Default VM Environment: Runs steps directly on the host machine or virtual machine of the build cluster. Syntax:

jobs:  
  my_job:    
    name: My Job      
      runsOn:        
        group: private/<your_self-hosted_build_cluster_id>   // Only supports self-hosted build clusters.
        labels: windows, amd64          // Specifies the OS and architecture for scheduling. If omitted, the job is scheduled to a random machine in the cluster.
        vm: true                        // Specifies the VM build environment.

Self-hosted build clusters support Linux, Windows, and macOS machines. The supported architectures and environments for each operating system are as follows:

Operating system	Architecture	Labels	Description
Linux	amd64	linux,amd64	Supports both the Default Environment and Default VM Environment.
Linux	arm64	linux,arm64	Supports only the Default VM Environment. You must specify `vm: true`.
Windows	amd64	windows,amd64	Supports only the Default VM Environment. You must specify `vm: true`.
Windows	arm64	windows,arm64	Supports only the Default VM Environment. You must specify `vm: true`.
macOS	amd64	darwin,amd64	Supports only the Default VM Environment. You must specify `vm: true`.
macOS	arm64	darwin,arm64	Supports only the Default VM Environment. You must specify `vm: true`.

stages.<stage_id>.jobs.<job_id>.runsOn.instanceType

Optional. The instance type for the job's execution environment. Alibaba Cloud DevOps automatically allocates a DEFAULT instance type based on the steps in the job. For more information about the default type, see https://www.alibabacloud.com/help/doc-detail/201868.html. You can specify a different instance type. Available options: SMALL_1C2G, MEDIUM_2C4G, LARGE_4C8G, and XLARGE_8C16G.

Example:

jobs:
  my_job:
    name: My Job
    runsOn:
      group: public/ap-southeast-1
      container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
      instanceType: LARGE_4C8G    # Specify the instance type

stages.<stage_id>.jobs.<job_id>.timeoutMinutes

Optional. The maximum number of minutes a job can run before it is automatically canceled. The default is 240 minutes. You can set any integer from 1 to 1440.

Example:

jobs:
  my_job:
    name: My Job
    runsOn:
      group: public/ap-southeast-1
      container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
    timeoutMinutes: 60		# The job will time out 60 minutes after it starts.

stages.<stage_id>.jobs.<job_id>.debugPolicy and stages.<stage_id>.jobs.<job_id>.debugRetentionMinutes

Optional. Retains the job's execution environment after the job completes, allowing you to connect to it for debugging.

This feature is only available for the Specified Container Environment.

These two parameters must be used together or not at all.

The available options for debugPolicy are:

onFailure: Retains the environment only if the job fails. The environment is not retained if the job succeeds or is blocked by a quality gate.
always: Retains the environment regardless of the job's final status.

debugRetentionMinutes is an integer from 1 to 240 that specifies the retention duration in minutes.

Example:

jobs:
  my_job:
    name: My Job
    runsOn:
      group: public/ap-southeast-1
      container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
    debugPolicy: always
    debugRetentionMinutes: 5

stages.<stage_id>.jobs.<job_id>.needs

Optional. The prerequisite jobs that must complete successfully before this job runs. By default, all jobs within a stage run in parallel. Use needs to create dependencies between jobs. Note the following:

The needs keyword supports dependencies between jobs in different stages.
Ensure that dependencies create a clear execution order. Avoid circular dependencies, such as A depending on B, B on C, and C on A.

Specify the <job_id> of the prerequisite job. Example:

jobs:
  test_job:
    name: Test Job
  build_job:
    name: Build Job
    needs: test_job

stages.<stage_id>.jobs.<job_id>.driven

Optional. The trigger mode for the job. By default, jobs trigger automatically (auto). Supported modes:

auto: The job runs automatically.
manual: The job must be triggered manually.

Example:

jobs:
  my_job:
    name: My Job
    runsOn:
      group: public/ap-southeast-1
      container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
    driven: manual		# Manually trigger the job execution

stages.<stage_id>.jobs.<job_id>.continueOnFail

Optional. If set to true, the pipeline continues to run subsequent jobs even if the current job fails. The default is false, which stops the pipeline run on failure.

jobs:
  my_job:
    name: My Job
    continueOnFail: true	# Continue running subsequent jobs on failure

stages.<stage_id>.jobs.<job_id>.condition

Optional. A conditional expression that determines whether the job runs. By default, a job runs only if all its needs dependencies have completed successfully. The job runs when condition evaluates to true.

jobs:
  my_job:
    name: My Job
    runsOn:
      group: public/ap-southeast-1
      container: build-steps-public-registry.ap-southeast-1.cr.aliyuncs.com/build-steps/alinux3:latest
    condition: |
      "${CI_COMMIT_REF_NAME}" == "master"		# Run this job only when the branch is master.

The condition supports relational and logical operators:

Operator	Description	Example	Example description
==	Equal to	condition: "${CI_COMMIT_REF_NAME}" == "master"	Runs when the branch is `master`.
!=	Not equal to	condition: "${CI_COMMIT_REF_NAME}" != "master"	Runs when the branch is not `master`.
&&	And	condition: "${CI_COMMIT_REF_NAME}" == "master" && succeed()	Runs when the branch is `master` and all prerequisite jobs are successful.
\|\|	Or	condition: "${CI_COMMIT_REF_NAME}" == "master" \|\| "${CI_COMMIT_REF_NAME}" == "develop"	Runs when the branch is `master` or `develop`.
!	Not	condition: succeed('job1') && !skipped('job1')	Runs when `job1` is successful and not skipped.
()	Logical group	condition: ("${CI_COMMIT_REF_NAME}" == "master" \|\| "${CI_COMMIT_REF_NAME}" == "develop") && succeed()	Runs when the branch is `master` or `develop`, and all prerequisite jobs are successful.

Built-in functions available in expressions:

Function	Description	Example
startsWith(searchString, searchValue)	Returns `true` if `searchString` starts with `searchValue`.	condition: startsWith('Hello world','He')
endsWith(searchString, searchValue)	Returns `true` if `searchString` ends with `searchValue`.	condition: endsWith('Hello world','ld')
contains(search, item)	Returns `true` if `search` is an array and contains `item`.	condition: contains('["aa", "bb", "cc"]', 'aa')
weekDay()	Returns the current day of the week (Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, or Sunday).	condition: weekDay()=="Thursday"
timeIn(startTime, endTime)	Returns `true` if the current time is between `startTime` and `endTime`.	condition: timeIn("20:00:00", "22:00:00")

Note: You can use existing variables as function parameters. For example, if you set a pipeline variable TEST_VAR=["aa", "bb", "cc"], you can reference this variable in a function by using ${}.

jobs:
  job_1:
    name: 1
    condition: contains('${TEST_VAR}', 'aa')

Use job status functions to check the execution status of prerequisite jobs. The function parameter is the <job_id> of the prerequisite job.

Function	Description	Example
always()	Always returns `true`.	condition: always()
succeed()	Returns `true` if all specified prerequisite jobs have a status of `Successful` or `Skipped`.	condition: succeed('job_id_1','job_id_2')
failed()	Returns `true` if any specified prerequisite job has a status of `Failed` or `Cancelled`.	condition: failed('job_id_1','job_id_2')
skipped()	Returns `true` if any specified prerequisite job has a status of `Skipped`.	condition: skipped('job_id_1','job_id_2')

Note: If you do not provide parameters to a job status function, it checks all prerequisite jobs. For example, succeed() returns true only if all prerequisite jobs have succeeded. The parameters for job status functions must be the <job_id> of a prerequisite job. Providing the <job_id> of a job that is not a dependency causes an error. Example:

jobs:
  job_1:
    name: Job 1
  job_2:
    name: Job 2
  job_3:
    name: Job 3
    needs: 
      - job_1
      - job_2
    condition: succeed(job_1) || succeed(job_2) # Run job_3 if job_1 succeeds or job_2 succeeds.

stages.<stage_id>.jobs.<job_id>.sourceOption

Optional. Controls which pipeline sources are downloaded for the job. By default, all configured sources are downloaded. When multiple sources exist, use this option to skip downloading or to download only specific sources by <source_id>.

Scenario	Description	Example
Download all pipeline sources	Do not specify `sourceOption`.	(not specified)
Do not download any pipeline sources	Specify `sourceOption` as an empty list.	sourceOption: []
Download specific pipeline sources	Specify the `<source_id>` in `sourceOption`.	sourceOption: [repo_1,repo_2]

stages.<stage_id>.jobs.<job_id>.steps

A job defined as a sequence of steps. All steps in a job share the same workspace and run sequentially.

For more information, see pipeline steps.

stages.<stage_id>.jobs.<job_id>.component

A job defined as a call to a single component. Component-based jobs support advanced features like retries and skips.

For more information, see pipeline components.

stages.<stage_id>.jobs.<job_id>.with

When a job calls a component, use with to pass the execution parameters for the component. For example:

jobs:
  deploy_job:
  name: VM Deploy Job
  component: VMDeploy             # Specify the component
  with:                           # Provide parameters for the component
    artifact: $[stages.build_stage.build_job.upload_step.artifacts.default]
    machineGroup: <YOUR-MACHINE-GROUP-ID>
    ......

For more information, see pipeline components.

stages.<stage_id>.jobs.<job_id>.plugins

Optional. Configure plugins to send job status notifications through DingTalk, email, Enterprise WeChat (WeCom), Feishu, or webhooks.
For more information, see pipeline plugins.