Supported formats of scheduling parameters

In DataWorks, a task is scheduled to run based on the scheduling parameters that you configure for the task. The values of the scheduling parameters are automatically replaced based on the value assignment rule for the scheduling parameters each time the task is run. This way, the values of the scheduling parameters are dynamically replaced at the scheduling time of the task. This topic describes the supported formats of scheduling parameters and how to use scheduling parameters.

Background information

In Data Studio, after you define variables in the ${Variable} format in the code of a node and configure the scheduling parameters that are assigned to the variables as values in the Scheduling Parameters section of the Properties tab, the values of the scheduling parameters are dynamically replaced in the node code when the node is scheduled to run.

Note

You can assign a constant to a scheduling parameter. If you assign a constant to a scheduling parameter for a task, the same value is passed to the scheduling parameter each time the task is run.
The format of a scheduling parameter varies based on the type of a node. For example, you can configure scheduling parameters for a Shell node only in the $N format. N specifies an integer that starts from 1. For more information, see Appendix: References.

Terms

In most cases, scheduling parameters are used in the scenarios in which you want to specify a dynamic point in time. In such scenarios, the value of a scheduling parameter is determined based on the data timestamp or scheduling time of a task. You must understand the concepts of the data timestamp and scheduling time of a task before you configure scheduling parameters. This helps you configure the value formats for the scheduling parameters. The following table describes the concepts.

Term

Description

Data timestamp

The previous day of the scheduling time (the time when you want to schedule a task). In offline computing scenarios, a data timestamp represents the date on which a business transaction is conducted. The value of a data timestamp is accurate to the day.

For example, if you collect statistical data on the turnover of the previous day on the current day, the previous day is the date on which the business transaction is conducted and represents the data timestamp.

Scheduling time

The time when you want to schedule the task to process business data. The scheduling time is accurate to the second. The scheduling time can be different from the actual time at which the task starts to be run. The actual time at which a task is run is affected by multiple factors.

Format display of scheduling parameters

In most cases, scheduling parameters are used to obtain different time data. You can choose to use the data timestamp or scheduling date of a task to obtain time data based on your business requirements. DataWorks provides data timestamp- and scheduling date-related variables that have fixed formats. You can use the methods that are described in the following table to specify custom time formats.

Method to obtain time data

Parameter format

Sample parameter

References

Obtain time data based on the data timestamp of a task

A custom time parameter that uses the data timestamp of a task to obtain time data consists of ${}, yyyy, yy, mm, and dd. You can use a ${} parameter to specify a time that is N years, months, or days earlier or later than the current data timestamp.

Note

The data timestamp of a task is calculated by subtracting one day from the date of the scheduling time. This indicates that the value of the ${yyyymmdd} parameter is the same as that of the $[yyyymmdd-1] parameter.

You can use a custom time parameter in the ${…} format, such as ${yyyymmdd} or $[yyyymmdd-1], to obtain time data. Examples:

${yyyymmdd±N}
${yyyymmdd±7*N}
${yy±N}
${mm}
${yyyy-mm-dd±N}

For more examples of custom time parameters in the ${...} format, see the Appendix 1: Parameters section in this topic.

Obtain time data based on the scheduling time of a task

A custom time parameter that uses the scheduling time of a task to obtain time data consists of $[...], yyyy, yy, mm, dd, hh24, mi, and ss. You can use a $[...] parameter to specify a time that is N years, months, days, hours, minutes, or seconds earlier or later than the current scheduling time.

You can use a custom time parameter in the $[...] format, such as $[yyyymmddhh24miss], to obtain time data. For example, you can use the $[yyyymmdd-1-1/24] parameter to specify a time that is 1 hour earlier than the current time on the previous day.

For more examples of custom time parameters in the $[...] format, see the Appendix 1: Parameters section in this topic.
If you want to specify a time that is accurate to the hour or minute, cross-day issues may occur. For information about solutions to the issues, see I run an instance of a node at 00:00 on the current day to analyze the data in the partition that corresponds to 23:00 on the previous day. However, the data in the partition that corresponds to 23:00 on the current day is analyzed. What do I do?

If you cannot use the preceding custom time parameters to obtain the desired time data, we recommend that you use the preceding custom time parameters to obtain data and then process the data by performing the operations that are described in the following table to obtain the desired time data.

Sample scenario	Description	References
Call engine functions to convert time data into timestamps.	Call engine functions in the code to obtain the desired time data.	Process the return values of scheduling parameters
${mm}01: Obtain the first day of each month.	Concatenate strings to obtain the desired time data.	None

Precautions

${…} parameters
- ${…} parameters are accurate to the day. You cannot use a ${…} parameter to specify a time that is accurate to the hour, minute, or second. For example, you cannot specify a ${…} parameter in the ${yyyy-mm-dd-1/24} format.
- If you want to specify a year- or month-specific time, we recommend that you use a ${…} parameter. For example, you can specify a parameter in the ${yyyy-N} or ${mm-N} format.
$[…] parameters
- If you want to specify a time that is N hours or minutes ago, use a $[…] parameter.
  - $[…] parameters can be used to specify a time that is accurate to the hour, minute, or second. You cannot use the $[yyyy-N] or $[mm-N] parameter to obtain a time that is N years or months ago. To specify a time that is in a specific year or month, use ${...} parameters.
  - If you want to specify a time that is accurate to the second, we recommend that you use a $[...] parameter. Example: $[yyyy-mm-dd-1-1/24].
- The value of a $[...] parameter does not change with the actual time at which the instance generated for a task is run.
  The value of a $[...] parameter is determined by the scheduling time of the instance generated for a task. The value is determined when the instance is generated. Therefore, the value of a $[...] parameter does not change with the actual time at which the instance generated for a task is run.
Note
- When you use a $[...] parameter to specify a time that is accurate to the hour, both the 24-hour clock format and the 12-hour clock format are supported.
- hh24 indicates a 24-hour clock. hh indicates a 12-hour clock.
${bizdate} variable
The code of some nodes contains the ${bizdate} variable. This variable has no special meaning. You can specify custom variable names in the code of a node based on your business requirements.
Solutions to the cross-day issues that may occur when you specify a time that is accurate to the hour or minute
If you want to specify a time that is accurate to the hour or minute, cross-day issues may occur. For information about solutions to the issues, see I run an instance of a node at 00:00 on the current day to analyze the data in the partition that corresponds to 23:00 on the previous day. However, the data in the partition that corresponds to 23:00 on the current day is analyzed. What do I do?

Appendixes: References

Appendix 1: Parameters

Custom parameters in the `${…}` format

The following table describes the sample common time formats.

Interval that you want to add or subtract	Custom parameter
N years ago or later	`${yyyy±N}`
N months ago or later	`${yyyymm±N}`
N weeks ago or later	`${yyyymmdd±7*N}`
N days ago or later	`${yyyymmdd±N}`
N days ago or later	`${yyyymmdd±N}`
N years ago or later in the yyyy format	`${yyyy±N}`
N years ago or later in the yy format	`${yy±N}`

Note

To specify a time that is N months or N years ago, we recommend that you use ${...} parameters.

You can use ${…} parameters to add or subtract only N days, months, or years.

Custom parameters in the `$[…]` format

The following table describes the sample common time formats.

Interval that you want to add or subtract	Custom parameter
N years later	`$[add_months(yyyymmdd,12*N)]`
N years ago	`$[add_months(yyyymmdd,-12*N)]`
N months later	`$[add_months(yyyymmdd,N)]`
N months ago	`$[add_months(yyyymmdd,-N)]`
N weeks ago or later	`$[yyyymmdd±7*N]`
N days ago or later	`$[yyyymmdd±N]`
N hours ago or later	You can use one of the following methods to add or subtract N hours: `$[hh24miss±N/24]` and `$[hh24±N/24]` `$[Custom time format±N/24]` For example, you can use the following custom time parameters to specify a time that is 1 hour ago: A specific month: `$[mm-1/24]` A specific year: `$[yyyy-1/24]` A specific month of a specific year: `$[yyyymm-1/24]` A specific day of a specific month of a specific year: `$[yyyymmdd-1/24]` The day before a specific day: `$[yyyymmdd-1-1/24]`
N minutes ago or later	You can use one of the following methods to add or subtract N minutes: `$[hh24miss±N/24/60]` `$[yyyymmddhh24miss±N/24/60]` `$[mi±N/24/60]` `$[Custom time format±N/24/60]` For example, you can use the following custom time parameters to specify a time that is 15 minutes earlier than the scheduling time of a task: `$[yyyy-15/24/60]` `$[yyyymm-15/24/60]` `$[yyyymmdd-15/24/60]` `$[hh24-15/24/60]` `$[mi-15/24/60]`

hh24 (24-hour format) and hh (12-hour format) indicate the hour. The value of the $cyctime parameter is accurate to the hour, minute, or second. Therefore, the value of a $[...] parameter is accurate to the hour, minute, or second. You can use engine functions to specify custom parameters in the formats that are not described in this topic. For more information, see I run an instance of a node at 00:00 on the current day to analyze the data in the partition that corresponds to 23:00 on the previous day. However, the data in the partition that corresponds to 23:00 on the current day is analyzed. What do I do?

Note

If you want to specify a time that is N hours or N minutes ago, use a $[...] parameter.

The value of a scheduling parameter is determined when an instance of a task is generated and does not change with the actual time at which the instance is run.

Built-in parameters

DataWorks supports the built-in parameters that are described in the following table. The built-in parameters can be used as values for the scheduling parameters. However, the built-in parameters are not flexible. We recommend that you use custom time parameters as values for the scheduling parameters.

Built-in parameter	Description
$bizdate	The data timestamp of the task. The value of this parameter is in the `yyyymmdd` format. The value of this parameter is the same as that of the `${yyyymmdd}` custom time parameter. This parameter is widely used. By default, the data timestamp of a task is one day earlier than the scheduling time of the task.
$cyctime	The scheduling time of the task. The value of this parameter is in the `yyyymmddhh24miss` format. The value of this parameter is the same as that of the `$[yyyymmddhh24miss]` custom time parameter.
$gmtdate	The current date. The value of this parameter is in the `yyyymmdd` format. By default, the value of this parameter is the current date. When you backfill data for a task, set this parameter to the time that is `one day later than the data timestamp of the task`.
$bizmonth	The month specified by the data timestamp of the task. The value of this parameter is in the `yyyymm` format. If the month specified by the data timestamp of the task is the current month, the value of this parameter is `one month earlier than the month specified by the data timestamp of the task`. If the month specified by the data timestamp of the task is not the current month, the value of this parameter is `the month specified by the data timestamp of the task`.
$jobid	The ID of the workflow to which the task belongs.
$nodeid	The node ID.
$taskid	The ID of the instance generated for the node.

Appendix 2: Relationships between scheduling parameters and the data timestamp, scheduling time, and actual running time of a node

Relationship	Description and example
Relationship between scheduling parameters and the actual running time of a node	The value of a scheduling parameter is determined when an instance of a node is generated and does not change with the actual time at which the node instance is run.
Relationship between scheduling parameters and the scheduling time of a node	If you use a $[...] parameter to specify a time that is accurate to the hour or minute, the value of the $[...] parameter is determined based on the scheduling time, which is the time at which the instance generated for the node is scheduled to run. Value replacement if the node is scheduled to run every hour from `00:00` to `23:59`: If the first instance generated for the node is scheduled to run at `00:00`, the value of hh24 or hh is replaced with `00`. If the second instance generated for the node is scheduled to run at `01:00`, the value of hh24 or hh is replaced with `01`.
Relationships between scheduling parameters and the data timestamp and scheduling time of a node

Appendix 3: Supported built-in scheduling parameters

Built-in scheduling parameter	Description
${projectId}	The ID of the MaxCompute project.
${projectName}	The name of the MaxCompute project.
${nodeId}	The node ID.
${gmtdate}	The date on which the instance is scheduled to run, in the yyyy-MM-dd 00:00:00 format.
${taskId}	The task instance ID.
${seq}	The sequence number of the task instance, which indicates the ranking of this instance among all instances on the same day.
${cyctime}	The time at which the instance is scheduled to run.
${status}	The status of the instance. Valid values: SUCCESS and FAILURE.
${bizdate}	The data timestamp.
${finishTime}	The time at which the instance finishes running.
${taskType}	The mode in which the instance is run. Valid values: NORMAL, MANUAL, PAUSE, SKIP, UNCHOOSE, and SKIP_CYCLE.
${nodeName}	The name of the node.

Appendix 4: Description of daylight saving time

In DataWorks, tasks can be run as expected on the day when daylight saving time begins or ends. To avoid impacts that are imposed by the switching of daylight saving time on the running of tasks, we recommend that you view the Impacts exerted by the switching of daylight saving time on the running of tasks topic if the region in which the tasks reside uses the daylight saving time.

Background information

Terms