In DataWorks, a node is scheduled to run based on the scheduling parameters that you configure for the node. The values of the scheduling parameters are automatically replaced with values in the specified format based on the data timestamp and scheduling time of the node. The values of the scheduling parameters are dynamically replaced at the time at which the node is scheduled to run. This topic describes the types of scheduling parameters and specific scheduling parameters of each type.

You can learn scheduling parameters from the following aspects:
  • The value of a scheduling parameter that you configure for an auto triggered node is determined based on the data timestamp and scheduling time of the auto triggered node. For definitions of the data timestamp and scheduling time of an auto triggered node, see Description of scheduling parameter values.
  • Scheduling parameters are classified into custom parameters and built-in variables based on the methods that are used to assign values to the parameters. For more information, see Types of scheduling parameters.
  • You can view specific scheduling parameters. For more information, see Scheduling parameters.

After you understand scheduling parameters, you can go to the Properties tab to configure scheduling parameters. For information about how to configure and use scheduling parameters, see Configure and use scheduling parameters.

DataWorks also provides examples on how to use scheduling parameters in typical scenarios:

Description of scheduling parameter values

The following table describes the values of scheduling parameters.
Parameter value Description
Data timestamp The data timestamp of a node, which is one day earlier than the time at which the node is scheduled to run as expected. The value is accurate to the day. The data timestamp of a node can be obtained by using the $bizdate or ${yyyymmdd} parameter. In most cases, the data timestamp of a node is calculated by subtracting one day from the time at which the node is scheduled to run.
Scheduling time The time at which a node is scheduled to run as expected. The value is accurate to the second. The scheduling time of a node can be obtained by using the $cyctime or $[yyyymmddhh24miss] parameter.

Types of scheduling parameters

Scheduling parameters are classified into custom parameters and built-in variables based on the methods that are used to assign values to the parameters.
  • Built-in variables: You can obtain the data timestamp and scheduling time of a node from the code of the node.
  • Custom parameters: You can customize the names of variables in the code of a node based on your business requirements. Then, in the Parameters section of the Properties tab, you can assign values in custom formats to the variables in the code of the node in a centralized manner. This way, time parameters of different formats can be dynamically obtained.
The following figure shows the types of scheduling parameters and value assignment logic. Scheduling parameters
Note
  • For nodes of common types, you can define parameters and assign values to the parameters by following the value assignment logic that is shown in the preceding figure. The scheduling parameters for the common Shell and PyODPS nodes differ from the scheduling parameters for the nodes of common types. For more information, see Configure scheduling parameters for different types of nodes.
  • Some nodes, such as HTTP Trigger nodes, do not support scheduling parameters. For more information about whether a type of node supports scheduling parameters, see the documentation of the type of node.
  • Custom parameters
    The following table describes the types of custom parameters to which you can assign values.
    Type Description Format Remarks
    Built-in parameters You can assign the $bizdate and $cyctime parameters to variables in the Parameters section of the Properties tab to obtain the data timestamp and scheduling time of a node. The value format is fixed.
    • The value format of $bizdate is yyyymmdd.
    • The value format of $cyctime is yyyymmddhh24miss.
    N/A.
    ${...} parameters ${...} parameters are time parameters that are based on the built-in parameter $bizdate. ${...} parameters are generated by combining yyyy, yy, mm, and dd. You can specify a custom value format.

    For example, you can specify a ${...} parameter in the format of ${yyyy}, ${yyyymm}, ${yyyymmdd}, or ${yyyy-mm-dd}.

    Note The value of the $bizdate parameter is the same as that of the ${yyyymmdd} parameter.
    • If you want to specify a time that is N years or months ago, use a ${...} parameter.
      Note
      • ${...} parameters are accurate to the day. You cannot use a ${...} parameter to specify a time that is accurate to the hour, minute, or second. For example, you cannot specify a ${...} parameter in the format of ${yyyy-mm-dd-1/24}.
      • If you want to specify a calculation-based time, we recommend that you use a ${...} parameter. For example, you can specify a parameter in the format of ${yyyy-N} or ${mm-N}.
    • If you want to specify a time that is N hours or minutes ago, use a $[...] parameter.
      Note
      • You cannot specify a $[...] parameter in the format of $[yyyy-N] or $[mm-N], in which N indicates the number of years or months. This indicates that you cannot specify a time that is N years or months ago by using a $[...] parameter in the preceding formats.
      • If you want to specify a time that is accurate to the second, we recommend that you use a $[...] parameter. For example, you can specify a parameter in the format of $[yyyy-mm-dd-1-1/24].
    After you configure the scheduling parameters, you can test whether the parameters work as expected. For more information, see Test scheduling parameters.
    $[...] parameters $[...] parameters are time parameters that are based on the built-in parameter $cyctime. $[...] parameters are generated by combining yyyy, yy, mm, dd, hh24, mi, and ss. You can specify a custom value format.

    For example, you can specify a $[...] parameter in the format of $[yyyymmdd], $[yyyy-mm-dd], $[hh24miss], $[hh24:mi:ss], or $[yyyymmddhh24miss].

    Note The value of the $cyctime parameter is the same as that of the $[yyyymmddhh24miss] parameter.
    Constants You can assign constants to variables in the Parameters section of the Properties tab. For example, you can assign "abc" or 1234 to a variable. The value format is not fixed. N/A.
    For information about the comparison of value formats of custom parameters, see Compare custom parameters.
  • Built-in variables
    The following table describes the built-in variables that are supported by DataWorks.
    Type Format Remarks
    Data timestamp: ${bdp.system.bizdate} Fixed format: yyyymmdd You can reference built-in variables in the code of your node without the need to assign values to them.
    Scheduling time: ${bdp.system.cyctime} Fixed format: yyyymmddhh24miss
    Note Only Alibaba Cloud public cloud and Alibaba Cloud Apsara Stack support the preceding parameters.

Scheduling parameters

  • Built-in parameters
    The following table describes the built-in parameters that are supported by DataWorks.
    Built-in parameter Description
    $bizdate The data timestamp of the node. The value of this parameter is in the format of yyyymmdd.

    This parameter is widely used. By default, the data timestamp of a node is one day earlier than the time at which the node is scheduled to run as expected.

    $cyctime The time at which the node is scheduled to run. The value of this parameter is in the format of yyyymmddhh24miss.
    $gmtdate The current date. The value of this parameter is in the format of yyyymmdd.

    By default, the value of this parameter is the current date. When you backfill data for a node, set this parameter to the time that is one day later than the data timestamp of the node.

    $bizmonth The month indicated by the data timestamp of the node. The value of this parameter is in the format of yyyymm.
    • If the month indicated by the data timestamp of the node is the current month, the value of this parameter is one month earlier than the month indicated by the data timestamp of the node.
    • If the month indicated by the data timestamp of the node is not the current month, the value of this parameter is the month indicated by the data timestamp of the node.
    $jobid The ID of the workflow to which the node belongs.
    $nodeid The ID of the node.
    $taskid The ID of the node instance.
  • Custom parameters in the ${...} format
    The following table describes the custom parameters that you can configure based on the built-in parameter $bizdate to specify a calculation-based time.
    Interval that you want to add or subtract Custom parameter
    N years ago or later ${yyyy±N}
    N months ago or later ${yyyymm±N}
    N weeks ago or later ${yyyymmdd±7*N}
    N days ago or later ${yyyymmdd±N}
    N days ago or later ${yyyymmdd±N}
    N years ago or later in the yyyy format ${yyyy±N}
    N years ago or later in the yy format ${yy±N}
    Take note of the following variables:
    • yyyy: indicates a four-digit year, which is the year in the value of the $bizdate parameter.
    • yy: indicates a two-digit year, which is the year in the value of the $bizdate parameter.
    • mm: indicates the month, which is the month in the value of the $bizdate parameter.
    • dd: indicates the day, which is the day in the value of the $bizdate parameter.
    Note
    • If you want to specify a time that is N months or years ago, use a ${...} parameter.
    • The value of the $bizdate parameter is accurate to the day. Therefore, the value of a ${…} parameter is accurate to the day.
    • You can use engine functions to specify custom parameters in the formats that are not described in this topic. For more information, see Process the return values of scheduling parameters.
  • Custom parameters in the $[...] format
    The following table describes custom parameters that you can configure based on the built-in parameter $cyctime to specify a calculation-based time.
    Interval that you want to add or subtract Custom parameter
    N years later $[add_months(yyyymmdd,12*N)]
    N years ago $[add_months(yyyymmdd,-12*N)]
    N months later $[add_months(yyyymmdd,N)]
    N months ago $[add_months(yyyymmdd,-N)]
    N weeks ago or later $[yyyymmdd±7*N]
    N days ago or later $[yyyymmdd±N]
    N hours ago or later You can specify a parameter in one of the following formats:
    • $[hh24miss±N/24]
    • $[Custom time format±N/24]. Example: $[hh24±N/24]
    N minutes ago or later You can specify a parameter in one of the following formats:
    • $[hh24miss±N/24/60]
    • $[Custom time format±N/24/60]. Examples: $[mi±N/24/60] and $[yyyymmddhh24miss±N/24/60]
    Take note of the following variables:
    • yyyy: indicates a four-digit year, which is the year in the value of the $cyctime parameter.
    • yy: indicates a two-digit year, which is the year in the value of the $cyctime parameter.
    • mm: indicates the month, which is the month in the value of the $cyctime parameter.
    • dd: indicates the day, which is the day in the value of the $cyctime parameter.
    • hh24 (24-hour format) and hh (12-hour format): indicate the hour, which is the hour in the value of the $cyctime parameter.
    • ss: indicates the second, which is the second in the value of the $cyctime parameter.
    • mi: indicates the minute, which is the minute in the value of the $cyctime parameter.
    Note
    • If you want to specify a time that is N hours or minutes ago, use a $[...] parameter.
    • The value of the $cyctime parameter is accurate to the hour, minute, or second. Therefore, the value of a $[…] parameter is accurate to the hour, minute, or second.
    • You can use engine functions to specify custom parameters in the formats that are not described in this topic. For more information, see Process the return values of scheduling parameters.
    • The value that replaces a scheduling parameter is determined when a node instance is generated and does not vary based on the time at which the node instance is actually run.
    • If you use a $[...] parameter to specify a time that is accurate to the hour or minute, the value that replaces the $[...] parameter is determined based on the time at which a node instance is scheduled to run.
      The following examples show how a $[...] parameter is replaced:
      • If the node is scheduled to run at 01:00 every day, the value of hh24 or hh is replaced with 01.
      • Value replacement if the node is scheduled to run every hour from 00:00 to 23:59:
        • If the first instance generated for the node is scheduled to run at 00:00, the value of hh24 or hh is replaced with 00.
        • If the second instance generated for the node is scheduled to run at 01:00, the value of hh24 or hh is replaced with 01.
        • The preceding process is repeated every hour until the last instance is generated for the node within the day.