Configure and use scheduling parameters

Process of configuring and using scheduling parameters

1. Go to the Parameters section of the Properties tab on the configuration tab of a desired node. For more information, see Go to the Parameters section of the Properties tab.
2. Configure scheduling parameters for the node. For more information, see Configure scheduling parameters.
3. Test whether the scheduling parameters work as expected. For more information, see Test scheduling parameters.
4. Check the configurations of the scheduling parameters in the production environment. For more information, see View the configurations of scheduling parameters in Operation Center in the production environment.

Go to the Parameters section of the Properties tab

1. Go to the DataStudio page.
   1. Log on to the DataWorks console. Select a region in the top navigation bar. Then, click Workspaces in the left-side navigation pane.
   2. On the Workspaces page, find your workspace and click DataStudio in the Actions column. The DataStudio page appears.
2. Go to the Parameters section of the Properties tab.
   
   1. In the Scheduled Workflow pane of the DataStudio page, double-click a desired node to go to the configuration tab of the node.
   2. On the configuration tab, click the Properties tab in the right-side navigation pane.
   3. In the Parameters section of the Properties tab, configure scheduling parameters for the node.

Configure scheduling parameters

In the Parameters section, you can add parameters or load existing parameters in the code of the node on a visual interface. Alternatively, you can use expressions to define scheduling parameters.

Method to define scheduling parameters	Feature	Description	Screenshot
Visual interface	Add Parameter	You can click Add Parameter to configure multiple scheduling parameters for a node. DataWorks provides the following scheduling parameters: Previous day: ${yyyymmdd} Current day: $[yyyymmdd] Previous hour: $[hh24-1/24] First day of the current month: ${yyyymm}01 First day of the previous month: ${yyyymm-1}01 For more information about scheduling parameters, see Overview of scheduling parameters.
	Load Parameters in Code	After you click Load Parameters in Code, DataWorks identifies the variable names defined in the code of the current node and adds the identified variable names to the Parameters section. Note You do not need to assign values to built-in variables. The variables that are loaded are custom variables. You must manually assign values to them. In most cases, custom variables are defined in the format of ${Custom variable name} in the code. For PyODPS nodes or common Shell nodes, variable names are defined by using a different method from that for other types of nodes. For more information about how to define custom variables for different types of nodes, see Configure scheduling parameters for different types of nodes.
Expressions	Define by expression	By default, the system provides a visual interface on which you can configure scheduling parameters for a node. You can also click Define by expression to configure scheduling parameters by using an expression. Note When you use an expression to configure multiple scheduling parameters, separate multiple assignment equations with spaces. For example, if you want to configure two assignment equations datetime1=$[yyyymmdd] and datdatetime2=$bizdate, the expression is datetime1=$[yyyymmdd] datetime2=$bizdate. If you add, delete, or modify scheduling parameters by using expressions, DataWorks checks the syntax of the expressions. If the check fails, the scheduling parameters fail to be configured. For example, if a space exists on either side of the equal sign (=), DataWorks reports an error.

Note If you use the built-in variable ${bdp.system.bizdate} or ${bdp.system.cyctime} in the code of a node, you do not need to assign values to the variable in the Parameters section.

Test scheduling parameters

After you configure scheduling parameters, you must click the icon to perform smoke testing in the development environment. You can configure a data timestamp to simulate the scenario in which the node is scheduled to run. Then, you can check whether the scheduling parameters work as expected.

Note When you perform smoke testing in the development environment, you are charged for the generated test instances. For more information about instance pricing, see Shared resource group for scheduling and Billing of exclusive resource groups for scheduling (subscription).

After the smoke testing is complete, click the icon to check whether the results in smoke testing logs meet your expectations.

Note

• Before you click the or icon in the toolbar to run a node, you must manually assign constants to the variables that you want to reference in the code of the node. In this case, you cannot check whether the configured scheduling parameters work as expected.
• After you modify the code of a node, you must click the icon and then the icon to save the modifications and commit the node to the development environment. Otherwise, you cannot perform smoke testing on the node in the development environment.

View the configurations of scheduling parameters in Operation Center in the production environment

To prevent unexpected configurations of scheduling parameters from affecting the running of an auto triggered node, we recommend that you check the configurations of the scheduling parameters for the auto triggered node on the Cycle Task page in Operation Center in the production environment after the auto triggered node is deployed. For more information about auto triggered nodes, see View and manage auto triggered nodes.

Note If the scheduling parameters of an auto triggered node are not configured as expected or you cannot find the auto triggered node in Operation Center, check whether the node is successfully deployed. For more information about how to deploy nodes, see Deploy nodes.

Example on how to configure scheduling parameters

1. Edit the code of the node and configure the scheduling parameters.
   The following figure shows the code and scheduling parameter configuration of the ODPS SQL node.
   1. Define variables in the code.
      Define variables '${var1}' and '${var2}' to reference built-in parameters and define variables '${var3}' and '${var4}' to reference custom parameters in the code of the ODPS SQL node, as shown in Section 1 in the preceding figure.
   2. Assign values to the variables.
      Go to the Parameters section of the Properties tab and assign values to the variables, as shown in Section 2 in the preceding figure.

      • var1=$bizdate: the data timestamp in the yyyymmdd format.
      • var2=$cyctime: the time at which the node is scheduled to run in the yyyymmddhh24miss format.
      • var3=${yyyymmdd}: the data timestamp in the yyyymmdd format.
      • var4=$[yyyymmddhh24miss]: the time at which the node is scheduled to run in the yyyymmddhh24miss format.
   3. Optional:Configure a scheduling cycle.
      Configure the ODPS SQL node to run at an interval of a specified number of hours, as shown in Section 3 in the preceding figure.

      Note You can determine whether to configure a scheduling cycle for a node based on your business requirements. In this example, a scheduling cycle is configured.

      • Start from: 16:00
      • End at: 23:59
      • Interval: 1 hour

      For more information about how to configure a scheduling cycle, see Configure time properties.
2. In the top toolbar of the configuration tab of the node, click the icon and then the icon to save the configurations of the ODPS SQL node and commit the ODPS SQL node.
3. Perform smoke testing in the development environment.
   1. Click the icon. In the Smoke Testing in Development Environment dialog box, configure a data timestamp to simulate the scheduling cycle of the node.
      In this example, configure a data timestamp in the following way:

      • Data timestamp: 2022-03-09
      • Start from: 16:00
      • End at: 17:00

      The ODPS SQL node is scheduled by hour. Two instances are expected to be generated for the node at 16:00 and 17:00 on March 10, 2022.

      Note The data timestamp is one day earlier than the time at which the node is scheduled to run. Therefore, the node is scheduled to run on March 10, 2022.

      The following results are expected for the instance that is generated at 16:00:

      • var1=20220309
      • var2=20220310160000
      • var3=20220309
      • var4=20220310160000

      The following results are expected for the instance that is generated at 17:00:

      • var1=20220309
      • var2=20220310170000
      • var3=20220309
      • var4=20220310170000
   2. Click OK. The node runs at the scheduled time.
   3. After the smoke testing is complete, click the icon to view the smoke testing logs.
      The two instances generated by the node are successfully run and the test result meets expectations.
4. On the configuration tab of the ODPS SQL node, click Deploy on the right side of the top navigation bar to deploy the current node.
   For more information about how to deploy nodes, see Deploy nodes.
5. Go to Operation Center and check the configurations of the scheduling parameters of the node.
   
   1. On the DataStudio page, click Operation Center on the right side of the top navigation bar to go to the Operation Center page.
   2. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task. On the Cycle Task page, find the node that you want to manage.
      Note You can find a node on the Cycle Task page only after the node is deployed.
   3. Choose More > View Node Details in the Actions column of the node. On the Node Information page, view the value of Execution Parameters.
      In this example, the scheduling parameters of the node are configured as expected in the following way: var1=$bizdate var2=$cyctime var3=${yyyymmdd} var4=$[yyyymmddhh24miss].