DataWorks:Configure and use scheduling parameters

1. Define parameters

1. Double-click the target node, such as an ODPS SQL node, to open the node editor.

2. In the code of an ODPS SQL node or other SQL-based nodes, define a parameter name by using the ${param} syntax. DataWorks recommends using a descriptive parameter name for easier reference and management.

   Scheduling parameter syntax:

   Type	Syntax	Applicability	Notes
   General format	${parameter_name}	Applicable to most node types, such as ODPS SQL and data synchronization nodes.	This is the most common format.
   Special format	Varies by node type and does not use the ${...} format.	PyODPS and Shell	For more information, see Configuration examples for scheduling parameters of different node types.

   -- Example: Define a variable named pt_date for partition filtering.
   SELECT * FROM my_table WHERE ds = '${pt_date}'; 

3. In the right-side pane, click Scheduling to go to the Scheduling Parameter section.

4. Configure the scheduling parameters as described in the next section.

2. Configure parameters

You can set scheduling parameters in either Visual definition or Define by expression. You can switch between these modes by clicking Define by expression in the upper-right corner of the parameter list. Visual definition is the default.

1. Configure parameters

   Visual mode

   Click Scheduling in the right-side pane to open the scheduling parameters configuration panel.

   

   • Add a parameter

     Click Scheduling to the right of the node and add parameters in the Scheduling Parameter section. You can add parameters in one of the following two ways.

     1. Click Add Parameter and manually enter a name and value. The parameter name must be the same as the variable name defined in the code.

     2. Click Load Parameters in Code. DataWorks automatically parses the variables in your code, such as ${pt_date}, and populates them in the parameter list. You only need to enter a value for each parameter.

   • Assign a value

     You can assign built-in parameters, custom time variables, or constants as values.

     • Click the input box. A drop-down list appears with commonly used parameter expressions that you can select directly. You can also manually enter a custom expression or a built-in variable.

     • You can enter a value based on your business requirements. For more information about supported value formats, see Supported formats of scheduling parameters.

   Expression mode

   Click Define by expression to switch to Expression mode. Use this mode if you prefer configuring parameters with expressions.

   

   • In Expression mode, separate multiple parameters with spaces.

   • When you add, delete, or modify scheduling parameters in Define by expression, DataWorks validates the syntax of the expression. If the syntax is invalid, the parameter cannot be configured.

     For example, DataWorks enforces rules, such as not allowing spaces on either side of the equals sign (=).

2. Preview parameters

   After you define the parameters, click Preview Scheduling Parameters to see how the parameters will be resolved for upcoming instances based on a specified data timestamp. This helps you test whether the definitions are correct. You can adjust the data timestamp and the number of instances to preview.

   

3. Smoke testing

After you assign values to the scheduling parameters, use the smoke testing feature. This allows you to set a data timestamp and simulate how the task will run, which helps you verify that the code executes and the scheduling parameters are replaced as expected. If the results are not as expected, adjust the configuration promptly to avoid issues with scheduled runs.

Smoke testing generates instances, which will incur fees. For more information about instance pricing, see Billing of serverless resource groups.

1. Commit the node code.

   1. Configure the scheduling time and scheduling dependencies.

   2. Click the Save icon . After the code and configurations are saved, click the Commit icon to commit the latest code and configuration to Operation Center. You can use the smoke testing feature in the development environment only after the node is committed.

      Note

      If you find that smoke testing is not running the latest code or parameters, commit the node again.

2. Run the smoke test.

   Click the Smoke Testing icon in the toolbar. In the Perform Smoke Testing dialog box, select a data timestamp and click OK to start the test.

3. View the smoke testing logs.

   1. In the Smoke Testing Records window that appears, find the latest record and click View Log.

      

   2. Check the parameter output in the logs to confirm that the values are correct.

      Note

      If you accidentally close the window, you can reopen it by clicking the Smoke Testing Logs icon in the toolbar.

4. Deploy and verify

After verification in the development environment, you can deploy the task to Operation Center for scheduled execution in production. After deployment, check the scheduling parameters in the production environment to prevent runtime errors.

1. Check the parameter definitions.

   Go to Operation Center, switch to the target region and workspace, and navigate to the Auto Triggered Task O&M > Periodic tasks page. In the task list, click the task name and check the Property in the Property panel to ensure they are correct.

   

2. Perform smoke testing in Operation Center.

   You can also use smoke testing in Operation Center to confirm that a deployed task replaces parameters and executes code as expected in the production environment. For more information, see Run a test and view the test instance.

   Important

   Note that smoke testing runs on real production data. Proceed with caution to avoid corrupting data in your production database.

   

3. Observe the actual scheduling results.

   After the task is automatically scheduled, you can verify that the parameters were replaced correctly by checking the parameters of an auto triggered instance in Auto Triggered Instances.