DataWorks:Configure and use scheduling parameters

1. Define parameters

1. Double-click the target node, such as an ODPS SQL node, to open the node editor.

2. Define parameters in the code. In the code of an ODPS SQL node or another SQL node, you can use the ${param} syntax to define a parameter name. DataWorks recommends that you use meaningful parameter names for easier reference and management.

   Scheduling parameter call formats:

   Format type	Call syntax	Scope	Notes
   General format	${ParameterName}	Applies to most node types, such as ODPS SQL and synchronization nodes.	This is the most common format.
   Special format	Varies by node. Not in the ${...} format.	PyODPS, Shell	For more information, see Examples of scheduling parameter configurations for different node types.

   -- Example: Define a variable named pt_date for partition filtering
   SELECT * FROM my_table WHERE ds = '${pt_date}'; 

3. On the right side of the page, click Scheduling Configurations to go to the Scheduling Parameters section.

4. Configure the scheduling parameters as described in the next section.

2. Configure parameters

You can set scheduling parameters in two ways: Visual Definition and Define By Expression. You can switch between these modes by clicking Define By Expression in the upper-right corner of the parameter list. The default mode is Visual Definition.

1. Configure parameters

   Visual definition

   Click Scheduling Configurations to the right of the node to open the scheduling parameter configuration interface.

   

   • Add parameters

     To the right of the node, click Scheduling Configurations. In the Scheduling Parameters section, you can add parameters in one of the following two ways.

     1. Click Add Parameter and manually enter the parameter name and value. The parameter name must match the variable name defined in the code.

     2. Click Load Parameters From Code. DataWorks automatically parses variables in the code, such as ${pt_date}, and adds them as parameters. Then, enter a value for each parameter.

   • Assign values to parameters

     You can set built-in system variables, custom time variables, and constants.

     • Click the input box. The drop-down list shows some common parameter expressions that you can select directly. You can also manually enter a custom expression or a built-in system variable.

     • Enter values as needed. For more information about the supported range of parameter values, see Supported formats for scheduling parameters.

   Define by expression

   Click Define By Expression to configure parameters using expressions.

   

   • When you use an expression to define multiple parameters, you must separate them with spaces.

   • When you use the Define By Expression method to add, delete, or modify scheduling parameters, DataWorks validates the expression syntax. You cannot configure the scheduling parameter if the syntax is invalid.

     For example, DataWorks checks for syntax rules such as no spaces are allowed on either side of the equal sign.

2. Parameter preview

   After you define the parameters, click Scheduling Parameter Preview to preview the parameters for the next N instances that run after a specified data timestamp. This helps you verify that the parameter definitions are configured as expected. You can adjust the data timestamp and the number of instances for the preview.

   

3. Smoke testing

After you assign values to the scheduling parameters, you can use the smoke testing feature. You can configure a data timestamp to simulate the scheduling scenario for the target node. You can verify that the code execution and parameter replacement work as expected in this scenario. If they do not, you must adjust the settings as needed to prevent issues with normal task scheduling.

Smoke testing generates instances and incurs instance fees. For more information about instance fees, see Billing of serverless resource groups.

1. Submit the node code.

   1. Configure the schedule time and scheduling dependencies.

   2. Click the save icon to save the code and configuration. Then, click the submit icon . You can use the smoke testing feature in the development environment only after the latest code for the node is submitted to Operation Center.

      Note

      If you find that the smoke test is not running the latest code or parameters, you must submit the node again.

2. Run a smoke test.

   Click the smoke testing icon in the toolbar. In the Smoke Testing dialog box, select a data timestamp and click OK to run the smoke test.

3. View smoke test logs.

   1. In the Smoke Test Records window, find the latest record and click View Log.

      

   2. In the log, check the parameter output to confirm that it meets your expectations.

      Note

      If you accidentally close the window, you can click the smoke test records icon in the toolbar to reopen it.

4. Publish and verify

After verification in the development environment, you can submit and publish the task to Operation Center for production and automatic scheduling. After you publish the task, you must check the scheduling parameters in the production environment to prevent runtime errors.

1. Check parameter definitions.

   Go to Operation Center, switch to the destination region and workspace, and navigate to the Recurring Task O&M > Recurring Tasks page. In the task list, click the task name and verify that the execution parameters in the Properties panel are correct.

   

2. Smoke testing in Operation Center.

   In Operation Center, you can also use smoke testing to confirm whether a submitted and published task performs parameter replacement and code execution as expected in the production environment. For more information, see Run a test and view the test instance.

   Important

   Note that smoke testing will actually execute on production data. You must proceed with caution to avoid contaminating the production database.

   

3. Observe actual scheduling results.

   After the task is automatically scheduled, you can further verify that the parameters were replaced as required by checking them in the Recurring Instance.