DataWorks:Configure and use scheduling parameters

Step 1: Define parameters

1. Double-click the target node, such as a MaxCompute SQL node, to open the node editing page.

2. Define parameters in the code: In a MaxCompute SQL node, use the ${param} syntax to define a parameter name. DataWorks recommends using meaningful parameter names for easy reference and management.

   Scheduling parameter reference format:

   Format type	Syntax	Scope	Remarks
   General format	${parameter_name}	Applicable to most node types, such as MaxCompute SQL nodes and batch synchronization nodes.	This is the most commonly used format.
   Special format	Varies by node type. Does not use the ${...} format.	PyODPS, Shell	For details, see Scheduling parameter configuration for each node type.

   -- Example: Define a variable named pt_date for partition filtering
   SELECT * FROM my_table WHERE ds = '${pt_date}'; 

3. Click Scheduling Settings on the right side of the page to open the Scheduling Parameters configuration.

4. Configure scheduling parameters as described below.

Step 2: Configure parameters

You can configure scheduling parameters by using Define with table or Defined with an expression. You can switch between the two modes by clicking Defined with an expression in the upper-right corner of the parameter list. The default mode is Define with table.

1. Configure parameters

   Define with table

   Click Scheduling Settings on the right side of the node to open the scheduling parameter configuration page.

   

   • Add parameters

     Click Scheduling Settings on the right side of the node, and add parameters in Scheduling Parameters. The following methods are supported for adding parameters.

     1. Click Add parameters to manually enter the parameter name and value. The parameter name must match the variable name defined in the code.

     2. Click Loading parameters in code. DataWorks automatically parses variables from the code, such as ${pt_date}, and populates the parameter names. You only need to fill in the parameter values.

   • Assign values

     You can set system built-in parameters, custom time parameters, workspace parameters, output parameters of upstream nodes, and constants.

     • Click the input box. The drop-down list displays some commonly used parameter expressions that you can select directly. You can also manually enter a custom expression or a system built-in variable.

     • You can fill in values based on your business requirements. For supported value formats, see Supported formats for scheduling parameters.

     • Scheduling parameter assignment also supports quick binding of context parameters. Click the button next to the parameter. In the Associate Output Parameter of Ancestor Node dialog, select a node that has configured output parameters and quickly associate the dependency. For details about output parameters, see Configure output parameters.

   Define with an expression

   Click Defined with an expression to switch the configuration mode. Use this mode if you prefer configuring parameters with expressions.

   

   • When using expression-based definition, separate multiple parameters with spaces.

   • When you add, delete, or modify scheduling parameters in Defined with an expression mode, DataWorks validates the syntax of the current expression. If validation fails, the scheduling parameters cannot be configured.

     For example, DataWorks checks syntax rules such as no spaces are allowed around the equal sign.

2. Parameter preview

   After defining parameters, click Preview of scheduling parameters to preview the parameter values for N instances after a specified business date, and verify whether the parameter definitions meet expectations. You can adjust the business date and the number of instances to preview.

   

   Note

   Some nodes, such as batch synchronization nodes, come with a built-in ${bizdate} parameter. This parameter is automatically assigned the value $bizdate. You can replace the bizdate parameter name in your code with a custom parameter name. ${bizdate} itself has no special meaning and works the same as any other custom parameter.

   

Step 3: Smoke testing

After assigning values to scheduling parameters, we recommend that you use the smoke testing feature during the deployment process to configure a business date, simulate the scheduling scenario of the target task, and verify whether the code execution and parameter substitution meet expectations. If the results are not as expected, make adjustments in a timely manner to avoid affecting normal task scheduling.

1. Deploy the node code.

   

   1. Configure the scheduling time and scheduling dependency.

   2. Click Save to save the code and configuration. Then click the Deploy button on the toolbar to initiate smoke testing during the deployment process.

      Note

      If the smoke test does not execute the latest code or parameters, click Republish Production.

2. Run the smoke test.

   Click Initiate Smoke Testing. In the Smoke test dialog, select a business date, click Confirm, and start the smoke test.

3. View the smoke test logs.

   1. Click the smoke test records button on the left menu, and click Logs in the smoke test records.

      

   2. Check the parameter output in the logs to confirm whether the results meet expectations.

Step 4: Deploy and verify

After verification in the development environment, submit and deploy the task to Operation Center for production scheduling. After the task is deployed, make sure to check the scheduling parameters in the production environment to prevent runtime errors.

1. Check the parameter definitions.

   Go to Operation Center, switch to the target region and workspace, and navigate to the Auto Triggered Task O&M > Auto Triggered Task page. Click a task name in the task list and check whether the execution parameters in the General pane meet expectations.

   

2. Smoke testing in Operation Center.

   You can also use smoke testing in Operation Center to confirm whether a deployed task performs parameter substitution and code execution as expected in the production environment. For details, see Smoke testing in Operation Center.

   Important

   Note that smoke testing executes against real data in the production environment. Exercise caution to avoid polluting production data.

   

3. Observe the actual scheduling results.

   After the task enters automatic scheduling, you can further verify whether the parameter substitution is correct by checking whether the parameters in Cycle Examples have been automatically replaced as required.