You can reference the method that is used to configure scheduling parameters for SQL nodes, such as MaxCompute SQL nodes, to define custom variables and assign scheduling parameters to the custom variables for other types of nodes, excluding common Shell nodes and PyODPS nodes. Specific configurations differ for common Shell nodes and PyODPS nodes. This topic provides examples on how to configure scheduling parameters for different types of nodes.
SQL nodes and batch synchronization nodes
Scheduling parameters are configured for SQL nodes and batch synchronization nodes by using a method that is suitable for most types of nodes. You can reference the method to configure scheduling parameters for most types of nodes. In this example, a MaxCompute SQL node is used. This section describes how to assign custom parameters to custom variables, and how to reference the custom variables in code.
Specific nodes do not support scheduling parameters. For more information about whether a type of node supports scheduling parameters, see the documentation of the type of node.
-- Obtain the data timestamp.
SELECT '${var1}' '${var2}';
-- Obtain the scheduling time.
SELECT '${var3}' '${var4}';
-- Obtain a constant.
SELECT '${var5}';
Assign values to your parameters in the Scheduling Parameters section. Then, you can reference parameters such as var1, var2, and var5 in the code editor. The following examples show how to assign values to these parameters from sources like built-in system parameters, custom formats, and constants:
var1: Specify $bizdate as var1. This variable is used to obtain the data timestamp. $bizdate is a built-in system parameter.var3: Specify $cyctime as var3. This variable is used to obtain the scheduling time. $cyctime is a built-in system parameter.var2: Specify ${yyyymmdd} as var2. This variable is used to obtain the data timestamp.var4: Specify $[yyyymmddhh24:mi:ss] as var4. This variable is used to obtain the scheduling time.var5: Assign the constant abc to this variable.
For more information about how to configure and use scheduling parameters, see the Configure node scheduling properties section of the "Scheduling properties" topic. For more information about how to assign custom parameters to custom variables, see Sources and expressions of scheduling parameters.
PyODPS nodes
To avoid code changes, a PyODPS node does not allow you to replace defined variables with strings that are in the ${param_name} format in the code. However, you can add a dictionary object named args to the global variable before the code is run. This way, the PyODPS node can obtain the values of scheduling parameters from the dictionary object.
# Custom parameter: var1 gets a built-in parameter value
a =args['var1']
print (format(a))
# Custom parameter: var2 gets a value in the ${...} format
b =args['var2']
print (format(b))
# Custom parameter: var3 gets a value in the $[...] format
c =args['var3']
print (format(c))
As shown in the figure above, assign values to parameters in the parameter assignment area. Then, reference the built-in parameter var1 and the custom parameters var2 and var3 in the code invocation area. After a dictionary object is added, the parameters are referenced as args['var1'], args['var2'], and args['var3']. An assignment example is as follows:
var1: Specify $bizdate as var1. This variable is used to obtain the data timestamp. $bizdate is a built-in system parameter.var2: Specify ${yyyymmdd} as var2. This variable is used to obtain the data timestamp.var3: Specify $[yyyymmdd] as var3. This variable is used to obtain the data timestamp.
For more information about how to configure and use scheduling parameters, see the Configure node scheduling properties section of the "Scheduling properties" topic. For more information about how to assign custom parameters to custom variables, see Sources and expressions of scheduling parameters.
Common Shell nodes
You are not allowed to specify custom variable names for common Shell nodes. The variables must be named based on their ordinal numbers, such as $1, $2, and $3. If the number of parameters reaches or exceeds 10, use ${Number} to declare the excess variables. For example, use ${10} to declare the tenth variable.
# Positional parameter: Get a built-in parameter value
echo $1
# Positional parameter: ${...} format
echo $2
# Positional parameter: $[...] format
echo $3
As shown in the preceding figure, you assign values in the Scheduling Parameters section. These values are then passed to the positional parameters $1, $2, and $3 that are referenced in the code editor. The following examples show how to assign these values:
For common Shell nodes, you can assign custom parameters to custom variables only by using expressions. Separate multiple custom parameters with spaces. Make sure that you assign the custom parameters to the custom variables in the same order in which the custom variables are defined. For example, the first parameter $bizdate that you enter in the Scheduling Parameters section is assigned to the first variable $1.
$1: Specify
$bizdateas $1. This variable is used to obtain the data timestamp. $bizdate is a built-in system parameter.$2: Specify
${yyyymmdd}as $2. This variable is used to obtain the data timestamp.$3: Specify
$[yyyymmdd]as $3. This variable is used to obtain the scheduling time.
For more information about how to configure and use scheduling parameters, see the Configure node scheduling properties section of the "Scheduling properties" topic. For more information about how to assign custom parameters to custom variables, see Sources and expressions of scheduling parameters.