This topic describes the FAQ about scheduling parameters.

How do I specify a table partition format that contains a space?

  • Symptom: The table partition format is pt=yyyy-mm-dd hh24:mi:ss, but spaces are not allowed in scheduling parameters.
  • Workaround: Use the custom variables datetime=$[yyyy-mm-dd] and hour=$[hh24:mi:ss] to obtain the date and time, respectively. Then, join them together to form pt=${datetime} ${hour} in the code.

    Separate the two variables with a space.

What can I do if an instance that is run at 00:00 analyzes data for 23:00 of the current day, instead of 23:00 of the previous day?

  • Symptom: The table partition is pt=${datetime} ${hour} in the code. To obtain the data for the last hour when the node is run, the custom variables datetime=$[yyyymmdd] and hour=$[hh24-1/24] can be used. However, an instance that is run at 00:00 analyzes data for 23:00 of the current day, instead of 23:00 of the previous day.
  • Workaround:
    Change the formula of datetime to $[yyyymmdd-1/24] and retain the formula $[hh24-1/24] for hour. The node is run as follows:
    • For an instance that is scheduled to run at 2015-10-27 00:00:00, the values of $[yyyymmdd-1/24] and $[hh24-1/24] are 20151026 and 23, respectively. This is because the scheduled time minus 1 hour is a time point of the previous day.
    • For an instance that is scheduled to run at 2015-10-27 01:00:00, the values of $[yyyymmdd-1/24] and $[hh24-1/24] are 20151027 and 00, respectively. This is because the scheduled time minus 1 hour is a time point of the current day.

In which modes can nodes be run in DataWorks?

DataWorks offers the following node running modes:
  • You can run a node on the DataStudio page in one of the following modes:
    • Run: After you click the Run icon for the first time, you must manually assign values to the variables in the code. DataWorks records these values. If you modify the code, the variables still use the values assigned when the node is run for the first time.
    • Run with Arguments: If you need to re-assign values to the variables in the code or select another resource group on which the node is to be run, select this mode.
    • Run Smoke Test in Development Environment: You can enter a data timestamp to simulate automatic node scheduling and obtain the replaced values of scheduling parameters at the specified data timestamp.
    Note If you select the Run or Run with Arguments mode, the system directly replaces the variables in the code with the values you specify and does not verify the values. If you want the system to verify the values, select the Run Smoke Test in Development Environment mode.
  • Run a node in the production environment: The scheduling system automatically replaces the values of scheduling parameters, including system parameters and custom parameters, based on the scheduled running time of the current instance.
  • Test a node or generate retroactive data: You must specify the data timestamp. The scheduled time of each instance can be calculated by using the following formula: Scheduled time = Data timestamp + 1.

What can I do if the system does not parse the specified custom parameter, which leads to the failure to run the corresponding node?

  • Symptom: The system does not parse the custom parameter $[yyyymmdd] specified for INSERT into table log_in PARTITION (dat) SELECT id,name,age,sex,${var} from test1;. As a result, the corresponding node fails to be run.
  • Workaround: Perform smoke testing for the system to parse the custom parameter.

How are node instances generated on the days when the daylight saving time begins and ends, respectively?

DataWorks supports the immediate instance generation and daylight saving time-based parameter computing features so that nodes can be run properly when the daylight saving time begins or ends. Assume that the time zone is UTC-8.
  • When the daylight saving time begins, 23 instances are generated on that day. Ten minutes before 03:00 is 01:50. The system does not actually run the instance that is scheduled to run at 02:00 on that day.
  • When the daylight saving time ends, 24 instances are generated on that day. Ten minutes before 03:00 is 02:50.

If a node scheduled by day, week, or month is scheduled to run within the skipped period of the day when the daylight saving time begins, a node instance is generated and run at 00:00 on that day.