Process scheduling parameter return values with functions to generate custom time formats - DataWorks

Scheduling parameters return a standard time format by default. When that format does not fit complex business needs—such as getting the last day of a month, calculating a quarter, or building precise time windows for incremental synchronization—use engine functions to post-process the return values.

Two approaches are available:

Direct processing: Use engine functions inside a compute node that supports function execution, such as an ODPS SQL node.
Indirect processing: If the target node (for example, a batch synchronization node) does not support function execution, add an upstream assignment node to perform the transformation and pass the result downstream as a context parameter.

Choose an approach

Situation	Approach
Target node supports function execution (e.g., ODPS SQL node)	Direct processing
Target node does not support function execution (e.g., batch synchronization node)	Indirect processing via an assignment node

Scheduling parameter quick reference

The following table lists the parameter expressions used in this document. For the examples below, the scheduled time (CYCTIME) is 20250727000500.

Expression	Returns	Example output
`$[yyyy-mm]`	Year-month of scheduled time	`2025-07`
`$[mm]`	Two-digit month	`07`
`$[yyyy-15/24/60]`	Year, 15 minutes before scheduled time	`2025`
`$[yyyymm-15/24/60]`	Year and month, 15 minutes before	`202507`
`$[yyyymmdd-15/24/60]`	Date, 15 minutes before	`20250726`
`$[hh24-15/24/60]`	Hour (24-hour), 15 minutes before	`23`
`$[mi-15/24/60]`	Minute, 15 minutes before	`50`
`$[yyyymmdd-1]`	Yesterday's date	`20250726`
`$[yyyymmdd]`	Today's date	`20250727`
`$[yyyymmddhh24-1/24]`	Start of the previous hour	`20250726230000`
`$[yyyymmddhh24]`	Start of the current hour	`20250727000000`
`$[yyyymmddhh24miss]`	Full datetime of scheduled time	`20250727000500`

Direct processing in an ODPS SQL node

ODPS SQL nodes support MaxCompute SQL functions directly in SQL statements. The following examples use a CYCTIME of 20250727000500. For other types of SQL nodes, refer to the function types supported by the specific engine.

Get the last day of the previous month

Use this pattern to get the last day of the previous month in yyyymmdd format—commonly used for monthly partition keys or financial settlement dates.

Parameter configuration

last_month=$[yyyy-mm]

This returns the year and month of the scheduled time (for example, 2025-07).

SQL

SELECT REPLACE(DATEADD(date'${last_month}-01', -1, 'dd'), '-', '');

The query constructs the first day of the current month, subtracts one day with DATEADD, then removes the - separators with REPLACE.

Result: 20250630

Get the current quarter

Use this pattern to calculate the quarter number (1–4) based on the scheduled month.

Parameter configuration

month=$[mm]

This returns the two-digit month (for example, 07).

SQL

SELECT CEIL(CAST('${month}' AS INT) / 3);

The query casts the month string to an integer, divides by 3, and rounds up with CEIL.

Result: 3

Get time components 15 minutes before the scheduled time

Use this pattern when a pipeline needs to slice data at a precise time offset relative to when the task runs. Use the -15/24/60 arithmetic expression directly in the parameter definition to shift each time component back 15 minutes.

Parameter configuration

year=$[yyyy-15/24/60]
month=$[yyyymm-15/24/60]
day=$[yyyymmdd-15/24/60]
hour=$[hh24-15/24/60]
mi=$[mi-15/24/60]

SQL

SELECT 'year=${year} month=${month} day=${day} hour=${hour} mi=${mi}';

Reference the pre-calculated parameters directly—no additional function calls are needed.

Result: year=2025 month=202507 day=20250726 hour=23 mi=50

In this example, shifting back 15 minutes from 00:05 on July 27 crosses midnight into July 26. The scheduling parameter expressions compute the correct date and time components for the offset automatically based on the CYCTIME.

Generate time intervals for incremental synchronization

Batch synchronization tasks often require a yyyymmddhh24miss time range using a left-closed, right-open interval [begin, end). The following examples show how to build that interval for common scheduling frequencies.

Daily interval

Get the interval from 00:00:00 yesterday to 00:00:00 today.

Parameter configuration

beginDateTime=$[yyyymmdd-1]
endDateTime=$[yyyymmdd]

Filter condition

event_time >= '${beginDateTime}000000' AND event_time < '${endDateTime}000000'

Concatenate 000000 to each date parameter to complete the hhmmss portion.

Result: 20250726000000 to 20250727000000

Hourly interval

Get the interval from the start of the previous hour to the start of the current hour.

Parameter configuration

beginDateTime=$[yyyymmddhh24-1/24]
endDateTime=$[yyyymmddhh24]

Filter condition

event_time >= '${beginDateTime}0000' AND event_time < '${endDateTime}0000'

Concatenate 0000 to each parameter to complete the mmss portion.

Result: 20250726230000 to 20250727000000

Pre-process parameters using an assignment node

If a target node, such as a batch synchronization node, does not support function execution in its code, use an assignment node for the transformation. The assignment node runs the conversion and exposes its output as a context parameter for downstream nodes.

How the data flows

Assignment node                     Downstream node
─────────────────────               ──────────────────────────────
Compute: unix_timestamp(...)   →    Reference: ${context_param_name}
Output: context parameter

Steps

Create an assignment node.
In the assignment node code, use an engine function to convert a scheduling parameter to the required format. For example, use unix_timestamp to convert $[yyyymmddhh24miss] to a Unix timestamp.
Configure the assignment node's output as a context parameter. For details, see Configure node context.
In the downstream batch synchronization node, reference the context parameter by name to use the converted value.

Use the assignment node pattern for scalar values such as timestamps and date strings.