You can obtain only time data by configuring scheduling parameters for nodes. After
you configure scheduling parameters for specific nodes, such as batch synchronization
nodes, the nodes cannot use the return values of the scheduling parameters unless
you process the return values by using methods such as functions. This topic describes
how to process the return values of scheduling parameters in typical scenarios.
Background information
The time data that scheduling parameters support is not applicable to all scenarios.
If you need time data in special formats based on your business requirements, you
can use engine functions to process the time data. You cannot use functions to process
the return values of the scheduling parameters for some nodes. In this case, you can
use assignment nodes to process the return values of the scheduling parameters.
You can process the return values of scheduling parameters by using functions or assignment
nodes in the following scenarios:
- Use functions
In this topic, an ODPS SQL node is used. You can process the return values of scheduling
parameters by using functions in the following scenarios:
- Use assignment nodes
For specific nodes, you cannot use functions to process the return values of scheduling
parameters. If such a node requires timestamps or time data in another format, you
can use assignment nodes to convert the return values of scheduling parameters to
the required format, and pass the processed values to the node. For more information
about how to use assignment nodes, see Configure an assignment node.
For example, if a batch synchronization node needs to use timestamp data for incremental
synchronization, you can convert time data to timestamp data by using a function defined
for an assignment node, and pass the timestamp data to the batch synchronization node.
For more information about operations related to scheduling parameters, see the following
topics:
Obtain the last day of the previous month.
The following table describes how to configure a scheduling parameter to obtain the
last day of the previous month.
| Parameter configuration |
Scheduled time for testing |
Returned result |
- Assignment:
last_month=$[yyyy-mm]
- Syntax to process the return value:
SELECT REPLACE(DATEADD(date'${last_month}-01',-1,'dd'),'-','');
- Format of the returned result:
yyyymmdd
|
20210926000000 |
20210831 |
Obtain the quarter to which the scheduled time belongs
The following table describes how to configure a scheduling parameter to obtain the
quarter to which the scheduled time belongs.
| Parameter configuration |
Scheduled time for testing |
Returned result |
- Assignment:
month=$[mm]
- Syntax to process the return value:
SELECT CEIL(INT('${month}')/3);
- Type of the returned result: positive integer
|
20211025001700 |
4 |
Obtain the year, month, day, hour, and minute of the time that is 15 minutes before
the scheduled time
The following table describes how to configure scheduling parameters to obtain the
year, month, day, hour, and minute of the time that is 15 minutes before the scheduled
time.
| Parameter configuration |
Scheduled time for testing |
Returned result |
- Assignment:
year=$[yyyy-15/24/60]month=$[yyyymm-15/24/60]day=$[yyyymmdd-15/24/60]hour=$[hh24-15/24/60]
- Syntax to process the return value:
select 'year=${year} month=${month} day=${day} hour=${hour} mi=${mi}';
- Format of the returned result:
- Year:
yyyy
- Month:
yyyymm
- Day:
yyyymmdd
- Hour:
hh
- Minute:
mm
|
20210727000500 |
year=2021month=202107day=20210726hour=23mi=50
|
Obtain the interval for data synchronization with the interval for periodic scheduling
being one day
Obtain the interval for data synchronization between
00:00:00 of the previous day and
00:00:00 of the current day. The interval for periodic scheduling is one day and accurate
to seconds, in the format of
yyyymmddhh24miss.
Note If you create a batch synchronization node to synchronize data generated in a Kafka
or LogHub data source in a specified interval and you need to configure the scheduling
parameters, you must set the data timestamp in the format of
yyyymmddhh24miss, and the interval must be left-closed and right-open. For more information, see
Kafka Reader and
LogHub Reader. For more information about incremental synchronization in different scenarios, see
Synchronize incremental data.
The following table describes how to configure scheduling parameters to obtain the
interval for data synchronization.
| Parameter configuration |
Scheduled time for testing |
Returned result |
- Assignment:
beginDateTime=$[yyyymmdd-1]endDateTime=$[yyyymmdd]
- Syntax to process the return value:
select '${beginDateTime}000000 ${endDateTime}000000';
- Format of the returned result:
yyyymmddhh24miss
|
20220117002300 |
2022011600000020220117000000
|
Obtain the interval for data synchronization with the interval for periodic scheduling
being one hour
Obtain the interval for data synchronization between
00:00:00 of the previous day and
00:00:00 of the current day. The interval for periodic scheduling is one hour and accurate
to seconds, in the format of
yyyymmddhh24miss.
Note If you create a batch synchronization node to synchronize data generated in a Kafka
or LogHub data source in a specified interval and you need to configure the scheduling
parameters, you must set the data timestamp in the format of
yyyymmddhh24miss, and the interval must be left-closed and right-open. For more information, see
Kafka Reader and
LogHub Reader. For more information about incremental synchronization in different scenarios, see
Synchronize incremental data.
The following table describes how to configure scheduling parameters to obtain the
interval for data synchronization.
| Parameter configuration |
Scheduled time for testing |
Returned result |
- Assignment:
beginDateTime=$[yyyymmddhh24-1/24]endDateTime=$[yyyymmddhh24]
- Syntax to process the return value:
select '${beginDateTime}0000 ${endDateTime}0000';
- Format of the returned result:
yyyymmddhh24miss
|
20220117002300 |
2022011623000020220117000000
|