Task orchestration overview
Task orchestration in Data Management (DMS) lets you build task flows from one or more task nodes and run them on timed or event-based schedules to automate complex data workflows.
Background information
As data volumes grow and sources diversify, manual data processing no longer scales. Automated workflows are essential to ensure correct, repeatable data processing, analysis, and backup.
Task orchestration in DMS automates these processes, reduces errors, improves data development efficiency, and enhances data value and reliability.
Supported database types
-
Relational databases
-
MySQL: RDS MySQL, PolarDB for MySQL, MyBase MySQL, PolarDB Distributed Edition, and other MySQL sources
-
SQL Server: RDS SQL Server, MyBase SQL Server, and SQL Server from other sources
-
PostgreSQL: RDS PostgreSQL, PolarDB for PostgreSQL, MyBase PostgreSQL, and PostgreSQL from other sources
-
OceanBase: OceanBase MySQL mode, OceanBase Oracle mode, or self-managed OceanBase
-
PolarDB for PostgreSQL (Compatible with Oracle)
-
Oracle
-
Dameng Database
-
DB2
-
-
NoSQL: Lindorm
-
Data warehouses
-
AnalyticDB for MySQL
-
AnalyticDB for PostgreSQL
-
DLA
-
MaxCompute
-
Hologres
-
-
Object Storage Service: OSS
Job types
|
Category |
Task type |
Description |
References |
|
Data manipulation |
Single-instance SQL |
Runs an SQL statement on a specified relational database. Note
If lock-free schema evolution is enabled for the target instance, the single-instance SQL task uses it to prevent table locks. Enable lock-free schema evolution. |
None |
|
General |
Single-instance SQL assignment |
Assigns SELECT query results to an output variable for use in the next node. |
|
|
Conditional branch |
Evaluates a conditional expression in a task workflow. If the expression is true, subsequent tasks run; otherwise, they are skipped. |
||
|
ECS remote command |
Runs Shell, PowerShell, or Bat commands on a remote ECS instance through Cloud Assistant. |
||
|
Status check |
Table time column status check |
Checks whether new data exists in the target table after a specified time. |
None |
|
Audit task |
Runs automated data quality checks on a schedule based on predefined quality rules and generates reports. |
None |
|
|
Task workflow dependency check |
Manages dependencies within a workflow or between workflows. The dependency target can be an entire workflow or a specific task node. |