All Products
Search

This Product

    Data Transmission Service:Configure an ETL task in DAG mode

    Document Center

    Data Transmission Service:Configure an ETL task in DAG mode

    Last Updated:Sep 16, 2026

    Data Transmission Service (DTS) provides an extract, transform, and load (ETL) feature that cleanses and transforms streaming data, delivering the data you need accurately and efficiently. This topic describes how to configure an ETL task in DAG mode (drag-and-drop visualization).

    Background information

    Note

    This feature is being deprecated and is available for free only to users who have previously used it. New users can no longer access this feature. We recommend that you configure ETL tasks within a data synchronization or data migration instance. For more information, see Configure an ETL task in a DTS data migration or synchronization task.

    • Before you configure an ETL task, review the following information:

      • Input/Dimension Table refers to the source database of the ETL task.

      • Output refers to the destination database to which the processed data is written after ETL processing.

    • DTS provides a streaming ETL feature for the data synchronization process. You can add various transformation components between the source database and the destination database to perform rich transformation operations, and then write the processed data to the destination database in real time. For example, you can join two stream tables to form a large table and write it to the destination database, or add a field to the source table, configure a function to assign a value to the field, and then write the transformed field to the destination database.

    Prerequisites

    • ETL tasks can be created only in the following regions: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Guangzhou), and China (Hong Kong).

    • The supported source database types include MySQL, PolarDB for MySQL, Oracle, PostgreSQL, DB2 for iSeries (AS/400), DB2 for LUW, PolarDB-X (Formerly DRDS) (PolarDB-X 1.0), PolarDB PostgreSQL, MariaDB, PolarDB for Oracle, SQL Server, and PolarDB-X 2.0.

    • The supported destination database types include MySQL, PolarDB for MySQL, Oracle, AnalyticDB for MySQL V3.0, PolarDB PostgreSQL, PostgreSQL, DB2 for LUW, DB2 for iSeries (AS/400), AnalyticDB for PostgreSQL, SQL Server, MariaDB, PolarDB-X (Formerly DRDS) (PolarDB-X 1.0), PolarDB for Oracle, and Tablestore.

    • The ETL feature does not support schema migration. You must create the corresponding table schemas in the destination database in advance based on the transformation conditions. For example, table A contains Field 1, Field 2, and Field 3, and table B contains Field 2, Field 3, and Field 4. If you join the two tables and want to output Field 2 and Field 3, you must create table C in the destination database with Field 2 and Field 3.

    • The ETL feature does not support full data synchronization. You can only perform real-time transformation on incremental data.

    Usage notes

    • All source and destination databases must reside in the same region.

    • All Stream Table must originate from the same instance.

    • The database name and the table name must be unique.

    • The transformation components have the following limits:

      • Table Join: Only join operations between two tables are supported.

      • Field Calculator and Table Record Filter: Only single-table structures are supported.

    Configuration flow

    ETL task configuration flow

    To perform ETL processing, you select the source databases you need (multiple source databases are supported), the transformation components, and the destination database to write to. The procedure for creating an ETL task is as follows:

    Task

    Description

    References

    Configure source database information

    You can select one or more data types for the Input/Dimension Table node. For each source database, configure the following information in sequence:

    Note

    You can select the same data type multiple times.

    1. Configure the node information.

    2. Configure the field information.

    3. Optional: If the table type you select is Stream Table, you must also configure the time attribute.

    Configure a source database

    Configure transformation components

    You can select one or more transformation components for the Transform node. For each transformation component, configure the following information in sequence:

    Note

    You can select the same transformation component multiple times.

    • Table Join

      1. Configure the node information.

      2. Configure the field information.

    • Field Calculator

      1. Configure the transformation name.

      2. Configure the calculator.

    • Table Record Filter

      1. Configure the transformation name.

      2. Configure the WHERE clause.

    Configure transformation components

    Configure destination database information

    You can select one destination database type for the Output node. For the destination database, configure the following information in sequence:

    1. Configure the node information.

    2. Configure the field mapping information.

    Configure destination database

    Procedure

    1. Go to the ETL task list page.

      1. Log on to the Data Transmission Service (DTS) console.

      2. In the left-side navigation pane, click ETL.

    2. In the upper-left corner, click Add data flow. In the Create Data Flow dialog box, enter a name for the ETL task in the Data Flow Name field, and set Development Method to DAG.

    3. Click Confirm.

    4. Configure the ETL task based on the on-screen instructions.

      1. Create the ETL task.

        Note

        • The following procedure uses an Input/Dimension Table (MySQL) node, a Field Calculator node, and an Output (MySQL) node as an example.

        • You can select one or more source databases. Source databases of the same type can be selected multiple times.

        • You can select one or more transformation components. The same transformation component can be selected multiple times.

        • You can select only one destination database.

        1. From the component panel on the left, drag an Input/Dimension Table (MySQL) node to a blank area of the canvas on the right.

        2. Click Input/Dimension Table MySQL-1 on the canvas and configure the node information, field information, and time attribute in sequence. For information about how to configure these settings, see Configure a source database.

        3. From the component panel on the left, drag a Field Calculator node to a blank area of the canvas on the right.

        4. Hover over the source node Input/Dimension Table MySQL-1, click the dot, and drag a connection line to connect the source node Input/Dimension Table MySQL-1 to Field Calculator-1.

        5. Click Field Calculator-1 on the canvas and configure the node information and field information in sequence. For information about how to configure these settings, see Configure transformation components.

        6. From the component panel on the left, drag an Output MySQL-1 node to a blank area of the canvas on the right.

        7. Hover over Field Calculator-1, click the dot, and drag a connection line to connect Field Calculator-1 to the destination node Output MySQL-1.

        8. On the Output MySQL-1 page, configure the node information and field mapping information in sequence. For information about how to configure these settings, see Configure destination database.

        9. After the configuration is complete, click Generate Flink SQL Validation or Publish as needed.

          • Generate Flink SQL Validation: If a Flink SQL validation succeeds. message appears at the top of the page, the Flink SQL has been generated successfully. If the validation fails, click View ETL Validation Details next to Generate Flink SQL Validation, fix the issues based on the details, and then run the Flink SQL validation again.

          • Publish: The system first runs the Flink SQL validation. Only after the Flink SQL validation succeeds can the precheck start. You can click View ETL Validation Details to view the ETL validation details.

      2. After the Flink SQL validation is complete, run the Precheck. Only after the precheck passes can DTS start the ETL task. If the precheck fails, click View Details next to the failed item, fix the issues based on the details, and then run the precheck again.

      3. After the precheck is complete, click Next: Purchase Instance at the bottom of the page.

      4. On the Purchase page, select the Instance Class and Compute Units, and select the Data Transmission Service (Pay-as-you-go) Service Terms and Public Preview Agreement checkboxes.

      5. Click Buy and Start. The ETL task starts.

        You can return to the ETL task list page to view the Task Status of the ETL task.