This topic describes how to configure a source database for an extract, transform, and load (ETL) task.

Prerequisites

  • An ETL task is created in the China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Shenzhen), or China (Guangzhou) region.
  • A source database is created.
  • The source database belongs to one of the following types: self-managed MySQL databases, ApsaraDB RDS for MySQL, PolarDB for MySQL, PolarDB-X V1.0 (formerly DRDS), self-managed Oracle databases, self-managed PostgreSQL databases, ApsaraDB RDS for PostgreSQL, Db2 for LUW, Db2 for i, and PolarDB for PostgreSQL.

Procedure

Note In this example, a self-managed MySQL database is used.
  1. Go to the ETL page.
    Note

    You can also perform the following steps to configure an ETL task in the Data Management (DMS) console:

    • Go to the DMS console.
    • In the top navigation bar, click DTS. Then, in the left-side navigation pane, choose Data integration > Streaming ETL.
    • Click Create Data Flow. In the Create Data Flow dialog box, specify an ETL task name in the Data Flow Name field and set the Development Method parameter to DAG.
    • Click OK.
  2. In the upper-left corner of the page, select the region where you want to create an ETL task.
  3. In the left-side navigation pane, click ETL.
  4. On the ETL page, click Create Task (Pay-as-you-go).
  5. On the Create Task page, set parameters for the ETL task.
  6. In the Input/Dimension Table section on the left side of the page, select MySQL and drag it to the canvas on the right side of the page.
  7. Click MySQL on the canvas.
  8. In the Input/Dimension Table: MySQL section, set parameters for the source database.
    1. On the Node Settings tab, set the required parameters.
      Data Source_Node Settings
      Parameter Description
      Data Source Name DTS automatically generates a task name. We recommend that you specify a descriptive name that makes it easy to identify. You do not need to use a unique task name.
      Region Select the region where the source database resides.
      Database Connection Template Select the name of the template that stores the connection settings of the source database. You can also click Create Template to create a connection template. For more information, see Create a connection template.
      Note For more information about how to create a connection template in the Data Management (DMS) console, see Create a connection template.
      Node Type Select a node type.
      • Stream Table
      • Dimension Table
      Select Databases and Tables Select the databases and tables that you want to transform.
      Note After you select the databases and tables, you are redirected to the Output Fields tab.
    2. Select the column names based on your needs.
      Data Source_Output Fields
    3. Optional: If you select Stream Table for the node type, click the Time Attributes tab and set the required parameters.
      Data Source_Time Attributes
      Parameter Description
      Event Time Watermark Select the field that represents the time when the event is generated.
      Latency of Event Time Watermark Enter the latency for the event to be generated.
      Processing Time Select the field that represents the processing time of the event.

Result

After a source database is configured, the Configured icon appears on the right side of the source database.

What to do next

Configure transformation components