This topic describes how to configure a source database for an extract, transform, and load (ETL) task.

Prerequisites

  • An ETL task is created in the China (Hangzhou), China (Beijing), or China (Zhangjiakou) region.
  • The source database belongs to one of the following types: self-managed MySQL databases, ApsaraDB RDS for MySQL, PolarDB for MySQL, PolarDB-X V1.0 (formerly DRDS), self-managed Oracle databases, self-managed PostgreSQL databases, ApsaraDB RDS for PostgreSQL, Db2 for LUW, Db2 for i, and .
  • A source database is created.

Procedure

Note In this example, the source database is a self-managed MySQL database.
  1. Go to the ETL page.
    Note

    You can also perform the following steps to configure an ETL task in the DMS console:

    • Log on to the DMS console.
    • In the top navigation bar, click DTS.
    • In the left-side navigation pane, click Streaming ETL.
    • Click Create Data Flow in the upper-left corner. In the Create Data Flow dialog box, specify an ETL task name in the Data Flow Name field, set Processing Method to Stream Processing, and set Development Method to DAG Development.
    • Click OK.
  2. In the upper-left corner of the page, select the region where you create the ETL task.
    Note You can create an ETL task only in the China (Hangzhou), China (Beijing), or China (Zhangjiakou) region. Select a region based on the actual scenario.
  3. In the left-side navigation pane, click ETL.
  4. On the ETL page, click Create Task (Pay-as-you-go).
  5. On the Create Task page, set parameters for the ETL task.
  6. In the Input/Dimension Table section on the left side of the page, select MySQL and drag it to the canvas on the right side of the page.
  7. Click MySQL on the canvas.
  8. In the Input/Dimension Table: MySQL section, set parameters for the source database.
    1. On the Node Settings tab, set the required parameters.
      Data Source_Node Settings
      Parameter Description
      Data Source Name DTS automatically generates a name. We recommend that you specify an informative name for easy identification. You do not need to use a unique name.
      Region Select the region where the source database resides.
      Database Connection Template Select the name of the template that stores the connection settings of the source database. You can also click Create Template to create a connection template. For more information, see Create a connection template.
      Node Type Select a node type.
      • Stream Table
      • Dimension Table
      Select Databases and Tables Select the databases and tables that you want to transform.
      Note After you select the databases and tables, you are redirected to the Output Fields tab.
    2. Select the column names based on your needs.
      Data Source_Output Fields
    3. Optional:If you select Stream Table as the node type, click the Time Attributes tab and set the parameters.
      Data Source_Time Attributes
      Parameter Description
      Select Event Time Watermark Select the field that represents the time when the event is generated.
      Latency of Event Time Watermark Enter the latency of event time processing.
      Processing Time Enter the field that represents the processing time of the event.

Result

After a source database is configured, the Configured icon appears on the right side of the source database.

What to do next

Configure the transformation components