All Products
Search

This Product

    Dataphin:Configure the SelectDB Input Component

    Document Center

    Dataphin:Configure the SelectDB Input Component

    Last Updated:Mar 09, 2026

    The SelectDB input component reads data from a SelectDB data source. When you sync data from a SelectDB data source to another data source, first configure the SelectDB input component with the source data source information. Then configure the destination data source for the sync task. This topic explains how to configure the SelectDB input component.

    Prerequisites

    • You have created a SelectDB data source. For more information, see Create a SelectDB Data Source.

    • The account used to configure the SelectDB input component must have sync-read permission on the data source. If the account does not have this permission, request it. For more information, see Request Data Source Permissions.

    Procedure

    1. In the top menu bar on the Dataphin homepage, choose Develop > Data Integration.

    2. In the top menu bar of the Integration page, select a Project. In Dev-Prod mode, also select an environment.

    3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop. The configuration page for the pipeline opens.

    4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

    5. In the navigation pane on the left of the Component Library panel, click Input. In the input component list on the right, find the SelectDB component and drag it onto the canvas.

    6. Click the image icon in the SelectDB input component card to open the SelectDB Input Configuration dialog box.

    7. In the SelectDB Input Configuration dialog box, configure the parameters.

      Parameter

      Description

      Step Name

      The name of the SelectDB input component. Dataphin generates a step name automatically. You can change it based on your business scenario. Use the following naming rules:

      • Use only Chinese characters, letters, underscores (_), and digits.

      • Use no more than 64 characters.

      Datasource

      In the data source drop-down list, Dataphin displays all SelectDB data sources in the current Dataphin instance and indicates whether you have read-through permission for each.

      • If you do not have sync-read permission for a data source, click Request next to the data source to request the permission. For more information, see Request Data Source Permissions.

      • If you do not have a SelectDB data source, click Create Data Source to create one. For more information, see Create a SelectDB Data Source.

      Source Table Count

      Select the number of source tables. Options are Single Table and Multiple Tables:

      • Single Table: Use this option when syncing business data from one source table to one destination table.

      • Multiple Tables: Use this option when syncing business data from multiple source tables to one destination table. When writing data from multiple tables to one table, the system uses the union algorithm.

      Table Matching Method

      Choose Generic Rule or Database Regex.

      Note

      This parameter is available only when you select Multiple Tables for Source Table Count.

      Table

      Select the source table:

      • If you selected Single Table for Source Table Count, search by entering a keyword in the table name field. Or enter the exact table name and click Exact Match. After you select a table, the system automatically checks its status. Click the image icon to copy the name of the selected table.

      • If you selected Multiple Tables for Source Table Count, enter an expression based on the table matching method:

        • If you chose Generic Rule: Enter an expression in the field to filter tables with the same structure. The system supports enumeration, regex-like syntax, and mixed formats. For example: table_[001-100];table_102;.

        • If you chose Database Regex: Enter a regex supported by the database. The system matches tables in the destination database using this regex. At runtime, the task matches new tables dynamically based on the regex.

        After entering the expression, click Exact Match. In the Confirm Match Details dialog box, view the list of matched tables.

      Split Key (Optional)

      The number of records to read at a time. Configure a batch size—such as 1,024 records—to read data from the source database in batches instead of one record at a time. This reduces interactions with the data source, improves I/O efficiency, and lowers network latency.

      Batch Read Size (Optional)

      The number of records to read at a time. Configure a batch size—such as 1,024 records—to read data from the source database in batches instead of one record at a time. This reduces interactions with the data source, improves I/O efficiency, and lowers network latency.

      Input Filter (Optional)

      Set conditions to filter the data to extract. Configure as follows:

      • Set a static value to extract matching data. For example: ds=20210101.

      • Set a variable parameter to extract part of the data. For example: ds=${bizdate}.

      Output Fields

      The Output Fields section lists all fields from the selected table and filters. You can perform the following actions:

      • Manage Fields: Delete fields that you do not need to pass to downstream components:

        • Delete One Field: To delete a few fields, click the sgaga icon in the Actions column.

        • Delete Multiple Fields: To delete many fields, click Field Management. In the Field Management dialog box, select multiple fields. Click the image left arrow icon to move the selected fields to the unselected list. Click OK to complete the bulk deletion.

          image..png

      • Batch Add: Click Batch Add to add fields in JSON, TEXT, or DDL format.

        Note

        After you click OK, the batch-added fields overwrite any existing field configurations.

        • Configure settings in batches using JSON format, such as:

          // Example:
  [{
     "index": 1,
     "name": "id",
     "type": "int(10)",
     "mapType": "Long",
     "comment": "comment1"
   },
   {
     "index": 2,
     "name": "user_name",
     "type": "varchar(255)",
     "mapType": "String",
     "comment": "comment2"
 }]
          Note

          The index field indicates the column number of the specified object, the name field indicates the field name after import, and the type field indicates the field type after import. For example, "index":3,"name":"user_id","type":"String" imports the fourth column in the file, sets the field name to user_id, and sets the field type to String.

        • You can perform batch configurations in TEXT format. For example:

          // Example:
1,id,int(10),Long,comment1
2,user_name,varchar(255),Long,comment2

          • The row delimiter separates each field’s information. The default is a line feed (\n). You can also use a semicolon (;) or a period (.).

          • The column delimiter separates field names and types. The default is a comma (,). You can also use ','. The field type is optional and defaults to ','.

        • Batch configuration in DDL format, for example:

          CREATE TABLE tablename (
	user_id serial,
	username VARCHAR(50),
	password VARCHAR(50),
	email VARCHAR (255),
	created_on TIMESTAMP,
);

      • Add a New Output Field: Click +Add Output Field. Enter values for Column, Type, and Comment. Select a Mapping Type. Click the image icon to save the row.

    8. Click OK to complete the configuration of the SelectDB input component.