All Products
Search

This Product

    Dataphin:Configure the Greenplum input component

    Document Center

    Dataphin:Configure the Greenplum input component

    Last Updated:Jun 04, 2026

    The Greenplum input component reads data from a Greenplum data source for synchronization to other data sources. Configure this component to specify the source, then configure the destination.

    Prerequisites

    Procedure

    1. On the top menu bar of the Dataphin home page, choose Developer > Data Integration.

    2. On the top menu bar, select a Project. In Dev-Prod mode, also select an environment.

    3. In the left navigation pane, click Offline Integration. In the Offline Integration list, click the target offline pipeline to open its configuration page.

    4. In the upper-right corner, click Component Library to open the Component Library panel.

    5. In the Component Library panel, select Input, find Greenplum, and drag it to the canvas.

    6. Click the image icon on the Greenplum input component to open the Greenplum Input Configuration dialog box.

    7. In the Greenplum Input Configuration dialog box, configure the parameters.

      Parameter

      Description

      Step Name

      The component name. Dataphin auto-generates a name, which you can change. Naming rules:

      • Can contain only Chinese characters, letters, underscores (_), and digits.

      • Cannot exceed 64 characters.

      Datasource

      Lists all Greenplum data sources in Dataphin, including those you lack read-through permissions for.

      Schema

      Select the schema containing the source table. Cross-schema reads are supported. If the connection specifies a schema, it is selected by default. You can select a different schema that you have permissions for.

      Source Table Quantity

      Select the number of source tables:

      • Single table: Synchronize from one source table to one destination table.

      • Multiple tables: Synchronize from multiple tables to one destination table. Supports enumeration, regex-like patterns, or both, such as table_[001-100];table_102.

      Table Match Mode

      Select either General Rule or Database Regex.

      Note

      This parameter is available only when Source Table Quantity is set to Multiple tables.

      Table

      Select the source table or tables:

      • For Single table: search by keyword or enter the exact name and click Exact Match. The system validates the selected table automatically. Click the image icon to copy the table name.

      • For Multiple tables: enter an expression based on the selected match mode.

        • General Rule: enter an expression to filter tables with the same structure. Supports enumeration, regex-like patterns, or both. Example: table_[001-100];table_102;.

        • Database Regex: enter a database-supported regular expression. At runtime, the node matches the latest set of tables and synchronizes them.

        Click Exact Match to preview matched tables in the Confirm Match Details dialog box.

      Split Key

      Select an integer column as the split key. Use the primary key or an indexed column for best performance. The system partitions data by this key for concurrent reads.

      Batch Read Size

      The number of records per read batch (for example, 1024). Larger batches reduce data source interactions and network overhead.

      Input Filter

      Filter conditions to extract specific data:

      • Static value: ds=20210101.

      • Variable: ds=${bizdate}.

      Output Fields

      Displays all fields from the selected tables. Supported operations:

      • Field management: Delete fields you do not need in downstream components:

        • Single: Click the sgaga icon in the Actions column.

        • Batch: Click Field Management. In the Field Management dialog box, select fields, click the image icon to move them to the unselected list, and click OK.

          image..png

      • Batch add: Click Batch Add to add fields in JSON, TEXT, or DDL format.

        Note

        Batch-added fields overwrite existing field configurations.

        • To configure in JSON format, for example:

          // Example:
  [{
     "index": 1,
     "name": "id",
     "type": "int(10)",
     "mapType": "Long",
     "comment": "comment1"
   },
   {
     "index": 2,
     "name": "user_name",
     "type": "varchar(255)",
     "mapType": "String",
     "comment": "comment2"
 }]
          Note

          index: column number, name: field name, type: field type. Example: "index":3,"name":"user_id","type":"String" imports the fourth column as user_id with type String.

        • To configure in TEXT format, for example:

          // Example:
1,id,int(10),Long,comment1
2,user_name,varchar(255),Long,comment2

          • Row delimiter: default is line feed (\n). Semicolons (;) and periods (.) are also supported.

          • Column delimiter: default is comma (,). Supports ','. Field types can be omitted; default is ','.

        • To configure in DDL format, for example:

          CREATE TABLE tablename (
	user_id serial,
	username VARCHAR(50),
	password VARCHAR(50),
	email VARCHAR (255),
	created_on TIMESTAMP,
);

      • Add output field: Click + Add Output Field, enter the Column, Type, and Comment, select the Mapping Type, and click the image icon to save.

    8. Click Confirm.