All Products
Search

This Product

    Dataphin:Configure the AnalyticDB for PostgreSQL Input Component

    Document Center

    Dataphin:Configure the AnalyticDB for PostgreSQL Input Component

    Last Updated:Mar 05, 2026

    The AnalyticDB for PostgreSQL input component reads data from an AnalyticDB for PostgreSQL data source. When synchronizing data from an AnalyticDB for PostgreSQL data source to another data source, you must first configure the source data source information in the AnalyticDB for PostgreSQL input component. Then, configure the destination data source for data synchronization. This topic describes how to configure the AnalyticDB for PostgreSQL input component.

    Prerequisites

    Procedure

    1. In the top menu bar of the Dataphin homepage, choose Development > Data Integration.

    2. In the top menu bar of the integration page, select a project. In Dev-Prod mode, also select an environment.

    3. In the navigation pane on the left, click Offline Integration. In the Offline Integration list, click the offline pipeline you want to develop to open its configuration page.

    4. Click Component Library in the upper-right corner of the page to open the Component Library panel.

    5. In the navigation pane on the left of the Component Library panel, select Input. In the input component list on the right, find the AnalyticDB for PostgreSQL component and drag it onto the canvas.

    6. Click the image icon on the AnalyticDB for PostgreSQL input component card to open the AnalyticDB for PostgreSQL Input Configuration dialog box.

    7. In the AnalyticDB for PostgreSQL Input Configuration dialog box, configure the following parameters.

      Parameter

      Description

      Step Name

      The name of the AnalyticDB for PostgreSQL input component. Dataphin automatically generates a step name, but you can modify it based on your business scenario. The naming convention is as follows:

      • Can contain only Chinese characters, letters, underscores (_), and digits.

      • Must be no longer than 64 characters.

      Datasource

      The drop-down list displays all AnalyticDB for PostgreSQL data sources and project-level resources available in the current Dataphin instance, including those for which you do or do not have read-through permission. Click the image icon to copy the current data source name.

      For data sources without read-through permission, click Request next to the data source to apply for read-through permission. For more information, see Request data source permissions.

      If you do not have an AnalyticDB for PostgreSQL data source, click Create Data Source to create one. For more information, see Create an AnalyticDB for PostgreSQL data source.

      Time Zone

      Data in time format is processed based on the current time zone. By default, this is the time zone configured for the selected data source and cannot be changed.

      Note

      For tasks created before version V5.1.2, you can choose either Data Source Default Configuration or Channel Configuration Time Zone. The default selection is Channel Configuration Time Zone.

      • Data Source Default Configuration: The default time zone of the selected data source.

      • Channel Configuration Time Zone: The time zone configured under Properties > Channel Configuration for the current integration task.

      Schema (optional)

      Select a schema to enable cross-schema table selection. If not specified, the schema configured in the data source is used by default.

      Source Table Quantity

      Select the number of source tables. Options include Single Table and Multiple Tables:

      • Single Table: Use this option when synchronizing data from one source table to one destination table.

      • Multiple Tables: Use this option when synchronizing data from multiple source tables into the same destination table. The union algorithm combines data from multiple tables into one table.

      Table Matching Method

      Choose between General Rule and Database Regex.

      Note

      This parameter is configurable only when Source Table Quantity is set to Multiple Tables.

      Table

      Select the source table(s):

      • If Source Table Quantity is set to Single Table, enter a keyword to search for tables or enter the exact table name and click Exact Search. After selecting a table, the system automatically checks its status. Click the image icon to copy the selected table name.

      • If Source Table Quantity is set to Multiple Tables, enter different expressions based on the table matching method to add tables.

        • If Table Matching Method is set to General Rule, enter an expression in the input box to filter tables with identical structures. The system supports enumeration, regex-like patterns, or a mix of both. Example: table_[001-100];table_102;.

        • If Table Matching Method is set to Database Regex, enter a regular expression supported by the current database. The system matches tables in the destination database based on this regex. At runtime, the system dynamically matches new tables that fit the regex pattern for synchronization.

        After entering the expression, click Exact Search to view the matched table list in the Confirm Matching Details dialog box.

      Split Key (optional)

      The system uses the configured split key field for data partitioning. This works with concurrency settings to enable concurrent reading. You can use any column from the source table as the split key. For optimal performance, use a primary key or an indexed column as the split key.

      Important

      When using a datetime type, the system identifies the maximum and minimum values and performs brute-force splitting based on the total time range and concurrency level. This method does not guarantee even distribution.

      Batch Read Size (optional)

      The number of records read in a single batch. Instead of reading one record at a time, configure a specific batch size (for example, 1024 records) to reduce interactions with the data source, improve I/O efficiency, and lower network latency.

      Input Filter (optional)

      Configure the filtering conditions to extract data. For detailed configuration instructions, see the following:

      • Use a static value to extract matching data, such as ds=20210101.

      • Use a variable parameter to extract partial data, such as ds=${bizdate}.

      Output Fields

      Note

      When the tenant's compute engine is AnalyticDB for PostgreSQL, the output fields of the AnalyticDB for PostgreSQL input component support classification and categorization. This feature is unavailable for non-AnalyticDB for PostgreSQL compute engines.

      The output fields section displays all fields from the selected table(s) that match the filter conditions. You can perform the following operations:

      • Field Management: To exclude certain fields from being passed downstream, delete them:

        • Delete a single field: To remove a few fields, click the sgaga icon in the Actions column to delete each unwanted field.

        • Batch field deletion scenario: To delete many fields, you can click Field Management, select multiple fields in the Field Management dialog box, click the image left-moving icon to move the selected input fields to the unselected input fields list, and click OK to complete batch field deletion.

          image..png

      • Bulk Add: Click Bulk Add to configure fields in JSON, TEXT, or DDL format.

        Note

        After bulk adding fields and clicking OK, the existing field configuration will be overwritten.

        • JSON format example:

          // Example:
  [{
     "index": 1,
     "name": "id",
     "type": "int(10)",
     "mapType": "Long",
     "comment": "comment1"
   },
   {
     "index": 2,
     "name": "user_name",
     "type": "varchar(255)",
     "mapType": "String",
     "comment": "comment2"
 }]
          Note

          The index indicates the column number of the specified object, the name indicates the field name after import, and the type defines the field type after import. For example, "index":3,"name":"user_id","type":"String" imports the fourth column in the file, with the field name user_id and the field type String.

        • TEXT format example:

          // Example:
1,id,int(10),Long,comment1
2,user_name,varchar(255),Long,comment2

          • Row delimiters separate field entries. The default is line feed (\n), but you can also use semicolon (;) or period (.).

          • Column delimiters separate field names and types. The default is comma (,), and you can use ','. Field types are optional and default to ','.

        • Configure multiple objects in batch using the DDL format. For example:

          CREATE TABLE tablename (
	user_id serial,
	username VARCHAR(50),
	password VARCHAR(50),
	email VARCHAR (255),
	created_on TIMESTAMP,
);

      • Create Output Field: Click + Create Output Field and fill in Column, Type, Comment, and select a Mapping Type. After completing the configuration, click the image icon to save.

    8. Click OK to complete the property configuration for the AnalyticDB for PostgreSQL input component.