All Products
Search

This Product

    Dataphin:Configure OceanBase Input Component

    Document Center

    Dataphin:Configure OceanBase Input Component

    Last Updated:Jun 23, 2026

    The OceanBase input component reads data from an OceanBase data source. To synchronize data from OceanBase to other data sources, first configure the source information in the OceanBase input component, and then configure the target data source.

    Prerequisites

    Procedure

    1. Select Development > Data Integration from the top menu bar on the Dataphin home page.

    2. Choose Project from the top menu bar on the integration page (Dev-Prod mode requires selecting the environment).

    3. In the left-side navigation pane, click Batch Pipeline. From the Batch Pipeline list, select the offline pipeline you want to develop to access its configuration page.

    4. Click on the Component Library at the top right of the page to access the Component Library panel.

    5. In the Component Library panel's left-side navigation pane, select Input. Locate the OceanBase component in the list on the right and drag it onto the canvas.

    6. To configure the OceanBase input component, click the image icon on the component card, which opens the OceanBase Input Configuration dialog box.

    7. Configure the necessary parameters in the Oceanbase Input Configuration dialog box.

      Parameter

      Description

      Step Name

      The name of the OceanBase input component. Dataphin generates a default step name, which you can modify as needed. Naming rules:

      • Can only contain Chinese characters, letters, underscores (_), and numbers.

      • Cannot exceed 64 characters.

      Datasource

      Select an OceanBase data source. The drop-down list displays all OceanBase data sources in the current Dataphin instance, including those you have read-through permission for and those you do not. Click the image icon to copy the data source name.

      Table

      Select the source table for data synchronization. You can search by keyword or enter the exact table name and click Precise Search. After you select a table, the system automatically detects the table status. Click the image icon to copy the table name.

      Shard Key (Optional)

      Splits data into shards based on the specified shard key field, enabling concurrent reading when combined with the concurrency setting. You can use any column in the source table as the shard key. We recommend that you use a primary key or indexed column to optimize transmission performance.

      Important

      For date-time type shard keys, the system performs brute-force sharding based on the maximum and minimum values across the total time range and concurrency. Even distribution is not guaranteed.

      Batch Read Count (Optional)

      The number of records read per batch. Configure a batch size (such as 1024 records) to reduce interactions with the data source, improve I/O efficiency, and lower network latency.

      Input Filter (Optional)

      Specify filter conditions for data extraction:

      • Configure Static Field: Extract data that matches a fixed value, such as ds=20210101.

      • Configure Variable Parameter: Extract data using a variable, such as ds=${bizdate}.

      Output Fields

      Displays all fields that match the selected table and filter conditions. You can perform the following operations:

      • Field Management: To exclude certain fields from downstream components, delete them:

        • Single Field Deletion Scenario: To delete a few fields, click the sgaga icon in the operation column.

        • Batch Field Deletion Scenario: To delete many fields at once, click Field Management, select the fields in the Field Management dialog box, click the image shift left icon to move them to the unselected list, and click Confirm.

          image..png

      • Batch addition: Click Batch Addition to configure fields in bulk using JSON, TEXT, or DDL format.

        • Batch configuration in JSON format, for example:

          // Example:
  [{
     "index": 0,
     "name": "id",
     "type": "int(10)",
     "mapType": "Long",
     "comment": "comment1"
   },
   {
     "index": 1,
     "name": "user_name",
     "type": "varchar(255)",
     "mapType": "String",
     "comment": "comment2"
 }]
          Note

          Index specifies the column number. Name specifies the field name after import. Type specifies the field type after import. For example, "index":3,"name":"user_id","type":"String" imports the fourth column with the field name user_id and field type String.

        • Batch configuration in TEXT format, for example:

          // Example:
0,id,int(10),Long,comment1
1,user_name,varchar(255),Long,comment2

          • The row delimiter separates each field entry. The default is line feed (\n). Supported delimiters: line feed (\n), semicolon (;), and period (.).

          • The column delimiter separates the field name and field type. The default is a comma (,).

        • Batch configuration in DDL format, for example:

          CREATE TABLE tablename (
	user_id serial,
	username VARCHAR(50),
	password VARCHAR(50),
	email VARCHAR (255),
	created_on TIMESTAMP,
);

      • Create Output Field: Click +create Output Field, fill in Column, Type, Comment, and select Mapping Type according to the page prompts.

      Note

      • Field types in pipeline development are automatically mapped by Dataphin based on source field conditions. For manually added fields, set the field type in Type(Length) format, such as int(10).

      • If the selected data source has network interconnection with the Dataphin cluster, output fields can only be deleted, not edited.

    8. Click Confirm to save the Oceanbase Input Component configuration.