All Products
Search

This Product

    DataWorks:Configure a DataHub source

    Document Center

    DataWorks:Configure a DataHub source

    Last Updated:Nov 14, 2025

    Real-time sync for single tables lets you capture data from DataHub. You can configure a real-time sync task by dragging and dropping components. This topic describes how to configure the DataHub source component in a real-time sync task.

    Features

    After a real-time sync task is started, it runs continuously. The task waits for data to become available in the source DataHub, reads the data, and then writes it to the destination. The DataHub source component has the following features:

    • Reads data in real time.

    • Performs concurrent reads based on the number of DataHub shards.

    Procedure

    1. Go to the DataStudio page.

      Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Development and O&M > Data Development. On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.

    2. In the Scheduled Workflow pane of the DataStudio page, move the pointer over the 新建 icon and choose Create Node > Data Integration > Real-time Synchronization.

      Alternatively, find the desired workflow in the Scheduled Workflow pane, right-click the workflow name, and then choose Create Node > Data Integration > Real-time Synchronization.

    3. In the Create Node dialog box, set the Sync Method parameter to End-to-end ETL and configure the Name and Path parameters.

    4. Click Confirm.

    5. On the edit page for the real-time sync node, click Source > DataHub and drag the component to the canvas.

    6. Click the DataHub node and configure the parameters in the Node Configuration dialog box.

      image

      Parameter

      Description

      Data Source

      Select a configured DataHub data source. Only DataHub data sources are supported.

      If no data source is configured, click Create Data Source on the right to go to the Workspace Configurations > Data Source page and create one. For more information, see Configure a DataHub data source.

      Topic

      Select the topic to sync from the current data source. Click Data Preview on the right to confirm your selection.

      Use Subscription Feature

      If you enable subscription mode, a Subscription ID is automatically created and used to subscribe to data from DataHub. This improves stability and performance. Do not delete a subscription ID that is in use on DataHub. Otherwise, the task will fail.

      Output Fields

      Select the fields to sync.

    7. Click the 保存 icon in the toolbar.