LogHub Reader reads data from LogHub topics you specified in real time by using the Loghub SDK and supports shard merge and split. After shards are merged or split, duplicate data records may exist but no data is lost.
- Go to the DataStudio page.
- Log on to the DataWorks console.
- In the left-side navigation pane, click Workspaces.
- Select the region where the required workspace resides, find the workspace, and then click Data Analytics.
- Move the pointer over the icon and choose .Alternatively, you can click the required workflow, right-click Data Integration, and then choose .
- In the Create Node dialog box, set the Node Name and Location parameters.Notice The node name must be 1 to 128 characters in length. It can contain letters, digits, underscores (_), and periods (.).
- Click Commit.
- On the configuration tab of the real-time sync node, drag Loghub under to the canvas on the right.
- Click the new LogHub node. In the configuration pane that appears, set the required parameters in the
Node configuration section.
Parameter Description Data source The connection to the LogHub data store. In this example, you can select only a LogHub connection.
If no connection is available, click New data source on the right to create one on the page. For more information, see Configure a LogHub connection.
Logstore The name of the Logstore from which data is read in LogHub. You can click Data preview on the right to preview the selected Logstore. Advanced Configuration Specifies whether to split data in the Logstore. If you select Split for Split tasks, you must set Split rules. Output field The fields from which data is read.
- Click the icon in the toolbar.