All Products
Search
Document Center

View a DataConnector

Last Updated: Aug 24, 2021

You can create DataConnectors to synchronize DataHub data in real time or quasi-real time to other Alibaba Cloud services, such as MaxCompute, Object Storage Service (OSS), Elasticsearch, ApsaraDB RDS for MySQL, AnalyticDB for MySQL, and Tablestore. You can configure DataConnectors so that the data you write to DataHub can be used in other Alibaba Cloud services.

You can create DataConnectors to synchronize DataHub data in real time or quasi-real time to other Alibaba Cloud services, such as MaxCompute, OSS, Elasticsearch, ApsaraDB RDS for MySQL, AnalyticDB for MySQL, and Tablestore. You can configure DataConnectors so that the data you write to DataHub can be used in other Alibaba Cloud services.

Usage notes:

  1. All DataConnectors support data synchronization between Alibaba Cloud services only in the same region, and do not support data synchronization across regions.

  2. You must enter the internal endpoint of the destination service on the classic network when you create a DataConnector. You cannot use the public endpoint for synchronization.

  3. Data synchronization supports only at-least-once semantics. This ensures that no data is lost, but may result in duplicate records in the destination service if an error such as a network exception occurs during synchronization. In this case, you need to remove duplicate records after data synchronization.

Supported destination services

Destination service

Real-time performance

Support VPCs

Description

MaxCompute

Quasi-real time. The latency is no more than 5 minutes.

No

The field names and data types in the source topic must be the same as those in MaxCompute. The source topic must contain one or more partition fields in the MaxCompute table.

OSS

Real time

No

Data is synchronized to the specified bucket in OSS and are saved as CSV objects.

Elasticsearch

Real time

Yes

Data is synchronized to the specified index in Elasticsearch. The data may not be synchronized in the order of the record timestamps. If you want to synchronize data in the order of the record timestamps, you must write the data with the same ID into the same shard.

ApsaraDB RDS for MySQL

Real time

Yes

Data is synchronized to the specified table in ApsaraDB RDS for MySQL.

AnalyticDB for MySQL

Real time

No

Data is synchronized to the specified table in AnalyticDB for MySQL.

Tablestore

Real time

No

Data is synchronized to the specified table in Tablestore.

View a DataConnector

After a DataConnector is created, go to the Connector tab and click View in the Actions column to view the synchronization details, including the synchronization offset and synchronization status

mod_1mod_5