You can create DataConnectors to synchronize DataHub data in real time or quasi-real time to other Alibaba Cloud services, such as MaxCompute, Object Storage Service (OSS), Elasticsearch, ApsaraDB RDS for MySQL, AnalyticDB for MySQL, and Tablestore. You can configure DataConnectors so that the data you write to DataHub can be used in other Alibaba Cloud services.
You can create DataConnectors to synchronize DataHub data in real time or quasi-real time to other Alibaba Cloud services, such as MaxCompute, OSS, Elasticsearch, ApsaraDB RDS for MySQL, AnalyticDB for MySQL, and Tablestore. You can configure DataConnectors so that the data you write to DataHub can be used in other Alibaba Cloud services.
Usage notes:
All DataConnectors support data synchronization between Alibaba Cloud services only in the same region, and do not support data synchronization across regions.
You must enter the internal endpoint of the destination service on the classic network when you create a DataConnector. You cannot use the public endpoint for synchronization.
Data synchronization supports only at-least-once semantics. This ensures that no data is lost, but may result in duplicate records in the destination service if an error such as a network exception occurs during synchronization. In this case, you need to remove duplicate records after data synchronization.
Supported destination services
Destination service | Real-time performance | Support VPCs | Description |
|---|---|---|---|
MaxCompute | Quasi-real time. The latency is no more than 5 minutes. | No | The field names and data types in the source topic must be the same as those in MaxCompute. The source topic must contain one or more partition fields in the MaxCompute table. |
OSS | Real time | No | Data is synchronized to the specified bucket in OSS and are saved as CSV objects. |
Elasticsearch | Real time | Yes | Data is synchronized to the specified index in Elasticsearch. The data may not be synchronized in the order of the record timestamps. If you want to synchronize data in the order of the record timestamps, you must write the data with the same ID into the same shard. |
ApsaraDB RDS for MySQL | Real time | Yes | Data is synchronized to the specified table in ApsaraDB RDS for MySQL. |
AnalyticDB for MySQL | Real time | No | Data is synchronized to the specified table in AnalyticDB for MySQL. |
Tablestore | Real time | No | Data is synchronized to the specified table in Tablestore. |
View a DataConnector
After a DataConnector is created, go to the Connector tab and click View in the Actions column to view the synchronization details, including the synchronization offset and synchronization status
