This topic describes how to use the Data Integration service of DataWorks to synchronize data from Kafka to Hologres in real time.
Prerequisites
- A Hologres instance is purchased and connected to HoloWeb. For more information, see HoloWeb quick start.
- DataWorks is activated. For more information, see Overview.
- The Kafka environment and data are prepared. For more information, see Overview.
Background information
Kafka is a distributed message queue service that features high throughput and scalability. It is widely used in scenarios such as log collection, monitoring data aggregation, streaming data processing, and online and offline analysis. Hologres seamlessly integrates with the big data ecosystem and DataWorks. You can use the Data Integration service of DataWorks to synchronize data from Kafka to Hologres in real time, and then query, analyze, and process the data with high concurrency and low latency. For more information, see Kafka Reader and Hologres Writer.
Synchronize data from a single table in real time
The Data Integration service of DataWorks reads data from Kafka by using Kafka SDK for Java and synchronizes the data to Hologres in real time.
Use Hologres Connector to import data from Kafka to Hologres
You can also import data from Kafka to Hologres by using Hologres Connector. For more information, see hologres-connector-kafka.