This topic describes how to synchronize data from Kafka to Hologres in real time.
Background information
Kafka is a distributed message queue service with high throughput and scalability. It is widely used in scenarios such as log collection, monitoring data aggregation, stream data processing, and online and batch analytics. Hologres seamlessly integrates with the big data ecosystem. You can synchronize data from Kafka to Hologres in real time using various methods, such as DataWorks data integration and the Hologres Connector. Then, you can perform high-concurrency, low-latency query and analysis on the data. For more information about the underlying principles, see Kafka Reader and Hologres Writer.
Synchronize data from Kafka in real time using DataWorks
DataWorks data integration uses the Java SDK for Kafka to read data from Kafka and synchronize it to Hologres in real time.
For more information about DataWorks, see DataWorks overview.
Configure data sources
Before you synchronize data, configure Kafka as the source data source and Hologres as the destination data source. For more information, see the following topics:
Configure a sync task
After you configure the data sources, configure a sync task to synchronize data from Kafka to Hologres in real time. For more information, see the following topics:
Query data
After the data is synchronized, you can query the data in Hologres.
Write data from Kafka using a Hologres connector
You can also directly write data from Kafka to Hologres using the Hologres connector. For more information, see Write to Hologres from Kafka.