Use cases

Ingest heterogeneous data from multiple sources and route it to downstream big data systems

DataHub ingests heterogeneous data in real time from sources such as applications, websites, Internet of Things (IoT) devices, and databases. It centralizes data management and routes data to downstream systems for analysis and archiving, creating a reliable data pipeline.

Key benefits

• System decoupling

  Decouple your big data systems from business systems, and decouple components within the big data system itself — making each layer independently maintainable and scalable.

• Real-time data channel

  DataHub streams business data into your big data system with minimal latency, shortening the data analytics cycle.

Cleanse and normalize heterogeneous data from multiple sources in real time

DataHub and Realtime Compute work together to cleanse heterogeneous data from multiple sources and transform it into unified structured data in real time — ready for downstream analysis without additional preprocessing.

Key benefits

• Real-time extract, transform, and load (ETL)

  Connect to multiple data sources and run continuous cleansing, filtering, association, and transformation pipelines — producing structured data ready for downstream consumption.

• Sub-second analytics

  Generate business metrics in sub-seconds, capturing the value of time-sensitive data before it goes stale.

Replace a traditional Lambda architecture with DataHub to build a real-time data warehouse

Transition from a Lambda architecture to a Kappa architecture by using DataHub to build a raw data layer, a real-time detail layer, and a real-time summary layer — all in a single unified pipeline.

Key benefits

• Unified Kappa architecture

  Consolidate the two separate pipelines of a traditional Lambda architecture into one, significantly reducing development and maintenance overhead.

• Real-time big data

  A real-time data warehouse enables real-time processing across your entire big data system — powering business intelligence (BI), operational reporting, and user-tag-based recommendations with up-to-the-second data.