Alibaba Cloud Realtime Compute for Apache Flink is an enterprise-level, high-performance platform that is developed by Alibaba Cloud based on Apache Flink and enables real-time big data processing. Hologres is seamlessly integrated with Realtime Compute for Apache Flink. You can use Realtime Compute for Apache Flink to write data to Hologres and query the data in real time. This helps you build a real-time data warehouse for your enterprise.

Realtime Compute for Apache Flink does not store data. All data of Realtime Compute for Apache Flink comes from external storage systems. Realtime Compute for Apache Flink supports the following data storage types:
  • Source table

    Source tables contain data to be imported to Realtime Compute for Apache Flink. You can import data from source tables in batch mode instead of streaming mode. Before data is imported to Realtime Compute for Apache Flink, Hologres performs full table scans.

  • Dimension table

    Dimension tables are suitable for point query scenarios in which data is queried based on primary keys. We recommend that you use the row-oriented storage model for dimension tables. All fields in the primary key must be joined to a dimension table.

  • Result table

    Result tables are used to receive and store result data that is processed by Realtime Compute for Apache Flink and allow you to read or write data before the data is transmitted to downstream data stores.

Integrated with Hologres, Realtime Compute for Apache Flink also provides the following enterprise-level advanced features:
  • Hologres binary log

    You can consume the change logs of Hologres tables by using various consumption modes in Message Queue.

  • Hologres catalog

    Hologres catalogs are supported in Flink. You can read Hologres metadata in the console of fully managed Flink without the need to register Hologres tables. Hologres catalogs improve development efficiency and ensure schema accuracy.

  • Schema evolution

    Schema change is supported in fully managed Flink. When Flink reads JSON-formatted data, it automatically parses the data type and creates columns in the corresponding table. The schema can be in dynamic evolution.

The following table describes the Flink service types that are supported by Hologres and the features of these service types.
Service type Data storage Enterprise-level advanced feature Description
Source table Result table Dimension table Hologres binary log Hologres catalog Schema evolution
Semi-managed Flink Row-oriented storage and column-oriented storage are supported. Row-oriented storage and column-oriented storage are supported. Row-oriented storage is recommended. Supported. Supported. Supported. Uses the Amazon EMR Studio development platform.
Blink in exclusive mode (discontinued) Row-oriented storage and column-oriented storage are supported. Row-oriented storage and column-oriented storage are supported. Row-oriented storage is recommended. Hologres V0.8 supports only row-oriented storage. Hologres V0.9 and later support both row-oriented storage and column-oriented storage. Row-oriented storage is recommended. Not supported. Not supported.

Uses the Bayes development platform.

Fully managed Flink is recommended.

Open source Flink V1.10 Not supported. Row-oriented storage and column-oriented storage are supported. Not supported. Not supported. Not supported. Not supported. N/A
Open source Flink V1.11 and later Not supported. Row-oriented storage and column-oriented storage are supported. Row-oriented storage is recommended. Not supported. Not supported. Not supported. The source code of Hologres is available in open source Flink V1.11 and later. For more information, see alibabacloud-hologres-connectors.