Dataphin's real-time computing engine lets you create tables for real-time data sources using real-time nodes. This topic describes the real-time data sources that each Dataphin real-time computing engine supports, including their read and write capabilities.
Real-time data sources supported by Ververica Flink
Storage class | Data source type | Metatable | Read | Write | Dimension table read |
Big data storage | MaxCompute | Support | Supported | Support | Supported |
Hologres | Support | Support | Support | - | |
Hudi | Supported | Support | Supported | - | |
StarRocks | Supported | Support | Supported | - | |
Lindorm (compute engine) | - | - | Support | Support | |
Relational database | PolarDB | - | Support | Support | - |
PolarDB-X (formerly DRDS) | Support | - | Help and support | Help and support | |
MySQL | Support | Supported (CDC) | Supported | Supported | |
PostgreSQL | Support | Supported | - | - | |
AnalyticDB for MySQL 3.0 | Support | - | Supported | Support | |
AnalyticDB for PostgreSQL | Supported | - | Support | Support | |
Oracle | - | - | Support | - | |
ClickHouse | Support | - | Support | - | |
Message queue | Log Service | Help and Support | Support | Support | - |
Kafka | Support | Support | Support | - | |
RocketMQ | Supported | Support | Support | Supported | |
NoSQL | Elasticsearch | Support | Support | Support | Support |
Aliyun HBase | Support | - | Support | Support | |
Tablestore | Support | - | Support | Supported | |
Redis | Support | - | Support | Support | |
Lindorm (wide table) | Support | - | Support | Support |
Real-time data sources supported by Alibaba Blink
Storage Class | Data source type | Read | Write | Dimension table read |
Big data storage | MaxCompute | Support | Support | Supported |
DataHub | Supported | Support | - | |
Hologres | Support | Supported | Support | |
Relational database | PolarDB-X (formerly DRDS, same as RDS) | - | Support | Supported |
AnalyticDB for MySQL 2.0 | - | Support | - | |
AnalyticDB for MySQL 3.0 | - | Supported | Supported | |
Message queue | Log Service (SLS/LOG_HUB) | Support | Support | - |
Kafka | Support | Supported | - | |
RocketMQ | Supported | Support | - | |
NoSQL | Elasticsearch | Support | Support | Unsupported |
Aliyun HBase | - | Support | Support | |
Tablestore | - | Support | Supported | |
Redis | - | Supported | Support |
Real-time data sources supported by Apache Flink
Storage class | Data source type | Metatable | Read | Write | Dimension table read |
Big data storage | Hive | Supported (only for CDH 6.x Hive 2.1.1, CDP 7.x Hive 3.1.3, and AsiaInfo DP 5.x Hive 3.1.0) | Support | Support | Supported |
Hive (Hudi) | Support | Supported | Support | - | |
Hive (Iceberg) | - | Supported (only native DDL mode) | Supported (only native DDL mode) | ||
Hive (Paimon) | Support | Support | Support | Supported | |
Hudi | Support | Support | Supported | - | |
Doris | Support | Supported (flink1.14 and flink1.15) | Supported (flink1.14 and flink1.15) | Support | |
SelectDB | - | - | Supported | - | |
Paimon | Support | Supported (Flink 1.14 and Flink 1.15) | Supported (Flink 1.14 and Flink 1.15) | - | |
StarRocks | Support | Support | Support | - | |
TDH Inceptor (9.3.x) | Supported | - | Supported | Supported | |
MySQL | Support | Supported (CDC) | Support | Support | |
SAP HANA | Support | Supported | - | - | |
Microsoft SQL Server | Support | Supported (CDC) | - | - | |
PostgreSQL | Support | Support | Support | Support | |
AnalyticDB for PostgreSQL | Supported | - | Support | - | |
OceanBase | Support | Supported (CDC) | |||
Oracle | Supported | Supported (CDC) | Supported | - | |
ClickHouse | Support | - | Supported (only Flink 1.15) | - | |
TiDB | Support | Supported (CDC) | - | - | |
GaussDB (DWS) | - | - | Support | Support | |
Message queue | Log Service | Support | Supported (only flink 1.15) | Supported (only flink 1.15) | - |
Kafka | Support | Supported | Supported | - | |
RabbitMQ | Support | Supported | Support | - | |
NoSQL | HBase 0.9.4 | - | - | Support | Support |
HBase 1.1.x | Support | - | Supported | Support | |
HBase 2.x | Support | - | Support | Support | |
Elasticsearch | Support | - | Support | - | |
Redis | Support | - | Supported (only Flink 1.15) | Supported (only Flink 1.15) | |
MongoDB | Support | Supported (CDC) | - | - | |
Semi-structured storage | SAP TABLE | - | Supported | - | - |
File | OSS | Supported | - | Supported | - |
Real-time data sources supported by FusionInsight Flink
Storage class | Data source type | Read | Write | Dimension table read |
Relational database | MySQL | Supported (CDC) | Supported (Java Database Connectivity (JDBC)) | Supported (JDBC) |
PostgreSQL | - | Supported (JDBC) | Supported (JDBC) | |
Oracle | - | Supported (JDBC) | Supported (JDBC) | |
Message queue | Kafka | Supported | Supported | - |