In the Dataphin real-time compute engine, you can use real-time tasks to create table types for real-time data sources. This topic lists the supported real-time data sources and their read/write support.
Supported real-time data sources
Storage type | Data source type | Metadata table | Read | Write | Dimension table read |
Big data storage | MaxCompute | Supported | Supported | Supported | Supported |
Hologres | Supported | Supported | Supported | - | |
Hudi | Supported | Supported | Supported | - | |
StarRocks | Supported | Supported | Supported | - | |
Lindorm (compute engine) | - | - | Supported | Supported | |
Relational database | PolarDB | - | Supported | Supported | - |
PolarDB-X | Supported | - | Supported | Supported | |
MySQL | Supported | Supported (CDC) | Supported | Supported | |
PostgreSQL | Supported | Supported | - | - | |
AnalyticDB for MySQL | Supported | - | Supported | Supported | |
AnalyticDB for PostgreSQL | Supported | - | Supported | Supported | |
Oracle | - | - | Supported | - | |
ClickHouse | Supported | - | Supported | - | |
Message queue | Log Service | Supported | Supported | Supported | - |
Kafka | Supported | Supported | Supported | - | |
RocketMQ | Supported | Supported | Supported | Supported | |
NoSQL | Elasticsearch | Supported | Supported | Supported | Supported |
ApsaraDB for HBase | Supported | - | Supported | Supported | |
Tablestore | Supported | - | Supported | Supported | |
Redis | Supported | - | Supported | Supported | |
Lindorm (wide-column engine) | Supported | - | Supported | Supported |
Alibaba Blink real-time data sources
Storage type | Data source type | Read | Write | Dimension table read |
Big data storage | MaxCompute | Supported | Supported | Supported |
DataHub | Supported | Supported | - | |
Hologres | Supported | Supported | Supported | |
Relational database | PolarDB-X (formerly DRDS) | - | Supported | Supported |
AnalyticDB for MySQL 2.0 | - | Supported | - | |
AnalyticDB for MySQL 3.0 | - | Supported | Supported | |
Message queue | Log Service (SLS/LOG_HUB) | Supported | Supported | - |
Kafka | Supported | Supported | - | |
RocketMQ | Supported | Supported | - | |
NoSQL | Elasticsearch | Supported | Supported | - |
ApsaraDB for HBase | - | Supported | Supported | |
Tablestore | - | Supported | Supported | |
Redis | - | Supported | Supported |
Apache Flink: Supported real-time data sources
Storage type | Data source type | Meta table | Read | Write | Dimension table read |
Big data storage | Hive | Supported (for CDH 6.x Hive 2.1.1, CDP 7.x Hive 3.1.3, and AsiaInfo DP 5.x Hive 3.1.0 only) | Supported | Supported | Supported |
Hive (Hudi) | Supported | Supported | Supported | - | |
Hive (Iceberg) | - | Supported (native DDL only) | Supported (native DDL only) | - | |
Hive (Paimon) | Supported (for Alibaba Cloud EMR 5.x Hive 3.1.x and CDH 6.x Hive 2.1.1 only) | Supported (Alibaba Cloud EMR 5.x Hive 3.1.x: requires Flink 1.20.1 for Paimon 1.0.1; CDH 6.x Hive 2.1.1: requires Flink 1.15.3 for Paimon 0.8.2) | Supported (Alibaba Cloud EMR 5.x Hive 3.1.x: requires Flink 1.20.1 for Paimon 1.0.1; CDH 6.x Hive 2.1.1: requires Flink 1.15.3 for Paimon 0.8.2) | Supported (Alibaba Cloud EMR 5.x Hive 3.1.x: requires Flink 1.20.1 for Paimon 1.0.1; CDH 6.x Hive 2.1.1: requires Flink 1.15.3 for Paimon 0.8.2) | |
Hudi | Supported | Supported | Supported | - | |
Doris | Supported | Supported (for Flink 1.14 and Flink 1.15 only) | Supported (for Flink 1.14 and Flink 1.15 only) | Supported | |
SelectDB | - | - | Supported | - | |
Paimon | Supported | Supported (for Flink 1.14 and Flink 1.15 only) | Supported (for Flink 1.14 and Flink 1.15 only) | - | |
StarRocks | Supported | Supported | Supported | - | |
TDH Inceptor (9.3.x) | Supported | - | Supported | Supported | |
Relational data source | MySQL | Supported | Supported (CDC) | Supported | Supported |
SAP HANA | Supported | Supported | - | - | |
Microsoft SQL Server | Supported | Supported (CDC) | Supported (for Flink 1.20.x only) | - | |
PostgreSQL | Supported | Supported | Supported | Supported | |
AnalyticDB for PostgreSQL | Supported | - | Supported | - | |
OceanBase | Supported | Supported (CDC) | - | - | |
Oracle | Supported | Supported (CDC) | Supported | - | |
ClickHouse | Supported | - | Supported (for Flink 1.15 only) | - | |
TiDB | Supported | Supported (CDC) | - | - | |
GaussDB (DWS) | - | - | Supported | Supported | |
TDSQL for PostgreSQL | Supported (for Flink 1.20 only) | Supported (CDC, for Flink 1.20 only) | Supported (for Flink 1.20 only) | - | |
Message queue | Log Service | Supported | Supported (for Flink 1.15 only) | Supported (for Flink 1.15 only) | - |
Kafka | Supported | Supported | Supported | - | |
RabbitMQ | Supported | Supported | Supported | - | |
RocketMQ | Supported | Supported | Supported | - | |
NoSQL | HBase 0.9.4 | - | - | Supported | Supported |
HBase 1.1.x | Supported | - | Supported | Supported | |
HBase 2.x | Supported | - | Supported | Supported | |
Elasticsearch | Supported | - | Supported | - | |
Redis | Supported | - | Supported (for Flink 1.15 only) | Supported (for Flink 1.15 only) | |
MongoDB | Supported | Supported (CDC) | - | - | |
Semi-structured storage | SAP TABLE | - | Supported | - | - |
File | OSS | Supported | - | Supported | - |
Supported real-time data sources for FusionInsight Flink
Storage type | Data source type | Read | Write | Dimension table read |
Relational database | MySQL | Supported (CDC) | Supported (JDBC) | Supported (JDBC) |
PostgreSQL | - | Supported (JDBC) | Supported (JDBC) | |
Oracle | - | Supported (JDBC) | Supported (JDBC) | |
Message queue | Kafka | Supported | Supported | - |