This topic describes the table types and connectors that Realtime Compute for Apache Flink supports.
Table type definitions
Realtime Compute for Apache Flink uses Flink SQL to define tables that map to upstream and downstream storage. You can also use the Datastream API to connect to storage and perform read and write operations. The following Flink SQL table types are supported:
-
Source table: The entry point for a data stream. It continuously reads raw event data from external systems, such as Kafka or MySQL CDC.
-
Dimension table: A reference table used to enrich streaming data. It typically uses a JOIN operation to associate static or infrequently updated dimension information, such as user or product details.
-
Sink table: The exit point for data processing. It writes computed results, such as aggregated or joined data, to a destination system, such as a data warehouse, database, or message queue.
Supported connectors
|
Relational databases |
NoSQL databases |
Message queues |
|
Data warehouses |
Data lakes |
Log and object storage services |
|
Tools and debugging |
Key-value and time series databases |
General |
Connector support details
|
Connector |
Supported Types |
Execution mode |
API type |
Supports updates or deletions to sink table data |
||
|
Source table |
Dimension table |
Sink table |
||||
|
Note
Supports connections to ApsaraDB RDS for MySQL, PolarDB for MySQL, and self-managed MySQL. |
√ |
√ |
√ |
Streaming |
SQL, DataStream, and YAML for data ingestion |
Yes |
|
√ |
√ |
√ |
Streaming and batch |
SQL, DataStream, and YAML for data ingestion |
Yes |
|
|
√ |
× |
√ |
Streaming |
SQL, DataStream, and YAML for data ingestion |
Sink tables only support data insertion, not updates or deletions. |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL and DataStream |
Data can be inserted into a sink table, but it cannot be updated or deleted. |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL and YAML for data ingestion |
Yes |
|
|
√ |
× |
√ |
Streaming |
SQL and DataStream |
You can only insert data into a sink table. You cannot update or delete data. |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL, DataStream, and YAML for data ingestion |
Yes |
|
|
√ |
× |
√ |
Streaming and batch |
SQL and DataStream |
The sink table supports inserting data, but not updating or deleting data. |
|
|
√ |
√ |
√ |
Streaming |
SQL and DataStream |
Yes |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL |
Yes |
|
|
√ |
× |
√ |
Streaming and batch |
SQL and DataStream |
A sink table supports only insert operations. It does not support update or delete operations. |
|
|
Note
The ApsaraDB RDS for MySQL connector will no longer be supported. Use the MySQL connector instead. |
× |
√ |
√ |
Streaming and batch |
SQL |
Yes |
|
√ |
× |
√ |
Streaming |
SQL and DataStream |
The sink table supports inserts, but not updates or deletes. |
|
|
× |
√ |
√ |
Streaming |
SQL |
Yes |
|
|
× |
√ |
√ |
Streaming |
SQL |
Yes |
|
|
× |
× |
√ |
Streaming |
SQL |
Yes |
|
|
× |
√ |
√ |
Streaming |
SQL |
Yes |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL |
Yes |
|
|
× |
√ |
√ |
Streaming and batch |
SQL |
Yes |
|
|
× |
× |
√ |
Streaming and batch |
SQL |
Yes |
|
|
√ |
× |
√ |
Streaming |
SQL and YAML for data ingestion |
Yes |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL and DataStream |
Yes |
|
|
× |
× |
√ |
Streaming and batch |
SQL |
Yes |
|
|
√ |
√ |
√ |
Streaming |
SQL |
Yes |
|
|
× |
× |
√ |
Streaming and batch |
DataStream and SQL |
Yes |
|
|
× |
× |
√ |
Streaming |
SQL |
No |
|
|
√ |
× |
× |
Streaming |
SQL |
N/A |
|
|
√ |
√ |
√ |
Streaming and batch |
SQL |
Yes |
|
|
√ |
× |
√ |
Streaming and batch |
SQL |
Yes |
|
|
√ |
√ |
× |
Streaming and batch |
SQL |
N/A |
|
|
× |
× |
√ |
Streaming and batch |
SQL |
Yes |
|
|
√ |
× |
× |
Streaming and batch |
SQL |
N/A |
|
|
× |
× |
√ |
Streaming and batch |
SQL and YAML for data ingestion |
Yes |
|
|
√ |
× |
√ |
Streaming and batch |
SQL and DataStream |
Yes |
|