Realtime Compute for Apache Flink:Version of August 8, 2025

Flink SQL: 13 new built-in functions

VVR 11.2 adds 13 built-in scalar functions, reducing the need for custom UDFs.

String processing: PRINTF, TRANSLATE, ELT, BTRIM, STARTSWITH, ENDSWITH

JSON processing: JSON_QUOTE, JSON_UNQUOTE

Regular expressions: REGEXP_SUBSTR, REGEXP_INSTR, REGEXP_COUNT, REGEXP_EXTRACT_ALL

Arithmetic: UNHEX

For the full function reference, see Supported functions.

Flink SQL: Variant type

Flink SQL now supports the Variant data type. Use it when column schemas are semi-structured or evolve over time — for example, when ingesting JSON from heterogeneous sources. Variant provides a flexible container without requiring a rigid schema upfront.

For type conversion rules, see Data type conversion.

Table API: Hive SQL dialect

Table API jobs can now use the Hive SQL dialect. If your team is already familiar with HiveQL syntax, you can write Table API jobs without learning Flink-specific SQL variants.

See Get started with a Hive SQL deployment.

AI function: Configurable overflow behavior

When a message exceeds the AI model's context window, choose how to handle it: Discard the message or Truncate it to fit. This replaces the previous fixed behavior and lets you tune the tradeoff between data completeness and job stability.

Configure the strategy in the model DDL. See Model DDLs.

MySQL CDC: VARCHAR field optimization

The MySQL CDC connector now handles VARCHAR fields more efficiently, improving both synchronization throughput and stability for VARCHAR-heavy schemas.

See MySQL.

Flink CDC: Canal-JSON format and timestamp extraction

CDC jobs can now consume Canal-JSON formatted data from Apache Kafka. The connector extracts both event timestamps (ts) and event sequences (es) fields, enabling accurate ordering and deduplication in downstream pipelines.

See MySQL and Synchronize MySQL binary logging data to Kafka.

AnalyticDB for MySQL: INSERT IGNORE support

The AnalyticDB for MySQL connector now supports the INSERT IGNORE syntax. Duplicate records during write operations no longer cause job failures — the connector skips conflicting rows and continues processing.

See AnalyticDB for MySQL V3.0 connector.

Paimon and OSS: RAM-based authorization

The Paimon connector and Object Storage Service (OSS) connector now support authorization via Resource Access Management (RAM) roles, replacing AccessKey pairs. RAM roles provide finer-grained permission control and eliminate the need to embed long-lived credentials in connector configurations.

See Manage Paimon catalogs.

Tair (Redis OSS-compatible): Async lookup joins

The Tair connector now supports asynchronous lookup joins. Lookup join requests no longer block the processing pipeline, improving cache access efficiency and overall job throughput.

See ApsaraDB for Tair (Redis Open-Source Edition).

MongoDB CDC: Concurrent oplog parsing

The MongoDB CDC connector now parses oplogs concurrently instead of sequentially. This improves synchronization stability and reliability for MongoDB change streams.

See MongoDB.

Kafka-Paimon ingestion: Automatic schema evolution

When ingesting data from Apache Kafka into Paimon, schema changes in the Kafka topic are now automatically propagated to the Paimon table. This eliminates manual schema migration steps during data lake ingestion.

See Implement real-time data ingestion into a data lake.

PyFlink: Direct use of built-in connectors

PyFlink jobs can now use built-in connectors directly, without manually managing connector JAR dependencies. This simplifies the development setup for Python-based Flink jobs.

See Use Python dependencies.

Run multiple DDL/DML statements in a single batch job

A single batch job can now execute multiple DDL and DML statements in sequence — create tables, run computations, and drop tables, all in one job submission. This removes the need to chain separate jobs for multi-step batch workflows.

Refresh historical partitions on a schedule

Materialized tables now support scheduled refresh of historical partitions. Use this to backfill late-arriving data into past partitions, maintaining eventual consistency without manual intervention.

Auto-release idle session clusters

New session clusters that remain idle for more than 30 minutes are automatically released. This prevents idle clusters from consuming resources and reduces unnecessary CU usage.

Blackout periods for automatic tuning

Define blackout periods to prevent automatic resource scaling during business-critical hours. The platform continues to generate tuning recommendations during blackout periods but does not apply them — giving you full control over when scaling occurs.

Expanded Git integration

Git integration now supports additional tools, including Alibaba Cloud DevOps. Pull directory structures directly from connected repositories, and error messages during Git operations now include actionable troubleshooting guidance.

Granular access control for data queries

Data query operations now support finer-grained access control, enabling more precise permission boundaries for teams sharing a workspace.

New APIs

Previously, the Resource and DeploymentTarget APIs could not manage hybrid billing workspaces. This release adds two new APIs to address this:

The Resource object also gains new fields for configuring hybrid billing workspaces.

Deprecated APIs

Bug fixes

Connectors

SQL and data transformation

• Fixed LIKE syntax parsing in Paimon.

• Fixed date handling and REGEXP_REPLACE errors in YAML scripts.

• Resolved NullPointerException when accessing schema registry.

Stability and performance

• Fixed metadata inconsistencies that could occur after job failover.

• Fixed resource cleanup on unexpected job exits.

• Optimized the connector retry mechanism for greater job resilience.