VVR 8.0.1 introduced in this release occasionally causes data loss in specific scenarios, which affects data accuracy. Alibaba Cloud has announced End of Support (EOS) for VVR 8.0.1. Upgrade to VVR 8.0.5 or later as soon as possible. For upgrade instructions, see Upgrade the engine version of deployments. We provide necessary support and guidance to help you smoothly transition to a more secure and stable version. Thank you for your understanding and cooperation.
This release of Realtime Compute for Apache Flink (VVR 8.0.1) is based on Apache Flink 1.17.1 and includes engine improvements, connector additions, performance optimization, and bug fixes.
Engine updates
VVR 8.0.1 inherits the following engine-level changes from Apache Flink 1.17.1:
Generic Incremental Checkpoint (GIC): Introduced in Apache Flink 1.17, GIC improves the speed and stability of the checkpointing process.
Unaligned checkpoints (UCs): Stability under backpressure is improved. UCs are now suitable for production use.
Batch processing: Performance is significantly improved.
Enterprise-level state backend restructured
The core architecture of the enterprise-level state backend has been rebuilt, delivering improvements in performance and stability:
Performance
The state format, file storage system, and data cleaning strategy are optimized. This significantly reduces the pressure on local disk space and improves state access speed.
Average performance of large-state deployments increases by more than 40%.
State size decreases by approximately 30%.
Stability
For deployments with large state (such as 100 GB), interruption time caused by deployment updates is reduced from minutes to seconds.
The restructured enterprise-level state backend is enabled by default in VVR 8.0.1. No additional configuration is required.
New features
| Feature | Description | Reference |
|---|---|---|
| MongoDB Change Data Capture (CDC) connector | Creates source tables to read incremental data from MongoDB databases using replica set or sharded cluster architecture. Supports incremental snapshot reading: starts with a parallel full scan of historical data, then automatically switches to incremental capture of the changelog stream. Guarantees exactly-once processing and supports multiple startup modes. | MongoDB CDC connector |
Synchronization of new tables using CREATE DATABASE AS | When new tables are added to the source database after a deployment starts, the deployment can be restarted from a snapshot to capture and synchronize data from the new tables. | CREATE DATABASE AS statement |
CREATE TABLE AS inside BEGIN STATEMENT SET; ... END; | Adding a CREATE TABLE AS statement inside a statement set block supports snapshot-based restart. This improves flexibility and eliminates the need to create additional deployments. | CREATE TABLE AS statement |
| Per-stream time-to-live (TTL) configuration in regular joins | Configures TTL separately for each stream in a regular join. For example, set one stream to 15 days and the other to 1 day. This improves deployment stability and reduces operating costs. | Optimize Flink SQL |
| OceanBase connector | Creates sink tables and dimension tables for OceanBase, a distributed relational hybrid transactional/analytical processing (HTAP) database developed by Alibaba Group and Ant Group. OceanBase provides various benefits, including strong consistency, high availability, high performance, online scalability, high compatibility with SQL standards and mainstream relational databases, and low costs. | OceanBase connector |
| Query pushdown in the Simple Log Service (SLS) connector | Filters data at the source, improving read efficiency. | Simple Log Service connector |
| Sink tables in SLS catalogs | Writes data to SLS using catalog-managed tables, the same way as permanent tables. | Manage Simple Log Service catalogs |
| AnalyticDB for PostgreSQL V7.0 support | Reads from and writes to AnalyticDB for PostgreSQL V7.0 instances. | AnalyticDB for PostgreSQL connector |
| Additional Tair data types | Supports TairTs (time series datasets), TairVector (vector datasets for AI), TairCpc (real-time fraud detection), TairRoaring (real-time customer profiling), and TairGis. | Tair connector |
| Apache Paimon 0.5-snapshot support and column type propagation | Supports Apache Paimon 0.5-snapshot. Column type changes in the Flink CDC source table are automatically applied to the Apache Paimon table. | CREATE TABLE AS statement and CREATE DATABASE AS statement |
Fixed issues
| Issue | Description |
|---|---|
| MySQL connector error with PolarDB for MySQL | The error message Filtering update table metadata event: Event{header=EventHeaderV4 appeared when using the MySQL connector to read from a PolarDB for MySQL database. |
| No output from window table-valued function (TVF) with conditions | When a window TVF was used with filter conditions, no output was generated. |
Upgrade rollout
This version is rolled out using a canary release strategy over a two-week period. After the upgrade reaches your region and account, the new engine version becomes available for your deployments. For details, see Upgrade the engine version of deployments. We look forward to your feedback.