All Products
Search

This Product

    Realtime Compute for Apache Flink:November 27, 2023

    Document Center

    Realtime Compute for Apache Flink:November 27, 2023

    Last Updated:Jan 04, 2024

    This topic describes the release notes for fully managed Flink and provides links to relevant references. The release notes provide the major updates and bug fixes in fully managed Flink in the version that was released on November 27, 2023.

    Important

    A canary release will be complete within two weeks for the upgrade. If you cannot use new features in fully managed Flink, the new version is still unavailable for your account. If you want to perform an upgrade at the earliest opportunity, submit a ticket to apply for an upgrade. To learn about the upgrade plan, view the most recent announcement on the right side of the homepage of the Realtime Compute for Apache Flink console.

    Overview

    The new engine version 8.0.4 of Ververica Runtime (VVR) of fully managed Flink was officially released on November 27, 2023. This version includes the connector updates, performance optimization, and bug fixes.

    This version is a new enterprise-level Flink engine based on Apache Flink 1.17.1. This version supports TINYINT(1) for MySQL Change Data Capture (CDC) tables to ensure data consistency. The Elasticsearch connector supports Elasticsearch 8.X to provide more features and compatibilities. Multiple defects that exist in Apache Flink 1.17.1 are fixed, including the fixed defects of the Apache Flink community. Engine-related issues are also resolved to improve system stability and reliability.

    The canary release will be complete within two weeks on the entire network. After the canary release is complete, you can upgrade the engine that is used by your deployment to the new version. For more information, see Upgrade the engine version of deployments. We look forward to your feedback.

    Features

    Feature

    Description

    References

    Mapping of TINYINT(1) and BOOLEAN data types of MySQL to TINYINT when a MySQL catalog is used to obtain the schema of a table

    A parameter is provided to specify the data type in a Flink MySQL CDC table to which data of the TINYINT(1) and BOOLEAN types in a MySQL table is mapped. This ensures data consistency between the tables.

    Manage MySQL catalogs

    upsertCommitTimeoutMs parameter for data uploads by using MaxCompute Upsert Tunnel

    The upsertCommitTimeoutMs parameter is added to specify the timeout period for which MaxCompute Upsert Tunnel commits a session. This helps avoid occasional timeout issues.

    MaxCompute connector

    Support for auto-increment primary keys and a parameter used to specify the mechanism of dirty data processing

    • If the Tablestore connector is used for a result table, the table schema supports auto-increment primary keys.

    • If the Tablestore connector is used for a source table, you can set the skipInvalidData parameter to true to ignore dirty data. If the skipInvalidData parameter is set to false, an error is returned when dirty data is detected.

    Tablestore connector

    Change of the default value of the Hologres connection pool to a database name.

    The default value of the Hologres connection pool is changed from a fixed value to a database name. This can effectively reduce the number of connections that are used.

    Hologres connector