Realtime Compute for Apache Flink:Manage Apache Paimon catalogs

Read data from a catalog

SELECT * FROM `<catalog name>`.test_db.test_tbl;

Write data to a catalog

INSERT INTO `<catalog name>`.test_db.test_tbl VALUES ('2023-04-21', 1, 'AAA'), ('2023-04-21', 2, 'BBB');

Use an Apache Paimon catalog as the catalog of the destination store that is used in the CREATE TABLE AS statement

In Realtime Compute for Apache Flink whose engine version is vvr-6.0.7-flink-1.15 or later, you can use an Apache Paimon catalog as the catalog of the destination store that is used in the CREATE TABLE AS statement.

CREATE TABLE IF NOT EXISTS `<catalog name>`.`<db name>`.`<table name>`
WITH (
  'bucket' = '4' -- Specify the number of buckets for the result table.
) AS TABLE `<source table>`;

The CREATE TABLE AS statement allows you to configure physical table properties in the WITH clause. When you create a destination table, you can configure these properties for the table. For more information about the table properties supported by Apache Paimon catalogs, see Apache Paimon connector.

When you execute the CREATE TABLE AS statement, you may need to change the data type precision for the existing fields. For example, you can change the data type precision from VARCHAR(10) to VARCHAR(20). Apache Paimon catalogs allow you to configure 'enableTypeNormalization' = 'true' in the WITH clause to enable the type normalization mode. In this case, data type changes in the source table do not lead to a deployment failure only if the data types before and after the change can be converted into the same data type based on type normalization rules.

The type normalization mode has the following rules:

• The TINYINT, SMALLINT, INT, and BIGINT data types are converted into the BIGINT data type.

• The CHAR, VARCHAR, and STRING data types are converted into the STRING data type.

• The FLOAT and DOUBLE data types are converted into the DOUBLE data type.

• Other data types are not normalized.

Examples:

• When the type normalization mode is enabled, the SMALLINT and INT data types are converted into the BIGINT data type. If you change the SMALLINT data type to the INT data type, the change is considered successful. Therefore, the deployment that executes the CREATE TABLE AS statement runs as expected.

• When the type normalization mode is enabled, the FLOAT data type is converted into the DOUBLE data type and the BIGINT data type is converted into the BIGINT data type. If you change the FLOAT data type to the BIGINT data type, a data type incompatibility error occurs.

Use an Apache Paimon catalog as the catalog of the destination store that is used in the CREATE DATABASE AS statement

In Realtime Compute for Apache Flink whose engine version is vvr-6.0.7-flink-1.15 or later, you can use an Apache Paimon catalog as the catalog of the destination store that is used in the CREATE DATABASE AS statement.

CREATE DATABASE IF NOT EXISTS `<catalog name>`.`<db name>`
WITH (
 'bucket' = '4' -- Specify the number of buckets for each result table.
) AS DATABASE `<source database>`;

The CREATE DATABASE AS statement allows you to configure physical table properties in the WITH clause for a deployment. When the deployment starts, these parameters take effect on the result tables to which you want to synchronize data. For more information about the table properties supported by Apache Paimon catalogs, see Apache Paimon connector.

When you use the CREATE DATABASE AS statement, Apache Paimon catalogs also allow you to configure 'enableTypeNormalization' = 'true' in the WITH clause to enable the type normalization mode.