Data Lake Formation:Access Iceberg catalogs in DLF using Flink SQL

Prerequisites

Before you begin, ensure that you have:

• A Realtime Compute for Apache Flink workspace. See Activate Realtime Compute for Apache Flink.

• A session cluster running Ververica Runtime (VVR) 11.2.0 or later. See Create a session cluster.

Register the DLF catalog in Flink

Registering a catalog in Flink creates a mapping to your DLF catalog. Creating or deleting the catalog in Flink does not affect the actual data in DLF. All tables created in the DLF catalog through Iceberg REST are Iceberg tables.

1. Log on to the Realtime Compute for Apache Flink Management Console.

2. In the Actions column of your workspace, click Console.

3. In the left navigation pane, click Development > Scripts.

4. Create a new script, then paste the following SQL statement into the SQL editor.

   CREATE CATALOG `catalog_name`
    WITH (
       'type' = 'iceberg',
       'catalog-type' = 'rest',
       'uri' = 'http://{region-id}-vpc.dlf.aliyuncs.com/iceberg',
       'warehouse' = 'iceberg_test',
       'rest.signing-region' = '{region-id}',
       'io-impl' = 'org.apache.iceberg.rest.DlfFileIO'
   );

   Replace {region-id} with the region ID of your DLF catalog, for example cn-hangzhou or ap-southeast-1. For all supported regions and their endpoint values, see Regions and endpoints.

5. In the lower-right corner, click Environment, select a session cluster running VVR 11.2.0 or later, and run the SQL statement.

The following table describes the configuration options.

Query data in the catalog

After registering the catalog, run the following SQL statement in Scripts:

SELECT * FROM catalog_name.database_name.table_name;

Replace catalog_name, database_name, and table_name with the actual names from your DLF catalog.