This topic describes how to connect Serverless StarRocks to a DLF Catalog using Iceberg REST. - Data Lake Formation

This topic describes how to connect Serverless StarRocks to a Data Lake Formation (DLF) Catalog using Iceberg REST.

Connect Serverless StarRocks to Data Lake Formation (DLF) using the Iceberg REST protocol so that StarRocks can query Iceberg tables stored in DLF directly.

Prerequisites

Before you begin, make sure you have:

A Serverless StarRocks instance running version 3.3.20-2.0.0 or later. See Create an instance and EMR StarRocks release notes.
A DLF catalog is created.

Set up access control

DLF uses Resource Access Management (RAM) for access control. By default, StarRocks users have no access to DLF. To grant access, create a RAM user that acts as the StarRocks identity when connecting to DLF, grant that user data permissions in the DLF console, then map it to a StarRocks user.

Step 1: Create or select a RAM user

Log on to the RAM console as an Alibaba Cloud account owner or RAM administrator.
Create a RAM user or select an existing one.

The RAM user name can contain only letters, digits, and underscores (_) to satisfy StarRocks naming conventions.

Step 2: Grant data permissions in DLF

Log on to the DLF console.
Navigate to your Paimon catalog.
On the Permissions tab, click Grant Permissions and grant the necessary data access on your catalog to the RAM user. For more information, see Manage data permissions.

Step 3: Create and configure a StarRocks user

Go to the EMR Serverless StarRocks console.
Click Connect in the Actions column of your StarRocks instance. For more information, see Use EMR StarRocks Manager to connect to an EMR Serverless StarRocks instance.

Connect with an admin or other administrative user.
In StarRocks Manager, navigate to Security Center > User Management and click Create User.

In the Create User dialog, configure the following settings and click OK:

Field	Value
User Source	Select RAM User
Username	Select the RAM user you created in step 1
Password	Set a password for this StarRocks user
Role	Default is `public`

Grant StarRocks privileges to the new user:
1. On the User Management page, find the new user and click Add Permission in the Actions column.
2. On the Permissions tab, click Add Permission.
3. In the Add Permission panel, set Resource and Permission as needed, then click OK.
  
  This grants the user access to StarRocks internal resources. To modify permissions on your DLF catalog, go to the DLF console.

Step 4: Verify the user mapping (optional)

The StarRocks user is mapped to the RAM user within 10 to 20 seconds. To verify the mapping:

Log on to StarRocks Manager with the new StarRocks user.
Navigate to SQL Editor and click the icon to create an SQL script.
Run the following command:
```
show property;
```
If the RAM user ID appears in the output, the mapping is active:

Create a catalog

Note

All tables created in a DLF Catalog using Iceberg REST are Apache Iceberg tables.

Log on to StarRocks Manager as the RAM user, then run the following SQL to create an external catalog pointing to your DLF Catalog.

On the Queries page, click the icon to create a SQL query.

Enter the following statement and click Run:

Connection parameters

Parameter	Required	Description	Example
`type`	Yes	The StarRocks catalog type. Set to `iceberg`.	`iceberg`
`uri`	Yes	The URI of the DLF REST catalog endpoint. For endpoints by region, see Iceberg REST.	`http://cn-hangzhou-vpc.dlf.aliyuncs.com/iceberg`
`iceberg.catalog.type`	Yes	The catalog implementation type. Set to `dlf_rest` for DLF.	`dlf_rest`
`warehouse`	Yes	The name of the DLF Catalog.	`iceberg_test`

Authentication parameters

Parameter	Required	Description	Example
`rest.signing-region`	Yes	The region ID of the DLF service. For region IDs, see Service endpoint.	`cn-hangzhou`

CREATE EXTERNAL CATALOG iceberg_catalog
PROPERTIES
(
    "type" = "iceberg",
    "uri" = "http://cn-hangzhou-vpc.dlf.aliyuncs.com/iceberg",
    "iceberg.catalog.type" = "dlf_rest",
    "warehouse" = "iceberg_test",
    "rest.signing-region" = "cn-hangzhou"
);

Query Iceberg tables in the catalog:

SELECT * FROM catalog_name.database_name.table_name;