E-MapReduce:Use a DLF Catalog

Prerequisites

Before you begin, ensure that you have:

• A Serverless StarRocks instance at version 3.3 or later, with a Minor Version of 3.3.8-1.99 or later. To check the minor version, go to the Version Information section on the Instance Details page. If the minor version is earlier than 3.3.8-1.99, update it. To create an instance, see Create an instance

• A data catalog in DLF

• A RAM user. To create one, see Create a RAM user

Step 1: Add a RAM user in StarRocks

DLF uses Resource Access Management (RAM) for access control. By default, StarRocks users have no permissions on DLF resources. Add an existing RAM user to StarRocks before granting DLF permissions.

1. Go to the instance list page.

   1. Log on to the E-MapReduce console.

   2. In the navigation pane on the left, choose EMR Serverless > StarRocks.

   3. In the top menu bar, select the required region.

2. On the Instance List page, find your instance and click Connect in the Actions column. For more information, see Connect to a StarRocks instance using EMR StarRocks Manager. Connect using the admin user or a StarRocks super administrator account.

3. In the left-side menu, choose Security Center > User Management, then click Create User.

4. In the Create User dialog box, set the following parameters and click OK.

   Parameter	Value
   User Source	RAM User
   Username	Select the RAM user (for example, dlf-test)
   Password / Confirm Password	Enter a custom password
   Roles	Keep the default value public

Step 2: Grant catalog permissions in DLF

1. Log on to the Data Lake Formation console.

2. On the Catalogs page, click the name of your catalog.

3. Click the Permissions tab, then click Grant Permissions.

4. From the Select DLF User drop-down list, select the RAM user you added in Step 1 (for example, dlf-test).

5. Set Preset Permission Type to Custom and grant the ALL permission on the current data catalog and all its resources.

6. Click OK.

Step 3: Create a DLF catalog in StarRocks

Reconnect to the StarRocks instance using the RAM user you added in Step 1. All catalog creation and data access in the following steps uses this RAM user.

To create a query in the SQL Editor, go to the Querys page and click the icon.

CREATE EXTERNAL CATALOG `dlf_catalog`
PROPERTIES (
  'type' = 'paimon',
  'uri' = 'http://cn-hangzhou-vpc.dlf.aliyuncs.com',
  'paimon.catalog.type' = 'rest',
  'paimon.catalog.warehouse' = 'StarRocks_test',
  'token.provider' = 'dlf'
);

CREATE EXTERNAL CATALOG `iceberg_catalog`
PROPERTIES
(
  'type' = 'iceberg',
  'iceberg.catalog.type' = 'dlf_rest',
  'uri' = 'http://cn-hangzhou-vpc.dlf.aliyuncs.com/iceberg',
  'warehouse' = 'iceberg_test',
  'rest.signing-region' = 'cn-hangzhou'
);

Step 4: Read and write data

Read and write data (Paimon catalog)

Create a database and table, insert data, and then run a query:

-- Create a database
CREATE DATABASE IF NOT EXISTS dlf_catalog.sr_dlf_db;

-- Create a table
CREATE TABLE dlf_catalog.sr_dlf_db.ads_age_pvalue_analytics (
  final_gender_code STRING COMMENT 'Gender',
  age_level         STRING COMMENT 'Age level',
  pvalue_level      STRING COMMENT 'Consumption level',
  clicks            INT    COMMENT 'Number of clicks',
  total_behaviors   INT    COMMENT 'Total number of behaviors'
);

-- Insert data
INSERT INTO dlf_catalog.sr_dlf_db.ads_age_pvalue_analytics
  (final_gender_code, age_level, pvalue_level, clicks, total_behaviors)
VALUES
  ('M', '18-24', 'Low',    1500, 2500),
  ('F', '25-34', 'Medium', 2200, 3300),
  ('M', '35-44', 'High',   2800, 4000);

-- Query data
SELECT * FROM dlf_catalog.sr_dlf_db.ads_age_pvalue_analytics;

The query returns the inserted rows:

Query data (Iceberg catalog)

SELECT * FROM iceberg_catalog.`default`.test_iceberg;

The query result:

Prerequisites

Before you begin, ensure that you have:

• A Serverless StarRocks instance. To create one, see Create an instance

• A data catalog in DLF 1.0. To create one, see Data catalog