Create an Iceberg catalog in StarRocks to query Iceberg data - E-MapReduce

An Iceberg catalog is an external catalog that stores the mappings between Iceberg tables and their storage paths. With an Iceberg catalog in E-MapReduce (EMR) StarRocks, you can query Iceberg data directly.

Prerequisites

Before you begin, make sure that you have:

A cluster with the Iceberg service (for example, a DataLake cluster or a custom cluster). For more information, see Create a cluster.
A cluster with the StarRocks service (for example, an online analytical processing (OLAP) cluster or a custom cluster), and you have logged on to the cluster. For more information, see Create a cluster and Getting started.

Limitations

Both clusters must be deployed in the same virtual private cloud (VPC) and zone.
StarRocks can query only analytic data tables (Version 1). Row-level deletes tables (Version 2) are not supported. For details on Iceberg table versions, see Iceberg Table Spec.

Create an Iceberg catalog

Syntax

CREATE EXTERNAL CATALOG <catalog_name>
PROPERTIES
(
  "key"="value",
  ...
);

Parameters

`catalog_name` (required)

The name of the Iceberg catalog. The name must meet the following requirements:

Contains only letters, digits, and underscores (_), and starts with a letter.
Is 1 to 64 characters in length.

`PROPERTIES` (required)

The properties of the Iceberg catalog. Configurations vary based on the metadata service used by the Iceberg data source.

Property	Required	Description
`type`	Yes	The type of the data source. Set to `iceberg`.
`iceberg.catalog.type`	Yes	The catalog type of the Iceberg data source. For Hive Metastore, set to `HIVE`.
`hive.metastore.uris`	No	The URI of the Hive Metastore service, in the format `thrift://<IP address of the Hive metastore>:<Port number>`. The default port is 9083. Leave this blank if you use Data Lake Formation (DLF) to store metadata.

Example

Create an Iceberg catalog named iceberg_catalog:

CREATE EXTERNAL CATALOG iceberg_catalog
PROPERTIES
(
    "type" = "iceberg",
    "iceberg.catalog.type" = "HIVE",
    "hive.metastore.uris" = "thrift://xx.xx.xx.xx:9083"
);

Query data in an Iceberg table

Run the following statement to query data in a specific Iceberg table:

SELECT * FROM <catalog_name>.<database_name>.<table_name>;

What's next

For background on Apache Iceberg, see Overview.