Data analytics - E-MapReduce - Alibaba Cloud Documentation Center

Terms

Internal data: Data stored in StarRocks.
External data: Data stored in external data sources, such as Apache Hive, Apache Iceberg, and Apache Hudi.

Catalog

StarRocks 2.3 and later supports the catalog feature, which lets you access and query data from various external sources. StarRocks provides two types of catalogs: Internal Catalog and External Catalog.

Internal Catalog: Manages all internal data in StarRocks. For example, databases and tables created with the CREATE DATABASE and CREATE TABLE statements are managed by the Internal Catalog. Each StarRocks cluster has only one Internal Catalog, named default_catalog.
External Catalog: Manages access information for external data sources, such as the data source type and the URI of the Hive Metastore. You can query external data directly through an External Catalog.
StarRocks supports creating External Catalogs for the following data sources:
- Hive data source: Queries Hive data.
- Iceberg data source: Queries Iceberg data.
- Hudi data source
- Paimon data source: Queries Paimon data.
- Delta Lake data source: Queries Delta Lake data.
When you query data using an External Catalog, StarRocks uses two components of the external data source:
- Metadata service: Exposes metadata to the StarRocks frontend (FE) for query planning.
- Storage system: Stores the data. Data files are stored in various formats in a distributed file system or an object storage system. The FE distributes the generated query plan to each backend (BE). Each BE then scans the target data in the storage system in parallel, performs the computation, and returns the query result.

Query data

Query internal data

To query data stored in StarRocks, see Internal table data source.

Query external data

To query data stored in an external data source, see Data lake analytics.

Query data across catalogs

To query data in another catalog from the current catalog, reference the target data in the format catalog_name.db_name or catalog_name.db_name.table_name. For example:

From default_catalog.olap_db, you can query hive_table in hive_catalog.
```
SELECT * FROM hive_catalog.hive_db.hive_table;
```
From hive_catalog.hive_db, you can query olap_table in default_catalog.
```
SELECT * FROM default_catalog.olap_db.olap_table;
```
From hive_catalog.hive_db, you can perform a federated query on hive_table and olap_table in default_catalog.
```
SELECT * FROM hive_table h JOIN default_catalog.olap_db.olap_table o WHERE h.id = o.id;
```
From another database, you can perform a federated query on hive_table in hive_catalog and olap_table in default_catalog.
```
SELECT * FROM hive_catalog.hive_db.hive_table h JOIN default_catalog.olap_db.olap_table o WHERE h.id = o.id;
```