LakeBase Catalog is a unified catalog management and analytics capability provided by PolarDB for PostgreSQL for lake-based data. You can manage different types of catalogs in PolarDB for PostgreSQL and directly access and analyze data in those catalogs by using standard SQL.
The LakeBase Catalog feature is currently in public preview. To use this feature, submit a ticket to contact us for assistance.
Overview
After you register a catalog, you can query lake-based data directly from PolarDB for PostgreSQL without importing data into local PostgreSQL tables or switching between a catalog service and a separate query engine.
This capability supports multiple data types, including Lance, Paimon, PostgreSQL, and DuckDB. It covers scenarios such as OLTP, OLAP, AI and vector data analytics, and streaming-batch integrated data lake queries. LakeBase Catalog provides a consistent object model, SQL entry point, and management approach for different types of lake-based data.
Core features
Integrated catalog management and analytics
LakeBase Catalog integrates catalog lifecycle management and lake-based SQL analytics into PolarDB for PostgreSQL. You can create, modify, delete, view, and authorize catalogs in a unified manner, and directly run queries and analytics on the data managed by those catalogs.
Traditional catalog services typically store metadata, storage locations, and connection configurations. Actual analytics require additional compute engines such as Spark or Trino. LakeBase Catalog provides an analytics execution path alongside catalog management. PolarDB for PostgreSQL identifies the target catalog in a SQL statement, resolves metadata, verifies permissions, and routes the query to the corresponding data lake provider.
When you delete a catalog, only the mapping and related metadata in PolarDB for PostgreSQL are removed. The underlying data is not deleted. This keeps catalog management independent from the data storage lifecycle.
Unified access to multiple data types
LakeBase Catalog supports multiple data types, including Lance, Paimon, PostgreSQL, and DuckDB. Each catalog type targets a different data format and analytics scenario, but all types share the same catalog-schema-table object hierarchy and SQL access method.
Data type | Analytics capabilities | Data and storage | Typical analytics content |
Lance | Management, query, and analytics of AI and vector data lakes | Lance data on local storage or OSS. Supports Namespace or REST Catalog. | Embeddings, vector data, feature data, and AI data assets |
Paimon | Query and analytics of streaming-batch integrated lake tables | Paimon files on local storage or OSS. Supports REST Catalog. | Lake-based detail data, real-time and offline lake tables, and historical data |
PostgreSQL | Analytics of local row-store table data | PolarDB row-store data files on shared storage | Business data |
DuckDB | Creation, read/write, and aggregate analytics of local OLAP data | PolarDB DuckDB data files on shared storage | Analytics tables, metric data, temporary datasets, and aggregation results |
LakeBase Catalog supports federated queries across row-store, columnar, and multi-source heterogeneous lake data without requiring prior data movement or transformation. Cross-database analytics within PostgreSQL is also supported.
Direct lake data analytics with standard SQL
You can access data in different catalogs by using the three-part naming convention: catalog.schema.table. LakeBase Catalog identifies the target catalog, schema, and table based on the three-part name, automatically loads the corresponding metadata and connection configuration, and enters the appropriate analytics execution path.
Lake-based data can participate in SQL queries just like database tables. Common operations such as column selection, condition filtering, sorting, grouping, and aggregate analytics are supported. Applications can continue to use PostgreSQL clients, drivers, and SQL tools to access lake data, which reduces the differences in access methods caused by different data lake formats.
After you set a default catalog and schema, you can use the simpler two-part naming convention to access data. When you need to explicitly specify the data source, you can always use the full three-part name.
Lake object creation and data write
LakeBase Catalog supports not only reading lake data but also creating lake data objects and writing data.
For catalogs that support writes, you can create schemas and tables, insert or update data, and then run filtering and aggregate analytics by using SQL. For read-only catalogs, you can discover existing schemas and tables and query data in external data lakes.
Built-in high-performance DuckDB engine
LakeBase Catalog provides an analytics execution path for lake-based data within PolarDB for PostgreSQL. The execution capabilities are powered by a built-in DuckDB engine.
DuckDB uses columnar, vectorized execution and is suitable for filtering, aggregation, and exploratory analytics on small-to-medium-scale datasets. You do not need to deploy a separate distributed compute cluster to obtain lightweight lake analytics capabilities in PolarDB for PostgreSQL.
Catalog-level access control
Each catalog has an independent owner and access control list (ACL). LakeBase Catalog provides two types of permissions: data access and object creation. Unified permission checks are enforced during lake queries, data modifications, and object management.
Data access permissions: control whether a user can query or modify data in a catalog.
Object creation permissions: control whether a user can manage schemas and tables under a catalog.
The catalog owner can grant permissions to users or roles, revoke permissions, or transfer ownership. Permission checks are combined with three-part name routing. Whether a user accesses data by using the full three-part name or switches to the default catalog and uses a two-part name, the same catalog permission rules apply.
Lake analytics observability
LakeBase Catalog provides observability for catalog management and SQL analytics. Administrators can view the type, connection configuration, comments, and permission information of a catalog. Analysts can view SQL execution plans to understand the scan, filter, and aggregation processes for lake data.
The system supports logging catalog-related DDL, data modification, and query statements at different levels. It also provides configuration for metadata cache entry limits and cache TTL. Online feature switches allow you to enable or disable LakeBase Catalog at the session or global level without affecting other PolarDB for PostgreSQL features.
Use cases
Local OLAP and exploratory analytics
Use a DuckDB catalog to create local analytics tables and datasets in PolarDB for PostgreSQL. You can run filtering, grouping, aggregation, and exploratory queries on the data. Analytics data is stored in columnar files on PolarDB shared storage and serves as an independent analytics space separate from business data.
This use case is suitable for ad hoc data analysis, metric computation, analytics result persistence, and tasks that require fast processing of structured data by using PostgreSQL SQL.
Multi-source federated analytics
LakeBase Catalog can unify access to Lance vector data, Paimon lake tables, PolarDB for PostgreSQL row-store data, and DuckDB columnar data through multiple catalogs. It provides a unified SQL entry point for different data sources, storage formats, and analytics workloads.
Analysts can use three-part names to precisely identify data sources. In a single query, they can join, filter, and aggregate data across catalogs to achieve federated analytics across business data, analytics data, AI vector data, and streaming-batch integrated lake tables. The underlying data remains in its original storage without requiring prior movement or format conversion.
AI and vector data analytics
Use a Lance catalog to access vector data managed by a Lance Namespace or REST Catalog. You can use unified SQL to view and analyze embeddings, feature data, and other AI data assets.
Analysts can discover Lance data objects, view table schemas, and run queries in PolarDB for PostgreSQL without switching to a separate catalog client or data access entry point for vector data.
Streaming-batch integrated lake table analytics
Use a Paimon catalog to directly access a Paimon warehouse on the local file system or OSS. You can use standard SQL to analyze streaming-batch integrated lake tables produced by real-time and offline processing pipelines.
Analysts can run filtering, summarization, and join analytics on lake-based detail data, historical data, and processed results without importing data into PolarDB for PostgreSQL in advance.