Data Lake Formation (DLF) 1.0 is a fully managed service for building cloud data lakes and lakehouses. DLF provides unified metadata management, security and permission controls, and one-click data exploration across multiple compute engines to break down data silos.
Pricing
| Item | Billing model | Free tier |
|---|---|---|
| Metadata management | Pay-as-you-go | First 1 million stored metadata objects per month |
| API requests | — | First 1 million API requests per month |
| Data exploration | Free public preview | Not billed |
| Permission management | Free public preview | Not billed |
| Lake management | Free public preview | Not billed |
Metadata objects and API requests beyond the free tier incur charges. Pricing details are covered in Billing.
Architecture
Data Catalog
View and manage the Data Catalog for your data lake from the console.
Database tables and functions
View and manage database tables and functions from the console. The CreateDatabase and CreateTable APIs enable metadata operations and third-party service integration. DLF supports multi-version management and can automatically generate metadata through metadata extraction.
Data permission management
Data permission management controls lake-level data access at five granularity levels: data catalog, database, data table, data column, and function.
Data lake management
Data lake management provides analysis and optimization suggestions for data storage, manages data lifecycles, helps optimize costs, and simplifies O&M.
Data exploration
One-click querying and analysis using Spark 3.0 SQL. Save historical queries, preview data, export results, and generate TPC-DS test datasets with one click.
Scenarios
Build a cloud-based data lake
Combine DLF with E-MapReduce and Object Storage Service (OSS) to build a cloud data lake.
Build a data lakehouse architecture
Combine DLF with MaxCompute, DataWorks, and E-MapReduce to build a data lakehouse.
Build a fully managed data lakehouse architecture
Combine DLF with Databricks and OSS to build a fully managed cloud data lakehouse.
Analyze data in OSS
Use metadata extraction and data exploration to analyze and explore structured and semi-structured data in OSS.