Build and Configure Format Tables in DLF - Data Lake Formation - Alibaba Cloud - Data Lake Formation

This topic describes the concept, creation methods, and basic operations of Format tables in Data Lake Formation (DLF).

Format table overview

In DLF, a Format table is Apache Paimon's unified abstraction for traditional Hive-compatible tables. DLF classifies Hive-compatible tables that store data in standard file formats (Parquet, ORC, CSV, or JSON) as Format tables. You can read from and write to Format tables through compute engines such as Spark and Flink.

You can create a Format table in the following ways:

DLF console — guided UI for one-time setup
SQL — Flink SQL or Spark SQL for programmatic workflows (for example, USING Parquet in Spark SQL or type='format-table' in Flink SQL).

DLF Format table features:

Fully managed: DLF fully manages the data, including metadata and the actual data files.
Storage: DLF automatically generates data storage paths based on UUIDs. You do not need to specify storage paths.
Deletion behavior: When you delete a table, DLF retains the data for one day by default to reduce the risk of accidental deletion. After one day, the data is permanently deleted.

Prerequisites

Before you begin, ensure that you have:

A DLF catalog
A database in that catalog

Create a table

Use the console

Log on to the DLF console.
In the left-side navigation pane, click Catalogs, then click your catalog name.
In the Database section, click your database name.

Click Create Table and configure the following fields:

Field	Description
Table Format	Select Format Table.
Table Name	Enter a name. Must be unique within the database.
Table Description	Enter a description.
Columns	Define the table columns.
User-defined Table Properties	Set `file.format` to specify the file format: `Parquet`, `ORC`, `CSV`, or `JSON`. Add any custom properties to override DLF defaults. For all available properties, see Configuration in the Apache Paimon documentation.

Click OK.

Use SQL

All examples use type='format-table' to identify the table as a Format table.

Flink SQL

CREATE TABLE my_csv_table (
    a INT,
    b STRING
) WITH (
    'type'='format-table',
    'file.format'='csv'
    -- 'format-table.file.compression'='zstd',          -- optional: override default compression
    -- 'format-table.partition-path-only-value'='true'   -- optional: for value-only partition paths
);

Spark SQL

CREATE TABLE my_csv_table (
    a INT,
    b STRING
) USING CSV

View a Format table

In the Database section, click your database name.
On the Tables tab, click your table name.
On the Table Details tab, view basic information and column definitions.
On the Permissions tab, grant table access to DLF users or roles. For details, see Manage data permissions.

Delete a table

Warning

Deleted table data is retained for 1 day to prevent accidental deletion. Data is permanently deleted afterward.

In the Database section, click your database name.
On the Database tab, click Delete in the Actions column of the target table.
In the dialog box, click OK.

Advanced configuration

Support value-only partition paths

By default, partition paths use the key=value format, such as year=2025. If your storage directory uses only the partition value — for example, table-path/2025/xxxx.csv — set the following parameter:

format-table.partition-path-only-value: true

Set file compression

Use format-table.file.compression to specify the compression algorithm for output files. Default values by file type:

File type	Default compression
Parquet	snappy
Avro, ORC	zstd
CSV, JSON	none