MaxCompute:Use an external project with a FileSystem Catalog to read and write Paimon Data on a lakehouse

Supported operations

Only Paimon format tables are supported. The following table lists the supported read and write operations.

For data type mappings between Paimon and MaxCompute, see Data type mapping.

Prerequisites

Before you begin, make sure you have:

• An OSS bucket with Paimon table data, or permissions to create one

• Access to a Realtime Compute for Apache Flink workspace (required only if you need to generate sample data in Step 1)

• A Resource Access Management (RAM) role with permissions for both Data Lake Formation (DLF) and OSS — note the role's ARN (Alibaba Cloud Resource Name)

• A MaxCompute quota to assign to the external project

Step 1: Prepare source data

Skip this step if you already have Paimon table data in OSS.

This step uses Flink to create a Paimon table in a FileSystem Catalog and insert sample rows. The data is stored in OSS following Paimon's file system structure: <catalog_name>/<database_name>.db/<table_name>/.

Step 2: Create an external data source in MaxCompute

The external data source stores the OSS connection details and the RAM role that MaxCompute uses to access your Paimon data.

1. Log on to the MaxCompute console and select a region.

2. In the left navigation pane, choose Manage Configurations > External Data Source.

3. On the External Data Source page, click Create External Data Source.

4. Configure the following parameters:

   Core settings

   Parameter	Required	Description
   External Data Source Type	Required	Select Filesystem Catalog.
   External Data Source Name	Required	A custom name that starts with a letter and contains only lowercase letters, underscores (_), and digits. Maximum 128 characters. For example, external_fs.
   Description	Optional	A description for the data source.
   Region	Required	The current region is selected by default.

   Authentication settings

   Parameter	Required	Description
   Authentication And Authorization	Required	Alibaba Cloud RAM role (selected by default).
   RoleARN	Required	The ARN of a RAM role with permissions for DLF and OSS. To find the ARN: log on to the RAM console, go to Identities > Roles, and look in the Basic Information section. Example: acs:ram::124****:role/aliyunodpsdefaultrole.

   Storage settings

   Parameter	Required	Description
   Storage Type	—	OSS or OSS-HDFS.
   Endpoint	—	Auto-generated based on your region. For China (Hangzhou): oss-cn-hangzhou-internal.aliyuncs.com.
   Foreign Server Supplemental Properties	Optional	Additional attributes that control how tasks access the data source. See the official documentation for supported parameters as the product evolves.

5. Click OK.

6. On the External Data Source page, find the data source you just created and click Details in the Actions column to verify the configuration.

Step 3: Create an external project in MaxCompute

1. Log on to the MaxCompute console and select a region.

2. In the left navigation pane, choose Manage Configurations > Projects.

3. On the External Project tab, click Create Project.

4. Configure the following parameters:

   Project settings

   Parameter	Required	Description
   Project Type	Required	External project is selected by default.
   Region	Required	The current region is selected by default and cannot be changed.
   Project Name (Globally Unique)	Required	Must start with a letter and contain letters, digits, and underscores (_). Between 3 and 28 characters.
   Billing Method	Required	Subscription or Pay-as-you-go.
   Default Quota	Required	Select an existing quota.
   Description	Optional	A description for the project.

   External data source settings

   Parameter	Required	Description
   MaxCompute Foreign Server Type	Optional	Select Filesystem Catalog.
   MaxCompute Foreign Server	Optional	Use Existing to select a previously created data source, or Create Foreign Server to create a new one.
   MaxCompute Foreign Server Name	Required	Select an existing data source name, or specify a name for the new data source.
   Table Format	Required	Paimon is selected by default.
   Bucket Catalog	Required	The OSS bucket and directory at the catalog level. For example, oss://paimon-fs/paimon-test/.

   Authentication settings

   Parameter	Required	Description
   Authentication and Authorization	Required	The identity used to run tasks. If a service-linked role does not exist, create one before proceeding.
   RoleARN	Required	The ARN of a RAM role with permissions for DLF and OSS. To find the ARN: log on to the RAM console, go to Identities > Roles, and look in the Basic Information section. Example: acs:ram::124****:role/aliyunodpsdefaultrole.
   Storage Type	—	OSS or OSS-HDFS.
   Endpoint	Required	Generated by default.

5. Click OK to create the project.

Step 4: Read from and write to Paimon tables

Select a connection tool and log on to the external project you just created.

The following examples assume you are connected to the external project. The SET statements at the start of a session apply to all subsequent queries in that session.

-- Enable schema syntax for this session.
SET odps.namespace.schema=true;
SHOW schemas;

-- Expected output:
-- ID = 20250922********wbh2u7
-- default
--
-- OK

-- Run SET odps.namespace.schema=true first if you have not done so in this session.
SET odps.sql.allow.fullscan=true;
SELECT * FROM <external_project_name>.default.test_tbl;

-- Expected output:
-- +------------+------------+------------+
-- | id         | data       | dt         |
-- +------------+------------+------------+
-- | 1          | CCC        | 2024-07-18 |
-- | 2          | DDD        | 2024-07-18 |
-- | 3          | EEE        | 2025-06-18 |
-- +------------+------------+------------+

INSERT INTO test_tbl PARTITION(dt='2025-08-26') VALUES(4,'FFF');
SELECT * FROM test_tbl;

-- Expected output:
-- +------------+------------+------------+
-- | id         | data       | dt         |
-- +------------+------------+------------+
-- | 1          | CCC        | 2024-07-18 |
-- | 2          | DDD        | 2024-07-18 |
-- | 3          | EEE        | 2025-06-18 |
-- | 4          | FFF        | 2025-08-26 |
-- +------------+------------+------------+

-- Create a schema.
CREATE schema testschema;

-- Create a table in the new schema.
use schema testschema;
CREATE TABLE table_test(id INT, name STRING);

-- Insert data and verify.
INSERT INTO table_test VALUES (101,'Zhang San'),(102,'Li Si');
SELECT * FROM table_test;

-- Expected output:
-- +------------+------------+
-- | id         | name       |
-- +------------+------------+
-- | 101        | Zhang San  |
-- | 102        | Li Si      |
-- +------------+------------+

What's next

• Data type mapping — review how Paimon data types map to MaxCompute types

• Select a connection tool — choose the right tool for your workflow