MaxCompute:FileSystem Catalog external projects

Step 1: Grant permissions to a RAM user

If you are a Resource Access Management (RAM) user, attach the following access policy. For more information about how to add permissions, see Manage RAM user permissions.

• AliyunMaxComputeFullAccess: Grants permissions to create external data sources and external projects.

Step 2: Create a Filesystem Catalog external data source

1. Activate OSS and create a bucket to store Paimon data. For more information, see Quick Start.

2. Log on to the MaxCompute console and select a region in the top-left corner.

3. In the navigation pane on the left, choose Manage Configurations > Foreign Server.

4. On the Foreign Server page, click Create Foreign Server.

5. In the Add Foreign Server dialog box, configure the parameters. The following tables describe the parameters.

   Parameter	Required	Description
   Foreign Server Type	required	Select Filesystem Catalog.
   Foreign Server Name	Required	A custom name. The naming convention is as follows: Must start with a letter and contain only lowercase letters, underscores (_), and digits. Cannot exceed 128 characters. Example: external_fs.
   Foreign Server Description	Optional	Enter the information as required.
   Region	Required	The current region is used by default.
   Authentication And Authorization	Required	Alibaba Cloud RAM role is used by default.
   RoleARN	Required	The Alibaba Cloud Resource Name (ARN) of a RAM role. This role must include permissions to access the OSS service. Log on to the Resource Access Management (RAM) console. In the navigation pane on the left, choose Identities > Roles. In the Basic Information section, you can find the ARN. Example: acs:ram::124****:role/aliyunodpsdefaultrole.
   Storage Type	Required	OSS OSS-HDFS
   Endpoint	Required	Automatically generated. For the China (Hangzhou) region, the Endpoint is oss-cn-hangzhou-internal.aliyuncs.com.
   Foreign Server Supplemental Properties	Optional	Specifies additional properties for the external data source. After you specify these properties, tasks that use this data source can access the source system based on the defined behavior. Note For information about supported parameters, see the official documentation. Specific parameters will be added gradually as the product evolves.

6. Click OK to create the external data source.

7. On the Foreign Server page, find the target data source and click Details in the Actions column.

Step 3: Create an external project

1. Log on to the MaxCompute console and select a region in the top-left corner.

2. In the navigation pane on the left, choose Manage Configurations > Projects.

3. On the External Project tab, click Create Project.

4. In the Create Project dialog box, configure the project information as prompted and click OK.

   Parameter	Required	Description
   Project Type	Required	External project is used by default.
   Region	Required	The current region is used by default and cannot be changed.
   Project Name (Globally Unique)	Required	Must start with a letter and contain only letters, digits, and underscores (_). The length must be 3 to 28 characters.
   MaxCompute Foreign Server Type	Optional	Select Filesystem Catalog.
   MaxCompute Foreign Server	Optional	Use Existing: Lists the created external data sources. Create Foreign Server: Creates and uses a new external data source.
   MaxCompute Foreign Server Name	Required	Select existing: Select a created external data source from the drop-down list. Create external data source: The name of the new external data source is used.
   Bucket Catalog	Required	Select the complete path from the OSS bucket to the file system directory at the catalog level. In this example, the path is oss://paimon-fs/paimon-test/.
   Table Format	(Required)	Paimon is used by default.
   Billing Method	Required	Subscription or Pay-as-you-go.
   Default Quota	Required	Select an existing quota.
   Description	Optional	A custom description of the project.

Step 4: Use SQL to access the data source system

1. Select a connection tool to log on to an external project.

2. List the schemas in the external project. By default, only DB paths that store Paimon tables are listed.

   -- Enable the schema syntax at the session level.
   SET odps.namespace.schema=true;
   SHOW schemas;
   
   -- The following result is returned.
   ID = 20250922********wbh2u7
   default
   
   
   OK

3. List the tables in a schema within the external project.

   -- <schema_name> is the name of the schema displayed in the external project.
   USE SCHEMA <schema_name>; 
   SHOW tables;

4. Create a schema in the external project.

   CREATE schema <schema_name>;
   
   -- Example:
   CREATE schema schema_test;

5. Use the new schema.

   use schema <schema_name>;
   
   -- Example:
   use schema schema_test;

6. Create a table in the schema and insert data.

   • Syntax:

     -- Create a table.
     CREATE TABLE [IF NOT EXISTS] <table_name> 
     (
       <col_name> <data_type>,
       ...
     )
     [COMMENT <table_comment>]
     [PARTITIONED BY (<col_name> <data_type>, ...)] 
     ;
     
     -- Insert data.
     INSERT {INTO|OVERWRITE} TABLE <table_name> [PARTITION (<pt_spec>)] [(<col_name> [,<col_name> ...)]]
     <select_statement>
     FROM <from_statement>

   • Example:

     CREATE TABLE new_table(id INT,name STRING);
     
     INSERT INTO new_table VALUES (101,'Zhang San'),(102,'Li Si');
     
     -- Query the new_table table.
     SELECT * FROM new_table;
     
     -- The following result is returned.
     +------------+------------+
     | id         | name       | 
     +------------+------------+
     | 101        | Zhang San  | 
     | 102        | Li Si      | 
     +------------+------------+