Configure Paimon Metadata API Access via VPC and RAM - Data Lake Formation

The DLF metadata API is a standard Apache Paimon interface that gives you programmatic access to DLF catalog metadata. Its software development kit (SDK) depends only on Paimon-related classes, so it won't conflict with your existing dependencies.

Two endpoint types are available:

Endpoint type	When to use	URI format
VPC endpoint (recommended)	Access from within a VPC — lower latency, no public network traffic	`http://<region-id>-vpc.dlf.aliyuncs.com`
OpenAPI endpoint	Cross-region access or local debugging	See Endpoints

Note

The OpenAPI endpoint requires an Apache Paimon 1.4 client to sign requests.

For region IDs and full endpoint addresses, see Endpoints.

Prerequisites

Before you begin, make sure you have:

A DLF service instance activated in the target region
An Elastic Compute Service (ECS) instance (for ECS role authentication) or an access key pair (for direct authentication)
Maven or a compatible Java build tool

Step 1: Configure network access

By default, the metadata API accepts requests only from whitelisted VPCs. When you activate DLF, the system automatically adds your current region's VPC IDs to a user-level whitelist.

To add a new VPC:

Log on to the Data Lake Formation console.
In the left navigation pane, click System and Security.
Click the System Security tab, then click Add VPC ID.
Enter the VPC ID and click OK.

Step 2: Grant DLF permissions

Grant RAM permissions to the ECS role

Log on to the Resource Access Management (RAM) console using an Alibaba Cloud account or as a RAM administrator.
In the left navigation pane, choose Identity Management > Roles, then search for the ECS role. For EMR clusters, the role is AliyunECSInstanceForEMRRole.
In the Actions column, click Add Permissions.
In the Access Policy section, search for and select AliyunDLFFullAccess, then click Confirm Authorization.

Grant catalog permissions to the ECS role

Log on to the Data Lake Formation console.
On the Catalogs page, click the name of a catalog to open its details page.
Click the Permissions tab, then click Grant.

On the authorization page, set the following parameters and click OK.

Note

If the ECS role doesn't appear in the drop-down list, go to System and Security > Access Control > Users and click Sync.

Parameter	Value
User/Role	RAM User/RAM Role
Select Authorization Object	Select the ECS role from the drop-down list
Predefined Permission Type	Data Editor

Step 3: Add the SDK dependency

Add the following dependency to your Maven project:

<dependency>
  <groupId>org.apache.paimon</groupId>
  <artifactId>paimon-api</artifactId>
  <version>1.3.0</version>
</dependency>

Alternatively, download the JAR directly: paimon-api-1.3.0.jar.

Step 4: Connect to the REST API

The following examples show how to initialize a RESTApi client. Choose the authentication method that matches your environment.

Option 1: ECS role (recommended for ECS environments)

Use this method when your application runs on an ECS instance. The SDK retrieves a temporary token from the ECS instance metadata service automatically.

import org.apache.paimon.options.Options;
import org.apache.paimon.rest.RESTApi;

import static org.apache.paimon.options.CatalogOptions.WAREHOUSE;
import static org.apache.paimon.rest.RESTCatalogOptions.DLF_TOKEN_LOADER;
import static org.apache.paimon.rest.RESTCatalogOptions.TOKEN_PROVIDER;
import static org.apache.paimon.rest.RESTCatalogOptions.URI;

public class RESTApiExample {
    public static void main(String[] args) {
        Options options = new Options();
        // VPC endpoint — format: http://<region-id>-vpc.dlf.aliyuncs.com
        options.set(URI, "http://cn-hangzhou-vpc.dlf.aliyuncs.com");
        // The DLF catalog name, not a file path
        options.set(WAREHOUSE, "dlf_test");
        options.set(TOKEN_PROVIDER, "dlf");
        options.set(DLF_TOKEN_LOADER, "ecs");

        RESTApi api = new RESTApi(options);
        System.out.println(api.listTables("my_database"));
    }
}

Option 2: Access key (for local debugging or non-ECS environments)

Use this method for local development or cross-region access via the OpenAPI endpoint. Store credentials in environment variables rather than hardcoding them. For the OpenAPI endpoint URI of your region, see Endpoints.

import org.apache.paimon.options.Options;
import org.apache.paimon.rest.RESTApi;

import static org.apache.paimon.options.CatalogOptions.WAREHOUSE;
import static org.apache.paimon.rest.RESTCatalogOptions.DLF_ACCESS_KEY_ID;
import static org.apache.paimon.rest.RESTCatalogOptions.DLF_ACCESS_KEY_SECRET;
import static org.apache.paimon.rest.RESTCatalogOptions.TOKEN_PROVIDER;
import static org.apache.paimon.rest.RESTCatalogOptions.URI;

public class RESTApiAKExample {
    public static void main(String[] args) {
        Options options = new Options();
        // OpenAPI endpoint — see Endpoints page for the URI of your region
        options.set(URI, "<your-openapi-endpoint-uri>");
        options.set(WAREHOUSE, "dlf_test");
        options.set(TOKEN_PROVIDER, "dlf");
        // Read credentials from environment variables
        options.set(DLF_ACCESS_KEY_ID, System.getenv("DLF_ACCESS_KEY_ID"));
        options.set(DLF_ACCESS_KEY_SECRET, System.getenv("DLF_ACCESS_KEY_SECRET"));

        RESTApi api = new RESTApi(options);
        System.out.println(api.listTables("my_database"));
    }
}

Note

Use a RAM user's access key rather than your Alibaba Cloud account's access key. RAM user credentials are more secure and can be scoped to the minimum required permissions.

Configuration reference

Parameter	Required	Description	Example
`URI`	Yes	The DLF REST Catalog Server endpoint. See Endpoints for all available addresses.	`http://cn-hangzhou-vpc.dlf.aliyuncs.com`
`WAREHOUSE`	Yes	The DLF catalog name — not a storage path.	`dlf_test`
`TOKEN_PROVIDER`	Yes	Fixed value: `dlf`.	`dlf`
`DLF_TOKEN_LOADER`	No	Set to `ecs` to issue temporary tokens via the ECS instance metadata service. Omit this parameter when using access key authentication.	`ecs`
`DLF_ACCESS_KEY_ID`	No	The access key ID for direct authentication. Required when `DLF_TOKEN_LOADER` is not set.	—
`DLF_ACCESS_KEY_SECRET`	No	The access key secret for direct authentication. Required when `DLF_TOKEN_LOADER` is not set.	—

What's next

Endpoints — full list of VPC and OpenAPI endpoint addresses by region