MaxCompute:Access the public network

How it works

Enabling public network access requires two actions:

1. Allowlist the target endpoint in MaxCompute. Add the public IP address or domain name (with port) to the MaxCompute External Network configuration in the console. For addresses outside the supported top-level domains, submit a request form for manual review.

2. Set the access parameter in your job. Each job type uses a specific configuration parameter to declare which endpoints the job can reach: odps.internet.access.list for SQL UDFs and MaxFrame; dedicated Spark parameters for Spark jobs.

Optionally, if the target server enforces access control, add MaxCompute's egress IP addresses to its allowlist so it accepts inbound connections from MaxCompute.

Prerequisites

Before you begin, ensure that you have:

• A MaxCompute project. In a data lakehouse scenario, set the data type to a Hive-compatible type

• (VPC targets only) Confirmed that the VPC owner account, the Alibaba Cloud account used to access the MaxCompute project, and the administrator account of the target service environment or cluster all belong to the same root account

Step 1: Add external network addresses in the console

For frequently used public IP addresses or domain names (for example, aliyun.com), add them directly in the MaxCompute console.

1. Log on to the MaxCompute console and select a region in the upper-left corner.MaxCompute console

2. In the navigation pane on the left, choose Manage Configurations > Projects.

3. On the Projects page, find the target project and click Manage in the Actions column.

4. On the Project Settings page, click the Parameter Configuration tab.

5. In the MaxCompute External Network section, click Edit and enter the external network addresses that MaxCompute is allowed to access.

6. Click Submit.

If an address fails automatic verification:

1. Delete the address and resubmit it. If the failure persists, submit a request form to add the IP address or domain name and port to the allowlist.

2. The MaxCompute technical support team reviews the request and completes the configuration, typically within three business days. If you have any objections to the review result, you can submit a ticket.

3. After the request is approved, proceed to Step 2.

For more information, see Network activation process.

Step 2: Configure your job

Add the external network configuration to your job before running it. The configuration syntax depends on the job type.

-- Specify the public IP address (or domain name) and port to access.
-- To access multiple endpoints, separate entries with commas.
SET odps.internet.access.list=<ip_address:port|domain_name:port>;

-- Call the UDF.
SELECT <UDF_name>("<http://ip_address|domain_name>");

package com.aliyun.odps.test.udf;
import com.aliyun.odps.udf.UDF;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;

public class url_fetch extends UDF {
    public String evaluate(String urlStr) throws IOException {
        URL url = new URL(urlStr);
        StringBuilder sb = new StringBuilder();
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                sb.append(line).append('\n');
            }
        }
        return sb.toString();
    }
}

SET odps.internet.access.list=www.aliyun.com:80;
SELECT url_fetch("http://www.aliyun.com");

spark.hadoop.odps.cupid.smartnat.enable = true
spark.hadoop.odps.cupid.internet.access.list=<ip_address:port>

from maxframe import options

options.sql.settings = {
    "odps.internet.access.list": "<host>:80,<host>:443",
}

(Optional) Step 3: Add MaxCompute egress IP addresses to the target server

If the target server enforces its own access control (for example, a firewall allowlist), add MaxCompute's egress IP addresses to the server's allowlist so it accepts inbound connections from MaxCompute jobs.

To get the current list of MaxCompute egress IP addresses, refer to the Project Management documentation or contact MaxCompute technical support.

Limitations