MaxCompute:Create and use MaxCompute resources

Prerequisites

Before you begin, ensure that you have:

• 已创建数据源并绑定至DataStudio（数据开发）。在目标工作空间的DataStudio（数据开发）模块绑定所需MaxCompute数据源后，才可基于该数据源进行数据开发相关操作。详情请参见创建MaxCompute数据源、开发前准备：绑定数据源或集群。

• A MaxCompute computing resource associated with DataStudio

• A workflow created in DataStudio (DataWorks stores resources inside workflows)

• A node created if you plan to reference a resource in compute code

• (Optional, for OSS uploads) Object Storage Service (OSS) activated, a bucket created, the file stored in that bucket, and your Alibaba Cloud account granted access to the bucket

• 已创建数据源并绑定至DataStudio（数据开发）。在目标工作空间的DataStudio（数据开发）模块绑定所需MaxCompute数据源后，才可基于该数据源进行数据开发相关操作。详情请参见创建MaxCompute数据源、开发前准备：绑定数据源或集群。

• 已创建数据源并绑定至DataStudio（数据开发）。在目标工作空间的DataStudio（数据开发）模块绑定所需MaxCompute数据源后，才可基于该数据源进行数据开发相关操作。详情请参见创建MaxCompute数据源、开发前准备：绑定数据源或集群。

Limits

Resource size

Standard mode workspaces

In a standard mode workspace, commit resources to the development environment and then deploy them to the production environment before production nodes can use them. Development and production environments use separate computing resource configurations — confirm that the configuration for the target environment is correct before querying tables or resources.

Resources added outside DataWorks

DataWorks only manages resources that were created or uploaded through its console. If you added a resource using another tool such as MaxCompute Studio, load it into DataWorks manually using the MaxCompute resource feature in DataStudio before it appears in the resource directory.

Billing

Creating and uploading resources in DataWorks is free. MaxCompute charges for data storage and backup. For pricing details, see Storage fees.

Key concepts

Resource is a concept specific to MaxCompute. Before you can run UDFs or MapReduce jobs, you must upload the required code and dependency files as resources. Resources are stored inside workflows in DataStudio.

The following resource types are supported:

How it works

The end-to-end flow for making a resource available to a node or function:

1. Create or upload the resource in DataStudio.

2. Commit the resource to the development environment.

3. Deploy the resource to the production environment (required for standard mode workspaces).

4. Reference the resource in a node or register it as a function.

Choose an upload method

Select the method based on your file type and size:

Step 1: Create or upload a resource

Create in the DataWorks console

Use this method for Python scripts up to 200 MB or file resources up to 500 KB.

On the resource configuration tab that opens, fill in the resource details and write or paste your code directly in the editor.

For Python resources larger than 200 MB and file resources larger than 500 KB, use the on-premises machine upload or OSS upload method instead. For a walkthrough of creating a Python resource and registering a Python UDF, see Use MaxCompute to query geolocations of IP addresses.

Upload from your on-premises machine

Use this method to upload a resource (up to 500 MB) from your local disk.

On the resource configuration tab, set File Source to upload from your machine and select the local file.

Upload from OSS

Use this method to upload a resource stored in an OSS bucket (up to 500 MB). Follow the on-screen instructions to assign the AliyunDataWorksAccessingOSSRole role to the Alibaba Cloud account you use for the upload.

Step 2: Commit and deploy the resource

After creating the resource, click the icon in the top toolbar of the resource configuration tab to commit it to the development environment.

If nodes in the production environment need to use the resource, also deploy it to the production environment. See Deploy nodes.

Step 3: Use the resource

Reference a resource in a node

After the resource is committed, open the target node and drag the resource name from the resource directory into the code editor. The editor inserts a reference in the following format:

• Standard nodes: @resource_reference{"Resource name"}

• PyODPS 2 nodes: ##@resource_reference{"Resource name"}

PyODPS code depends on third-party packages. Install the required packages using a custom image before running PyODPS code. See Manage images.

Register a resource as a function

Before registering a function, make sure the resource is committed (see Step 2).

Create a MaxCompute UDF following Create and use a MaxCompute UDF. On the function configuration tab, enter the name of the resource.

For information about built-in functions, see Use built-in functions. To view functions in a MaxCompute computing resource and their change history, see Manage MaxCompute functions.

More operations

-- List all resources in the current project
list resources;

-- List all resources in a specific project
use <MaxCompute project name>;
list resources;

What's next

• Create and use a MaxCompute UDF — register a function backed by a JAR or Python resource

• Manage MaxCompute functions — view, update, and delete registered functions

• Resource operations — full command reference for resource management