DataWorks:Create and use MaxCompute resources

Limits

• Resource size

  You can directly upload a resource whose size is a maximum of 200 MB to DataWorks. For more information, see Manage MaxCompute resources.

• Resource deployment
  If you use a workspace in standard mode, you need to deploy resources to the production environment. This way, the resources can be used by projects in the production environment.

  Note The information about a compute engine varies based on the environment of the workspace with which the compute engine is associated. You must be clear about the information about the compute engine that is associated with the workspace in the environment. This ensures that you can query valid table and resource data in subsequent operations. For information about the information about a compute engine that is associated with a workspace in a specific environment, see Associate a MaxCompute compute engine with a workspace.
• Resource management

  DataWorks allows you to view and manage resources that are uploaded by using the DataWorks console only in the DataWorks console. If you add resources to a MaxCompute compute engine by using other tools such as MaxCompute Studio, you must use the MaxCompute Resources feature in DataWorks DataStudio to manually load the resources to DataWorks. You can view and manage the resources in DataWorks after the loading is complete. For more information, see Manage MaxCompute resources.

Create resources or upload existing resources

DataWorks allows you to create resources or upload existing resources. You can select a method based on the GUIs for each type of resource.

1. Go to the DataStudio page.
   1. Log on to the DataWorks console.
   2. In the left-side navigation pane, click Workspaces.
   3. In the top navigation bar, select the region in which the workspace that you want to manage resides. Find the workspace and click DataStudio in the Actions column.
2. Create a resource.
   You can create the desired type of resource in the desired workflow based on your business requirements. The following figure shows the entry points of creating resources and the creation procedures.

   Note If no workflow is available, create one. For information about how to create a workflow, see Create a workflow.
3. Configure information about the resource.
   Configure information about the resource based on your business requirements. In this topic, a Python resource is created. Configuration items vary based on the type of the resource that you create.

   Note

   • If you create a JAR resource and the JAR resource is never uploaded to the MaxCompute client, you must select Upload to MaxCompute. If the JAR resource has been uploaded to the MaxCompute client, clear Upload to MaxCompute. Otherwise, an error is reported when you upload the JAR resource.
   • The resource name can be different from the name of the uploaded file.
   • The name of a JAR resource must end with .jar. The name of a Python resource must end with .py.
4. Optional:Write code for the resource.
   If you upload an existing resource, skip this step.

   The following code provides an example of the code that is written in the Python resource. You can replace the code based on your business requirements.

   from odps.udf import annotate
   @annotate("string->bigint")
   class ipint(object):
       def evaluate(self, ip):
           try:
               return reduce(lambda x, y: (x << 8) + y, map(int, ip.split('.')))
           except:
               return 0

5. Commit and deploy the resource.
   Click the icon in the top toolbar to commit the resource to the development environment.

   Note If nodes in the production environment need to use this resource, you also need to deploy the resource to the production environment. For more information, see Deploy nodes.

Enable a node to use the resource

The following figure shows the reference steps.

Use resources to register functions

For information about MaxCompute built-in functions, see Functions.

For information about how to view the functions in a MaxCompute compute engine and the change history of the functions, and perform other operations, see MaxCompute functions.

Other resource management operations

• Delete a resource

  Only the resource that is used by the compute engine that is associated with a workspace in the development environment can be deleted. If you want to delete the resource from the workspace in the production environment, you must deploy the resource deletion operation in the development environment to the production environment to make the deletion take effect in the production environment. After the operation is deployed, the resource can be deleted from the workspace in the production environment. For more information, see Deploy nodes.

• Compare resource versions and roll back a resource version
  You can click Versions to view the saved or committed resource versions, and compare the changes on the resource between different versions.

  Note When you compare resource versions, you must select at least two versions for comparison.

  

Appendix 1: View resources used by a compute engine by using commands

Appendix 2: Add compute engine resources to DataWorks for management

You can use the MaxCompute Resources feature in DataStudio to load a MaxCompute compute engine resource whose size is no more than 200 MB to DataWorks for visualized management. For more information, see Manage MaxCompute resources.