All Products
Search

This Product

    DataWorks:CDH MR node

    Document Center

    DataWorks:CDH MR node

    Last Updated:Apr 21, 2026

    In DataWorks, you can create a CDH MR (MapReduce) node to process massive datasets. This topic describes how to configure and use CDH MR nodes in DataWorks.

    Prerequisites

    • You have created an Alibaba Cloud CDH cluster and associated it with a DataWorks workspace. For more information, see New Data Development: Bind a CDH Compute Engine.

    • (Optional, for RAM users) The RAM user for task development has been added to the corresponding workspace and granted the Development or Workspace Administrator role. The Workspace Administrator role has extensive permissions, so grant it with caution. For more information about adding members, see Add members to a workspace.

      Note

      If you use an Alibaba Cloud account, you can skip this step.

    • You have configured a Hive data source in DataWorks and verified that the data source passed the connectivity test. For more information, see Manage data sources.

    Create a CDH JAR resource

    You can upload your task's JAR package to DataWorks and then schedule it to run periodically as a CDH Spark task.

    1. For more information, see Resource management. You can upload the JAR package from your local machine to the storage directory for JAR resources. Click Upload to upload the JAR resource.

    2. Select the storage path, data source, and resource group.

    3. Click Save.

    Create a node

    For instructions, see Create a node.

    Develop the node

    On the editor page of the CDH MR node, perform the following steps.

    1. Open the created CDH MR node and stay on the code editing page.

    2. In the left-side navigation pane, find the resource you want to reference under Resource Management, right-click it, and select Reference Resource.

    3. After the resource is referenced, if a statement in the ##@resource_reference{""} format appears on the code editing page of the CDH node, the resource is referenced successfully. Then, run the following command to execute the task. The resource package, bucket name, and path information in the command are examples. Replace them with your actual information.

    ##@resource_reference{"onaliyun_mr_wordcount-1.0-SNAPSHOT.jar"}
onaliyun_mr_wordcount-1.0-SNAPSHOT.jar cn.apache.hadoop.onaliyun.examples.EmrWordCount oss://onaliyun-bucket-2/cdh/datas/wordcount02/inputs oss://onaliyun-bucket-2/cdh/datas/wordcount02/outputs

    Debug the node

    1. In the Compute Resource section of Run Configuration, configure the compute resource and resource group.

      1. For Compute Resource, select the CDH cluster that you registered in DataWorks.

      2. For Resource Group, select the scheduling resource group that has passed the connectivity test with the data source. For more information, see Network connectivity solutions.

    2. On the toolbar at the top of the node editor, click Run.

    Next steps

    • Configure node scheduling: If you need to run a node periodically, configure its Scheduling Policy in the Scheduling Settings panel on the right.

    • Publish a node: To run a task in the production environment, click the image icon to publish the node. A node runs on schedule only after it is published to the production environment.

    • Task O&M: After a task is published, you can monitor the status of its periodic runs in the Operation Center. For more information, see Get started with Operation Center.