DataWorks:Develop a MaxCompute MapReduce task

Step 1: Upload the JAR resource

Upload, commit, and deploy mapreduce-examples.jar as a MaxCompute resource. See Create and use MaxCompute resources.

Step 2: Write and run the code

Enter the following code on the ODPS MR node and run it.

-- Create an input table.
CREATE TABLE if not exists wc_in (key STRING, value STRING);
-- Create an output table.
CREATE TABLE if not exists wc_out (key STRING, cnt BIGINT);
    --- Create a dual table.
    drop table if exists dual;
    create table dual(id bigint); -- If no dual table exists in the current workspace, create a dual table.
    --- Initialize the dual table.
    insert overwrite table dual select count(*)from dual;
    --- Insert the sample data into the wc_in table.
    insert overwrite table wc_in select * from (
    select 'project','val_pro' from dual
    union all
    select 'problem','val_pro' from dual
    union all
    select 'package','val_a' from dual
    union all
    select 'pad','val_a' from dual
      ) b;
-- Reference the uploaded JAR resource. To reference the resource, find the JAR resource in the resource list, right-click the JAR resource, and then select Reference Resources.
--@resource_reference{"mapreduce-examples.jar"}
jar -resources mapreduce-examples.jar -classpath ./mapreduce-examples.jar com.aliyun.odps.mapred.open.example.WordCount wc_in wc_out

The jar command runs the WordCount class from mapreduce-examples.jar. It reads from wc_in and writes results to wc_out.

When the job completes, the result returned is OK.

Step 3: Verify the results

Query the output table using an ODPS SQL node to confirm the results:

select * from wc_out;

Expected output:

+------------+------------+
| key        | cnt        |
+------------+------------+
| package    | 1          |
| pad        | 1          |
| problem    | 1          |
| project    | 1          |
| val_a      | 2          |
| val_pro    | 2          |
+------------+------------+

Each unique string from wc_in appears as a row in wc_out, with cnt showing how many times it appeared.