Run the JAR command to submit a MapReduce job - MaxCompute

Use the jar command in the MaxCompute client to submit a MapReduce job packaged as a JAR file.

Prerequisites

Before you begin, ensure that you have:

Added the required JAR files and resources to your MaxCompute project using the add jar and add file commands
The main class and its dependencies packaged and available locally

Syntax

jar [<GENERIC_OPTIONS>] <MAIN_CLASS> [ARGS];

Placeholder	Description
`<GENERIC_OPTIONS>`	Optional flags that control resource loading, configuration, and execution mode. See Parameters.
`<MAIN_CLASS>`	The main class to run. Example: `com.example.Main`.
`[ARGS]`	Arguments passed to the `main` method of the main class. Optional.

Parameters

The following table describes the available GENERIC_OPTIONS.

Parameter	Description
`-conf <configuration_file>`	Specifies the `JobConf` configuration file, which contains settings equivalent to `JobConf` in the SDK. See Configuration file format.
`-resources <resource_name_list>`	Specifies the resources required for distributed execution on the cluster. List the resources used by your map function, reduce function, or both. Separate multiple resources with commas (`,`). To reference a resource from another MaxCompute project, prefix it with `PROJECT/resources/` — for example, `-resources otherproject/resources/resfile`. If your map or reduce function reads additional MaxCompute resources at runtime, include those in this list as well.
`-classpath <local_file_list>`	Specifies the classpath for running a MapReduce job in local mode. Provide the relative or absolute local paths to the JAR packages containing the main class. Separate multiple paths with semicolons (`;`) on Windows, or commas (`,`) on Linux. If you run a MapReduce job on a cloud server, separate package names with commas (`,`).
`-D <name>=<value>`	Sets a Java property for local execution of the main class. Specify this option multiple times to set multiple properties.
`-l`	Runs the job in local mode, which is useful for debugging before submitting to the cluster.

Distributed execution vs. local execution

-resources and -classpath serve different purposes and are not interchangeable:

Option	Purpose	Execution context
`-resources`	Registers files, JAR packages, and tables with the MaxCompute cluster	Distributed (cluster)
`-classpath`	Specifies local paths used when the MaxCompute client starts the main class	Local (client machine)

If the main class is packaged in the same JAR as your map and reduce functions — as is common in the WordCount example — the same JAR appears in both options but serves a different role in each.

Configuration file format

The -conf option accepts an XML file in the following format:

<configuration>
   <property>
      <name>import.filename</name>
      <value>resource.txt</value>
   </property>
</configuration>

This example defines a variable named import.filename with the value resource.txt. Retrieve the value at runtime using the JobConf interface. For a complete example, see Resource samples.

Examples

The following example shows a typical jar command with multiple options:

jar -conf \home\admin\myconf -resources a.txt,example.jar -classpath ..\lib\example.jar:.\other_lib.jar -D java.library.path=.\native;

Submit a job with a single JAR

Add the JAR to the MaxCompute project, then submit the job:

add jar data\mapreduce-examples.jar;
jar -resources mapreduce-examples.jar -classpath data\mapreduce-examples.jar
    org.alidata.odps.mr.examples.WordCount wc_in wc_out;

mapreduce-examples.jar appears in both options: -resources makes it available to the map and reduce functions on the cluster, and -classpath lets the client locate the main class locally.

Submit a job with a file resource

Add a file and a JAR to the MaxCompute project, then submit the job:

add file data\src.txt;
add jar data\mapreduce-examples.jar;
jar -resources src.txt,mapreduce-examples.jar -classpath data\mapreduce-examples.jar
    org.alidata.odps.mr.examples.WordCount wc_in wc_out;

Both src.txt and mapreduce-examples.jar are registered with the cluster via -resources. The -classpath points only to the JAR containing the main class on the local machine.

Submit a job with a table resource and a configuration file

Create a table alias, add all resources to the MaxCompute project, then submit the job with a configuration file and a custom Java property:

add file data\a.txt;
add table wc_in as test_table;
add jar data\work.jar;
jar -conf odps-mapred.xml -resources a.txt,test_table,work.jar
    -classpath data\work.jar:otherlib.jar
    -D import.filename=resource.txt org.alidata.odps.mr.examples.WordCount args;

-conf odps-mapred.xml loads a JobConf configuration file. -D import.filename=resource.txt overrides a single property for local execution. test_table is a table alias created from wc_in using add table.

What's next

WordCount samples — A complete end-to-end MapReduce example showing how to write, package, and submit a WordCount job.
Resource samples — Examples of reading file, JAR, and table resources in map and reduce functions.