Command

Last Updated: Dec 11, 2017

MaxCompute console provides a jar command to run MapReduce job. The detailed syntax is shown as follows:

  1. Usage: jar [<GENERIC_OPTIONS>] <MAIN_CLASS> [ARGS]
  2. -conf <configuration_file> Specify an application configuration file
  3. -classpath <local_file_list> classpaths used to run mainClass
  4. -D <name>=<value> Property value pair, which will be used to run mainClass
  5. -local Run job in local mode
  6. -resources <resource_name_list> file/table resources used in mapper or reducer, seperate by comma
  7. For example:
  8. jar -conf /home/admin/myconf -resources a.txt,example.jar -classpath ../lib/example.jar:./other_lib.jar -Djava.library.path=./native -Xmx512M mycompany.WordCount -m 10 -r 10 in out;

<GENERIC_OPTIONS> includes the following parameters (optional parameters):

  • -conf <configuration file>: speficy the configuration file.
  • -classpath <local_file_list>: the classpath used to specify the local jar package of ‘main’ class (include relative paths and absolute paths). In most cases, users are more accustomed to write the main class and Map/Reduce function in a package, such as WordCount Code Example. Therefore, in the running period of the example program, mapreduce-examples.jar appears in ‘-resources’ parameter and ‘-classpath’ parameter, both have different meaning. ‘-resources’ quotes Map/Reduce function, running in distributed environment while ‘-classpath’ quotes ‘Main’ class, running on the local. The specified path of jar package is also a local path. The package name is separated by system default file separator. (For Windows system, use a semicolon (;), while for Linux system, use a colon (:)).
  • -D <prop_name>=<prop_value> : Multiple Java properties of <mainClass> in local mode can be defined.
  • -local:run MapReduce job in local mode, mainly used for program debugging.;
  • -resources <resource_name_list>: the resource declaration used in MapReduce running time; generally, the resource name in which Map/Reduce function is included must be specified in ‘resource_name_list’. If the user has read other MaxCompute resources in the Map/Reduce function, then these resource names also need to be added in ‘source_name_list’. The resources are separated by commas. If you need use span project resources, you should add the prefix ‘PROJECT/resources/’. For example: -resources otherproject/resources/resfile. About the example how to read the resource in the Map/Reduce function, please see Use Resource Example;User can specify the configuration file ‘JobConf’ by option ‘-conf’. This file can influence JobConf settings in SDK. Next gives an example of configuration file ‘JobConf’:
  1. <configuration>
  2. <property>
  3. <name>import.filename</name>
  4. <value>resource.txt</value>
  5. </property>
  6. </configuration>

In the example mentioned above, a variable named ‘import.filename’ is defined and its value is ‘resource.txt’. User can get this variable value through JobConf interface in MapReduce program. User can achieve the same purpose through JobConf interface in SDK. For more on detailed useage method, please see Use Resource Example.

Example:

  1. jar -resources mapreduce-examples.jar -classpath mapreduce-examples.jar
  2. org.alidata.odps.mr.examples.WordCount wc_in wc_out
  3. add file data/src.txt
  4. jar -resources src.txt,mapreduce-examples.jar -classpath mapreduce-examples.jar
  5. org.alidata.odps.mr.examples.WordCount wc_in wc_out
  6. add file data/a.txt
  7. add table wc_in as test_table
  8. add jar work.jar
  9. jar -conf odps-mapred.xml -resources a.txt,test_table,work.jar
  10. -classpath work.jar:otherlib.jar
  11. -D import.filename=resource.txt org.alidata.odps.mr.examples.WordCount args ...
Thank you! We've received your feedback.