This topic describes how to use MaxCompute Studio to develop a MapReduce program. This includes how to write, debug, package, upload, and run a MapReduce program.
Write a MapReduce program
- In the Project tool window, expand your MaxCompute Java module and choose . Then, right-click java and choose .
- Specify Name, select the Driver type, and then press Enter.
- Name: the name of the MaxCompute Java class. If no package is created, enter packagename.classname. The system automatically creates a package.
- Select the Driver, Mapper, or Reducer type.
- After you create a MaxCompute java class, develop a Java program in the editor.
The Java template is automatically filled with the framework code. You need only to configure the input table, output table, and the Mapper and Reducer classes.
Perform a local run to debug a MapReduce program
Perform a local run to test your MapReduce program and check whether the results meet your expectations.
- Right-click the compiled Java script and select Run.
- In the Run/Debug Configurations dialog box, select the name of the MaxCompute project where the MapReduce program
- Click OK to run the UDF.Note
- The system reads the specified table data in warehouse as the input during the local run. You can view the log output in the console.
- The system downloads the table data from the specified MaxCompute project to the warehouse directory. If the data is already downloaded, the system does not perform this step.
Perform unit testing to debug a MapReduce program
Package and upload a MapReduce program
After you debug a MapReduce program, compress the MapReduce program into a JAR package and upload the package to the MaxCompute server as a resource. For more information, see Package, upload, and register.
Run a MapReduce program
Use the MaxCompute client to run a MapReduce program.
- In the left-side navigation pane, click Project Explorer.
- Right-click the name of your project and select Open in Console.
- In the Console tool window, run the following command to start your MapReduce program. For more commands, see JAR command.
jar-libjars wordcount.jar -classpath D:\odps\clt\wordcount.jar com.aliyun.odps.examples.mr.WordCount wc_in wc_out;