Resolve the issue that you cannot upload files that exceed 10 MB to DataWorks

The JAR package or resource file that is used in a MapReduce job often exceeds 10 MB. However, the maximum size of a file that you can upload to DataWorks is 10 MB. This topic describes how to resolve this issue to schedule a MapReduce job where a JAR package or resource file that exceeds 10 MB is used.

1. Run the following command on the MaxCompute client to upload a JAR package that exceeds 10 MB:

   --Upload a JAR package.
   add jar C:\test_mr\test_mr.jar -f;

2. Resources that you upload on the MaxCompute client are not displayed on the DataStudio page of the DataWorks console. You must run the following command to check whether the resource is uploaded:

   --View resources.
   list resources;

3. Reduce the size of the JAR package. DataWorks runs a MapReduce job on the computer where the MaxCompute client resides. Therefore, you can submit only the Main function to DataWorks to run a MapReduce job.

   jar 
   -resources test_mr.jar,test_ab.jar --A file can be referenced after it is registered on the MaxCompute client.
   -classpath test_mr.jar --Reduce the size of a JAR package by using the following method: Submit only the Mapper and Reducer that contain the Main function on the gateway. You do not need to submit third-party dependencies. You can store the resources in the wc_in directory of the MaxCompute client.