MaxCompute:Usage guide

• Upload: Uploads data from a local file to a MaxCompute table. Each upload operation can write data to a single table or a single partition. For a partitioned table, you must specify the destination partition. For a multi-level partitioned table, you must specify the lowest-level partition. For more information, see Upload.

• Download: Downloads data from a MaxCompute table or the results of a specific instance to a local file. Each download operation can download one table or one partition to a single file. For a partitioned table, you must specify the source partition. For a multi-level partitioned table, you must specify the lowest-level partition. For more information, see Download.

• Resume: If an upload fails because of network issues or Tunnel service errors, you can use the Resume command to continue the file upload. This command resumes the previous upload operation. The Resume command does not currently support download operations. For more information, see Resume.

• Show: Displays information about historical tasks. For more information, see Show.

• Purge: Clears the session directory. By default, this command removes logs from the last three days. For more information, see Purge.

• Help: Displays help information. Short command formats are supported for each command and option.

• Upsert: Writes data using a combination of Update and Insert semantics. This command is supported only for Transaction Table 2.0 tables.

  If matching data is not found in the destination table, new data is inserted. If the data already exists, it is updated.

Before you start, prepare a data file named data.txt and save it to d:\data.txt. The file contains the following data.

shopx,x_id,100
shopy,y_id,200
shopz,z_id

The following steps describe how to upload and download table data.

1. In the MaxCompute client, run the following statements to create the partitioned table sale_detail and add a partition to it.

   CREATE TABLE IF NOT EXISTS sale_detail(
         shop_name     STRING,
         customer_id   STRING,
         total_price   DOUBLE)
   PARTITIONED BY (sale_date STRING,region STRING);
   
   alter table sale_detail add partition (sale_date='201312', region='hangzhou');

2. Use the upload command to upload the data.txt data file to the partitioned table sale_detail.

   • Sample command

     tunnel upload d:\data.txt sale_detail/sale_date=201312,region=hangzhou -s false;

   • Return value

     Upload session: 20230505xxxxxxxxxxxb0b02dbb6bd
     Start upload:d:\data.txt
     Using \r\n to split records
     Upload in strict schema mode: true
     Total bytes:42   Split input to 1 blocks
     2023-05-05 10:11:35     upload block: '1'
     ERROR: column mismatch -,expected 3 columns, 2 columns found, please check data or delimiter

     Note

     Because data.txt contains dirty data, the data import fails. A session ID and an error message are returned.

3. Run the following statement to verify the data.

   • Sample command

     select * from sale_detail where sale_date='201312';

   • Return values

     ID = 20230505xxxxxxxxxxxvc61z5
     +-----------+-------------+-------------+-----------+--------+
     | shop_name | customer_id | total_price | sale_date | region |
     +-----------+-------------+-------------+-----------+--------+
     +-----------+-------------+-------------+-----------+--------+

     Note

     The data import failed due to dirty data, leaving the table empty.

4. Use the show command to query the session ID of the failed upload.

   • Sample command

     tunnel show history;

   • Returned result

     20230505xxxxxxxxxxxb0b02dbb6bd  failed  'upload d:\data.txt sale_detail/sale_date=201312,region=Hangzhou -s false'

5. Modify the data.txt sample data file to contain the following data, which matches the table schema of sale_detail.

   shopx,x_id,100
   shopy,y_id,200

6. Run the resume command to resume the data upload. In this command, 20230505xxxxxxxxxxxb0b02dbb6bd is the session ID of the failed upload.

   • Sample command

     tunnel resume 20230505xxxxxxxxxxxb0b02dbb6bd --force;

   • Return value

     start resume
     20230505xxxxxxxxxxxb0b02dbb6bd
     Upload session: 20230505xxxxxxxxxxxb0b02dbb6bd
     Start upload:d:\data.txt
     Using \r\n to split records
     Upload in strict schema mode: true
     Resume 1 blocks 
     2023-05-05 10:32:39     upload block: '1'
     2023-05-05 10:32:40     upload block complete, block id: 1
     upload complete, average speed is 0 bytes/s
     OK

7. Run the following statement to verify the data. The upload is successful.

   • Sample command

     select * from sale_detail where sale_date='201312';

   • Return value

     ID = 20230505xxxxxxxxxxxx7afc9qcg
      +-----------+-------------+-------------+-----------+--------+
      | shop_name | customer_id | total_price | sale_date | region |
      +-----------+-------------+-------------+-----------+--------+
      | shopx     | x_id        | 100.0       | 201312    | hangzhou|
      | shopy     | y_id        | 200.0       | 201312    | hangzhou|
      +-----------+-------------+-------------+-----------+--------+

8. Run the download command to download the data from the sale_detail table to a local file named result.txt.

   Note

   The naming conventions for the local path path where the data is downloaded are as follows:

   • To save the file directly to the bin directory of the MaxCompute client, set path to filename.extension.

   • To save the file to another path, such as the test folder on drive D, set path to D:\test\filename.extension.

   • If a local file with the same name exists, it is overwritten.

   tunnel download sale_detail/sale_date=201312,region=hangzhou result.txt;

9. Verify the content of the result.txt file. The file contains the following content, which indicates that the download is successful.

   Note

   The downloaded file displays only the values of non-partition fields.

   shopx,x_id,100.0
   shopy,y_id,200.0