Notes and examples for using odpscmd in Cloud Shell - MaxCompute

MaxCompute integrates with Cloud Shell to provide a cloud-based client environment where you can use odpscmd without manual installation. This topic covers usage notes and examples.

Prerequisites

If you log in to odpscmd in Cloud Shell as a RAM user, the RAM user must have the AliyunCloudShellFullAccess permission. For more information, see Resource Access Management (RAM).

Limitations

Note

Cloud Shell is a web-based command-line tool provided by Alibaba Cloud that lets you use odpscmd without manual installation. For more information, see What is Cloud Shell?.

Cloud Shell limitations

Limit	Description
Virtual machines	Only one virtual machine runs at a time. All session windows connect to it automatically.
Session windows	You can open up to five session windows.
Inactivity timeout	After 30 minutes of inactivity or when all session windows close, the session stops. The virtual machine is destroyed 15 minutes later. Restarting Cloud Shell creates a new virtual machine.
Expiration	Each virtual machine has a 1-hour lifetime. After expiration, Cloud Shell destroys it immediately. Restarting Cloud Shell creates a new virtual machine.
File storage	Cloud Shell provides 10 GB of temporary storage in the /home/shell folder. This space resets when the virtual machine is destroyed. For persistent storage, mount a bucket. The bucket loads automatically each time Cloud Shell starts.
Prohibited use	Compute-intensive, network-intensive, long-running tasks, and malicious processes are prohibited. Violations may result in your session being stopped or disabled without notice.

Region limitations
Currently, you can use Cloud Shell to connect to MaxCompute only in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), and China (Shenzhen).

Usage notes

The Cloud Shell service and its temporary storage reside in the China (Shanghai) region by default, regardless of where you start odpscmd. If your project is in a different region, cross-region data transfer occurs, which may pose a data security risk. Use with caution.

Enter the odpscmd environment

Log in to the MaxCompute console and select a region in the upper-left corner.
On the Workspace > Projects page, click Command Line in the top-right corner.

In the Choose Project dialog box, select the project that you want to manage and click OK.

Note

If a prompt about mounting a NAS file system appears, you can mount one as needed. For more information, see (Optional) Mount a NAS file system.

After odpscmd loads in Cloud Shell, the output is similar to the following:

inflating: odpscmd_pub/plugins/cupid/lib/jackson-core-2.9.5.jar
inflating: odpscmd_pub/plugins/cupid/lib/odps-console-cupid-public-0.43.1-public.jar
inflating: odpscmd_pub/plugins/cupid/plugin.ini
inflating: odpscmd_pub/plugins/volume2/lib/odps-console-volume2-0.43.1-public.jar
inflating: odpscmd_pub/plugins/volume2/lib/odps-sdk-volumefs-0.43.1-public.jar
inflating: odpscmd_pub/plugins/volume2/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar
inflating: odpscmd_pub/plugins/volume2/lib/hadoop-annotations-3.3.3.jar
inflating: odpscmd_pub/plugins/volume2/lib/hadoop-auth-3.3.3.jar
inflating: odpscmd_pub/plugins/volume2/lib/hadoop-common-3.3.3.jar
/home/shell
Aliyun ODPS Command Line Tool
Version 0.43.1-public
@Copyright 2020 Alibaba Cloud Computing Co., Ltd. All rights reserved.
Connecting to http://service.odps.aliyun.com/api, project: xxx
Endpoint: http://service.odps.aliyun.com/api
Project: xxx
Schema: default
Quota: default in region N/A
Timezone: Asia/Shanghai
Connected!
xxx (default)>

(Optional) Mount NAS

Cloud Shell provides 10 GB of temporary storage that resets when the virtual machine is destroyed. For persistent storage, mount a NAS file system, which Cloud Shell automatically loads each time you start a session.

In the MaxCompute Cloud CMD window, click the icon and select Mount Storage Space to bind a NAS file system.

Note

Mounting a NAS file system incurs NAS storage fees. Unbind the storage space when you no longer need it. For more information about NAS storage fees, see Billing overview.

Use odpscmd in Cloud Shell

odpscmd in Cloud Shell supports all SQL commands available in the local client. The following examples show how to run SQL commands and transfer data.

Run SQL commands

Create the table result_table1:

CREATE TABLE IF NOT EXISTS result_table1 (
 education STRING comment 'Education level',
 num BIGINT comment 'Number of people');

After the table is created, run the following command to view it:

SHOW TABLES;

Data transfer

Upload a file

Click the icon and select Upload to transfer a local file. This topic uses the result.txt file as an example.

After the upload is complete, press Ctrl+c to exit the Cloud Shell (odpscmd) environment, and then run the ll command to check if the upload was successful and view the file path.

shell@Alicloud:~$ ll
total 24
drwxr-xr-x 3 shell shell 4096 Jul 19 08:21 ./
drwxr-xr-x 1 root  root  4096 Jul 19 08:00 ../
-rw------- 1 shell shell  292 Jul 19 08:22 .bash_history
drwx------ 3 shell shell 4096 Jul 19 08:00 .config/
-rw-r--r-- 1 shell shell  325 Jul 19 08:15 .odpscmd.history
-rw-r--r-- 1 shell shell   30 Jul 19 08:21 result.txt
-rw-r--r-- 1 shell shell    0 Jul 19 08:09 whoami
shell@Alicloud:~$

After confirming the file is present, run the following command to return to the odpscmd environment:

cloudshell-odpscmd -p <project.name>

Upload the result.txt file to MaxCompute:
```
tunnel upload result.txt result_table1;
```

Download a file
1. Download the MaxCompute table result_table1 to Cloud Shell:
```
Tunnel download result_table1 result_table1.txt;
```
2. Click the icon and select Download to save the result_table1.txt file to your local computer.