MaxCompute:Connect using the local client (odpscmd)

Pricing

Connecting to a MaxCompute project using the client is free. However, operations you run through the client may incur MaxCompute charges. For example, submitting an SQL query or a write command consumes computing resources. Writing data also uses storage space. You will be charged for both computing and storage. For details, see Billing items and methods.

Install and configure the MaxCompute client

Starting with v0.27.0, the client supports new MaxCompute V2.0 data types. We recommend using these new types. For a list of supported data types, see MaxCompute V2.0 data types.

Follow these steps:

1. Download the MaxCompute client package (GitHub).

   Note

   • Go to the client release page using the link above. Download the latest MaxCompute client package (odpscmd_public.zip).

   • If the link above fails, try downloading the MaxCompute client package (OSS). For GitHub access issues, search for solutions in a search engine.

2. Extract the downloaded package. You get the bin, conf, lib, and plugins folders.

3. Go to the conf folder and configure the odps_config.ini file.

   The odps_config.ini file uses the hash symbol (#) for comments. The following table describes the parameters.

   Parameter	Required	Description	Example
   project_name	Yes	The name of the MaxCompute project you want to access. If you created a workspace in standard mode, make sure to distinguish between production and development (_dev) project names when configuring project_name. See Differences between workspace modes. Log on to the MaxCompute console, and select a region in the upper-left corner. In the navigation pane on the left, choose Manage Configurations > Projects. On the Projects page, find your MaxCompute project name.	doc_test_dev
   access_id	Yes	Your Alibaba Cloud account or RAM user AccessKey ID. Go to the AccessKey management page to get your AccessKey ID.	None
   access_key	Yes	The AccessKey secret that matches your AccessKey ID.	None
   end_point	Yes	The endpoint for the MaxCompute service. Set this based on the region and network type you selected when creating your MaxCompute project. For endpoints by region and network, see Endpoint. Important This endpoint is for the MaxCompute service. Use the Tunnel endpoint for the MaxCompute Tunnel service. Enter the MaxCompute endpoint here. An incorrect endpoint causes connection errors. Double-check your entry.	http://service.cn-hangzhou.maxcompute.aliyun.com/api
   log_view_host	No	The LogView URL. We recommend setting this parameter. Without it, you cannot quickly locate issues when jobs fail. You can use this URL to view detailed job runtime information and troubleshoot errors. The value is fixed: http://logview.odps.aliyun.com.	http://logview.odps.aliyun.com
   https_check	No	Whether to enable HTTPS to encrypt requests to your MaxCompute project. Valid values: True: Use HTTPS. False: Use HTTP. Default: False.	True
   data_size_confirm	No	The maximum input data size in GB. No limit applies. We recommend 100 GB.	100
   update_url	No	A reserved parameter. Ignore it for now.	None
   use_instance_tunnel	No	Whether to use InstanceTunnel to download SQL query results. Valid values: True: Downloads the SQL execution results using InstanceTunnel. False: InstanceTunnel is not used to download SQL execution results. Default: False.	True
   instance_tunnel_max_record	No	The maximum number of records returned for SQL query results. Set this if use_instance_tunnel is True. Maximum: 10000.	10000
   tunnel_endpoint	No	The public endpoint for the Tunnel service. If you do not set a Tunnel endpoint, Tunnel automatically routes to the Tunnel endpoint for your MaxCompute service's region and network. If you set a Tunnel endpoint, Tunnel uses that endpoint and does not auto-route. For Tunnel endpoints by region and network, see Endpoint.	http://dt.cn-hangzhou.maxcompute.aliyun.com
   set.<key>	No	Set properties for your MaxCompute project. For more properties, see Property list.	set.odps.sql.decimal.odps2=true

   Make sure all settings are correct. Incorrect settings cause connection failures.

Run the MaxCompute client

1. Create a MaxCompute project.

2. If you use a RAM user to run the MaxCompute client, add that RAM user to your target MaxCompute project using your Alibaba Cloud account. For more details, see Grant permissions to other users.

3. Start the MaxCompute client in one of the following ways:

   Method 1: Script files in the installation package

   In the bin folder under your MaxCompute client installation path, double-click odpscmd.bat (Windows) or odpscmd (macOS). This starts the MaxCompute client. You see the following message when the connection succeeds.

   Method 2: System command-line window

   In your system command-line window, go to the bin directory under your MaxCompute client installation path. Then run odpscmd (Windows) or sh odpscmd (Linux or macOS). This starts the MaxCompute client. You see the following message when the connection succeeds.

   Note

   On Ubuntu, running sh odpscmd returns an error. Try ./odpscmd instead.

   

   If you start the MaxCompute client from the system command line, you can pass arguments to run commands. For more arguments, see Startup parameters.

Related operations

...\odpscmd\bin>odpscmd -h

odps@project_name>whoami;

odps@project_name>quit;
-- Equivalent to:
odps@project_name>q;

References

After logging on to the MaxCompute client, you can run SQL commands in your MaxCompute project. See Use MaxCompute with the client.

For full command syntax, see Common commands or SQL commands and functions.

FAQ

After you configure the odps_config.ini file and start the MaxCompute client, common errors include the following:

Startup parameters

Usage: odpscmd [OPTION]...
where options include the following:
    --help                                  (-h)for help
    --config=<config_file>                  specify another config file
    --project=<prj_name>                    use project
    --endpoint=<http://host:port>           set endpoint
    -k <n>                                  will skip begining queries and start from specified position
    -r <n>                                  set retry times
    -f <"file_path;">                       execute command in file
    -e <"command;[command;]...">            execute command, include sql command

In a shell or Windows command prompt, you might need to capture the dynamic return value of odpscmd -e <SQL statement> in a shell variable. Then use that variable in follow-up tasks. To avoid extra output such as runtime information or table headers, set the odps_config.ini parameter use_instance_tunnel to false. This disables Instance Tunnel. Then run set odps.sql.select.output.format={"needHeader":false,"fieldDelim":" "}; to hide table headers.

-- Run in Windows command prompt.
...\odpscmd\bin>odpscmd -e "set odps.sql.select.output.format={""needHeader"":false,""fieldDelim"":"" ""};select * from noheader;" >D:\test.txt
-- Output saves to D:\test.txt.

-- Run in shell.
/Users/.../odpscmd/bin/odpscmd -e "set odps.sql.select.output.format={\"needHeader\":false,\"fieldDelim\":\"\"};select * from noheader;" >/Users/A/temp/test.txt 
-- Output saves to /Users/A/temp/test.txt.
-- Output:
1
2
3

Version history

The following table lists recent updates to the MaxCompute client. Click each version link for full details.

For more versions and feature updates, see GitHub.