Use the Spark client - MaxCompute - Alibaba Cloud Documentation Center

After you download the Spark client from this page, you can use the spark-submit command to submit jobs. The following table lists the supported Spark versions. Spark 3 or later is recommended.

Spark-submit

The Spark bin directory contains the spark-submit script, which is used to launch applications on a cluster. This script supports all Spark cluster managers through a unified interface, so no special configuration is required for each one. For more information, see the cluster managers and Spark documentation.

Spark release list

Published Version	Download	Description
Spark 3.5.2	Spark-3.5.2-odps0.49.0	Supports concurrent reads. Provides full type support for vectorized table reads and writes. Supports reading and writing Layer 3 model projects. Supports K8s scheduling. Supports Shuffle Service.
Spark 3.4.2	Spark-3.4.2-odps0.48.0	Supports concurrent reads. Provides full type support for vectorized table reads and writes. Supports reading and writing Layer 3 model projects. Supports K8s scheduling. Supports Shuffle Service.
Spark 3.1.1	Spark-3.1.1-odps0.47.0	Provides full type support for vectorized table reads and writes. Supports reading and writing Layer 3 model projects. Supports reading schema evolution tables. Supports reading OSS/OTS foreign tables. Supports dynamic resource scaling.
Spark 2.4.5	Spark-2.4.5-odps0.47.0	Provides full type support for vectorized table reads. Supports reading schema evolution tables. Supports reading OSS/OTS foreign tables. Supports dynamic resource scaling.
Spark 2.3.0	Spark-2.3.0-odps0.47.0	Provides full type support for vectorized table reads. Improves table read performance. Upgraded the odps-sdk from 0.30.12 to 0.35.9. Supports submitting jobs with an StsAccount. Supports overwrite in local mode. Removed the fastjson dependency. Adds support for more partition key column types. Partition key columns support the STRING, VARCHAR, CHAR, TINYINT, SMALLINT, INT, and BIGINT data types.