EMR 5.x series release notes - E-MapReduce - Alibaba Cloud Documentation Center

EMR-5.21.x

Release date

Version	Date
EMR-5.21.0	October 27, 2025

What's new

Service	Changes
Hive	Adds a Profile mechanism that automatically detects lake storage file formats, such as ORC, and applies JindoSDK-optimized buffer and pre-read parameters. Introduces an ORC Stripe prefetch mechanism to improve throughput when processing medium-to-large ORC files. This feature overlaps computation and I/O by asynchronously prefetching subsequent Stripes while the current Stripe is being processed. Adds support for ORC vectorized read. Operations such as reading ORC index data or performing predicate pushdown can generate many scattered, non-contiguous file ranges. Vectorized read batches these requests to significantly improve throughput. Integrates the JindoSDK batch metadata API. This feature boosts metadata request throughput by batching requests such as `getFileStatus`.
Spark	Adds a Profile mechanism that automatically detects lake storage file formats, such as ORC, and applies JindoSDK-optimized buffer and pre-read parameters. Introduces an ORC Stripe prefetch mechanism to improve throughput when processing medium-to-large ORC files. This feature overlaps computation and I/O by asynchronously prefetching subsequent Stripes while the current Stripe is being processed. Adds support for parallel pre-opening of small files. This feature automatically detects small-file query scenarios and pre-opens a batch of files in parallel to significantly reduce I/O latency from frequent file opens. Adds support for ORC vectorized read. Operations such as reading ORC index data or performing predicate pushdown can generate many scattered, non-contiguous file ranges. Vectorized read batches these requests to significantly improve throughput.
Tez	Adds support for parallel pre-opening of small files. This feature automatically detects small-file query scenarios and pre-opens a batch of files in parallel to significantly reduce I/O latency from frequent file opens.
Ranger	Jindoauth Server supports custom RAM Roles for clients that access Object Storage Service (OSS). Fixes missing dependencies in `Ranger-yarn-plugin`.
Paimon	Upgraded to version 1-ali-16.3.
JindoCache	Upgraded to version 6.10.1.
Delta Lake	Introduces Delta Lake as a new component, version 3.2.1.

Release information

Data lake cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
Hive	3.1.3
Spark2	2.4.8
Spark3	3.5.3
Tez	0.10.2
Trino	422
Delta Lake	3.2.1
Hudi	0.15.0
Iceberg	1.5.0
Flume	1.11.0
Kyuubi	1.9.2
YARN	3.2.1
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger plugin	1.0.0
DLF-Auth	2.0.2
Presto	0.283
ZooKeeper	3.8.4
Sqoop	1.4.7
Knox	1.5.0
Celeborn	0.5.2
JindoCache	6.10.1
Paimon	1-ali-16.3

OLAP cluster

Service	Version
StarRocks2	2.5.22
StarRocks3	3.2.11
Doris	2.1.4
ClickHouse	23.3.13.6
ZooKeeper	3.8.4

Dataflow cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
YARN	3.2.1
OpenLDAP	2.4.46
ZooKeeper	3.8.4
Knox	1.5.0
Flink	1.17.2
Paimon	1-ali-6.2

Data serving cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger plugin	1.0.0
ZooKeeper	3.8.4
Knox	1.5.0
HBase	2.6.3
JindoCache	6.8.2
Phoenix	5.2.1

Custom cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
Hive	3.1.3
Spark2	2.4.8
Spark3	3.5.3
Tez	0.10.2
Trino	422
Delta Lake	3.2.1
Hudi	0.15.0
Iceberg	1.5.0
Flume	1.11.0
Kyuubi	1.9.2
YARN	3.2.1
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger plugin	1.0.0
DLF-Auth	2.0.2
Presto	0.283
StarRocks2	2.5.22
StarRocks3	3.2.11
ZooKeeper	3.8.4
Sqoop	1.4.7
Knox	1.5.0
Celeborn	0.5.2
Flink	1.17.2
HBase	2.6.3
JindoCache	6.10.1
Paimon	1-ali-16.3
Phoenix	5.2.1

EMR-5.20.x

Release date

Version	Date
EMR-5.20.0	July 10, 2025

What's new

Service	Changes
Hive	Improves performance when adding columns to partitioned tables.
YARN	Improves global scheduling to prevent certain applications from degrading cluster performance.

Component versions

DataLake

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
Hive	3.1.3
Spark2	2.4.8
Spark3	3.5.3
Tez	0.10.2
Trino	422
Hudi	0.15.0
Iceberg	1.5.0
Flume	1.11.0
Kyuubi	1.9.2
YARN	3.2.1
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger-plugin	1.0.0
DLF-Auth	2.0.2
Presto	0.283
ZooKeeper	3.8.4
Sqoop	1.4.7
Knox	1.5.0
Celeborn	0.5.2
JindoCache	6.8.2
Paimon	1-ali-6.2

OLAP

Service	Version
StarRocks2	2.5.22
StarRocks3	3.2.11
Doris	2.1.4
ClickHouse	23.3.13.6
ZooKeeper	3.8.4

DataFlow

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
YARN	3.2.1
OpenLDAP	2.4.46
ZooKeeper	3.8.4
Knox	1.5.0
Flink	1.17.2
Paimon	1-ali-6.2

DataServing

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger-plugin	1.0.0
ZooKeeper	3.8.4
Knox	1.5.0
HBase	2.6.3
JindoCache	6.8.2
Phoenix	5.2.1

Custom

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
Hive	3.1.3
Spark2	2.4.8
Spark3	3.5.3
Tez	0.10.2
Trino	422
Hudi	0.15.0
Iceberg	1.5.0
Flume	1.11.0
Kyuubi	1.9.2
YARN	3.2.1
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger-plugin	1.0.0
DLF-Auth	2.0.2
Presto	0.283
StarRocks2	2.5.22
StarRocks3	3.2.11
ZooKeeper	3.8.4
Sqoop	1.4.7
Knox	1.5.0
Celeborn	0.5.2
Flink	1.17.2
HBase	2.6.3
JindoCache	6.8.2
Paimon	1-ali-6.2
Phoenix	5.2.1

EMR-5.19.x

Release date

Version	Date
EMR-5.19.0	April 24, 2025

Updates

Service	Changes
Trino	Fixed an LDAP availability issue.
YARN	Optimized global scheduling for more efficient resource allocation. Added metrics monitoring for HTTP services. Fixed an open-source bug (YARN-10213).
HBase	Upgraded to 2.6.3. Changed the default runtime environment to Java 11. Set the G1 garbage collector as the default.
Phoenix	Upgraded to 5.2.1.
JindoCache	Upgraded to 6.8.2.
StarRocks	Enabled the creation of compute-storage separated clusters.
EMRHOOK	Added support for Spark 3.5. Enabled data lineage tracking for Paimon tables. Enhanced stability.

Component versions

DataLake cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
Hive	3.1.3
Spark2	2.4.8
Spark3	3.5.3
Tez	0.10.2
Trino	422
Hudi	0.15.0
Iceberg	1.5.0
Flume	1.11.0
Kyuubi	1.9.2
YARN	3.2.1
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger-plugin	1.0.0
DLF-Auth	2.0.2
Presto	0.283
ZooKeeper	3.8.4
Sqoop	1.4.7
Knox	1.5.0
Celeborn	0.5.2
JindoCache	6.8.2
Paimon	1-ali-6.2

OLAP cluster

Service	Version
StarRocks2	2.5.22
StarRocks3	3.2.11
Doris	2.1.4
ClickHouse	23.3.13.6
ZooKeeper	3.8.4

DataFlow cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
YARN	3.2.1
OpenLDAP	2.4.46
ZooKeeper	3.8.4
Knox	1.5.0
Flink	1.17.2
Paimon	1-ali-6.2

DataServing cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger-plugin	1.0.0
ZooKeeper	3.8.4
Knox	1.5.0
HBase	2.6.3
JindoCache	6.8.2
Phoenix	5.2.1

Custom cluster

Service	Version
Hadoop-Common	3.2.1
HDFS	3.2.1
OSS-HDFS	1.0.0
Hive	3.1.3
Spark2	2.4.8
Spark3	3.5.3
Tez	0.10.2
Trino	422
Hudi	0.15.0
Iceberg	1.5.0
Flume	1.11.0
Kyuubi	1.9.2
YARN	3.2.1
OpenLDAP	2.4.46
Ranger	2.3.0
Ranger-plugin	1.0.0
DLF-Auth	2.0.2
Presto	0.283
StarRocks2	2.5.22
StarRocks3	3.2.11
ZooKeeper	3.8.4
Sqoop	1.4.7
Knox	1.5.0
Celeborn	0.5.2
Flink	1.17.2
HBase	2.6.3
JindoCache	6.8.2
Paimon	1-ali-6.2
Phoenix	5.2.1

EMR-5.18.x

Release date

Version	Date
EMR-5.18.1	December 18, 2024
EMR-5.18.0 (Not available for purchase)	December 4, 2024

Updates

Service	Description
Spark 3	Upgraded to 3.5.3. Fixed a configuration issue that occurs during scale-out.
Trino	Fixed connection failures that occur after LDAP is enabled.
Presto	Fixed connection failures that occur after LDAP is enabled.
ZooKeeper	Added support for custom configuration.
Ranger	Replaced the existing Spark 3 Ranger plugin with the one from the open-source Kyuubi project.
Hudi	Upgraded to 0.15.0.
Celeborn	Upgraded to 0.5.2.
Paimon	Upgraded to 1.0-ali-1.
JindoCache	Upgraded to 6.5.3.
StarRocks 3	Upgraded to 3.2.11.
StarRocks 2	Upgraded to 2.5.22.
Impala	These services are no longer available. Use a recommended alternative or install the service manually. For Impala, use Presto, Trino, ClickHouse, or StarRocks.
Kudu
Kafka
Kafka-Manager

EMR-5.17.x

Release date

Version	Release date
EMR-5.17.4	December 18, 2024
EMR-5.17.3 (Not available for new purchases)	November 29, 2024
EMR-5.17.2 (Not available for new purchases)	August 29, 2024
EMR-5.17.1 (Not available for new purchases)	June 21, 2024
EMR-5.17.0 (Not available for new purchases)	April 23, 2024

Updates

EMR-5.17.4

Service	Changes
JindoCache	Upgraded to version 6.5.3.
StarRocks 2	Upgraded to version 2.5.22.
StarRocks 3	Upgraded to version 3.2.11.

EMR-5.17.3

Service	Changes
JindoSDK	Upgraded JindoSDK to fix a deadlock.

EMR-5.17.2

Service	Changes
JindoCache	Upgraded to version 6.5.1. Improved the read and write performance of the distributed hash table (DHT).
Spark	Resolved an issue where partition directories could not be deleted. Resolved a Hive package dependency issue to prevent client operations from interrupting the metastore client connection.
Trino	Resolved an issue that caused modified configurations to be reverted during a scale-out. Added support for querying data on high-security OSS-HDFS. Resolved a service exception caused by enabling DLF-Auth.
Presto	Added support for querying data on high-security OSS-HDFS.
HDFS	Resolved an issue that prevented modifying NameNode and DataNode memory.
HBase-HDFS
YARN	The ResourceManager now supports sending timeline events in batches, improving processing capacity. Resolved a logic issue in the ResourceManager's container and resource processing.
ZooKeeper	Resolved an issue that prevented modifying memory configurations for a node group. Added support for restructuring log configuration files.
Impala	Resolved an issue where auto scaling incorrectly modified customer configurations.
Ranger	Added support for the new JindoSDK kernel, which significantly reduces CPU usage.
Knox	Resolved an issue where the component URL became inaccessible when only one Master-Extend node group existed.
Kafka	Resolved an issue that prevented Kafka Connect clusters from starting.
StarRocks	Resolved an issue where newly added BEs did not appear after a scale-out.
Doris	Upgraded to version 2.1.4.
Paimon	Upgraded to version 0.9-ali-7.
EMR-HOOK	Added support for parsing data lineage from MaxCompute tables.

EMR-5.17.1

Service	Changes
Spark	Added support for deploying Master-Extend node groups.
Hive
Kyuubi
Paimon	Replaced the Flink dependency on the VVR edition with the community edition and added support for DLF Catalog.
Knox	Packaged with JDK 8.
Flink	Restored DLF configurations and dependencies removed in EMR-5.17.0.

EMR-5.17.0

Service	Changes
Spark	Upgraded Spark 3 to version 3.4.2.
Celeborn	Upgraded to version 0.4.0.
Doris	Upgraded to version 2.1.0.
StarRocks	Upgraded StarRocks 2 to version 2.5.18. Upgraded StarRocks 3 to version 3.2.4.
Delta Lake	Upgraded to version 3.0.0.
Iceberg	Upgraded to version 1.5.0.
ZooKeeper	Upgraded to version 3.8.4.
JindoCache	Upgraded to version 6.2.5.
Flink	Upgraded to version 1.17.2.

EMR-5.16.x

Release date

Version	Date
EMR-5.16.0	February 19, 2024

Updates

Service	Update
Hudi	Upgraded to version 0.14.0.
Flume	Upgraded to version 1.11.0.
Kyuubi	Upgraded to version 1.7.3.
Impala	Upgraded to version 4.3.0.
Celeborn	Upgraded to version 0.3.2.
JindoCache	Upgraded to version 6.2.0.
Paimon	Upgraded to version 0.7-ali-1.
Kafka	Upgraded to version 3.6.1.
StarRocks	Upgraded StarRocks2 to version 2.5.13. Upgraded StarRocks3 to version 3.1.5.
Spark	Fixed a Commons Text vulnerability.
Ranger	Vulnerabilities in the Commons Text library are fixed. The path matching permission bypass vulnerability in the Spring Security framework is fixed. The forward/include authentication bypass vulnerability in the Spring Security framework is fixed. The identity authentication bypass vulnerability in a special matching mode in Spring Framework is fixed. The interval at which Ranger obtains user information from the Lightweight Directory Access Protocol (LDAP) server and updates the user information can be modified.

EMR-5.15.x

Release date

Version	Date
EMR-5.15.1	November 16, 2023
EMR-5.15.0 (Not available for purchase)	October 27, 2023

Updates

Service	Description
JindoCache	Added the JindoCache service, version 6.1.1.
JindoData	JindoData is no longer available. You can use the JindoCache service for caching and DLF-Auth for authentication.
Spark	Removed the configurations related to `jdo` from hive-site.xml.
HBase	Added an option to select the HBase Thrift Server version (v1 or v2).
StarRocks	Upgraded StarRocks to version 2.5.10.
Doris	Upgraded Doris to version 1.2.7.
Celeborn	Upgraded Celeborn to version 0.3.1.
Paimon	Upgraded Paimon to version 0.6-ali-2.
ClickHouse	Upgraded ClickHouse to version 23.3.13.6.

EMR-5.14.x

Release date

Version	Date
EMR-5.14.2	August 17, 2023

Updates

Service	Changes
Trino	Fixed an issue that prevented the Paimon connector from querying HDFS tables. Fixed an issue where worker node metrics could not be read.
Presto	Upgraded to version 0.283. Fixed an issue where worker node metrics could not be read.
ClickHouse	The default user is now granted all permissions by default.
StarRocks	Renamed the previous StarRocks service to StarRocks2. Added StarRocks3 (version 3.1.2), which uses a compute-storage integration architecture by default. The compute-storage separation architecture is not yet supported.
Celeborn	Upgraded to version 0.3.0.

EMR-5.13.x

Release date

Version	Release date
EMR-5.13.0	August 3, 2023

Updates

Service	Description
Hudi	Upgraded to 0.13.1.
Paimon	Upgraded to 0.5-ali-1.
StarRocks	Upgraded to 2.5.8.
JindoData	Upgraded to 4.6.11.
Trino	Upgraded to 422. The Hudi connector now supports querying Merge on Read (MOR) tables. Improved error messages for dynamically loading user-defined functions (UDFs).

EMR-5.12.x

Release date

Version	Date
EMR-5.12.1	July 13, 2023
EMR-5.12.0 (Not available for purchase)	June 1, 2023

Updates

EMR-5.12.1

Service	Description
Spark	Spark History Server now uses OSS-HDFS for storage by default. Spark3 Native Engine now supports OSS and OSS-HDFS for storage.
Hive	Hive warehouse now uses OSS-HDFS for storage by default.
OSS-HDFS	Now available as a new service.
YARN	YARN now uses OSS-HDFS for storage by default.
HBase	HBase now uses OSS-HDFS to store HFile data by default. HBase now supports storing write-ahead logs on OSS-HDFS.

EMR-5.12.0

Service	Description
Kyuubi	Upgraded to version 1.7.1.
Celeborn	Upgraded to version 0.2.2.
Paimon	Flink Table Store is renamed to Paimon. Upgraded to version 0.4-ali-1.
StarRocks	Upgraded to version 2.5.5.
Doris	Upgraded to version 1.2.4.
ClickHouse	Upgraded to version 23.3.2.37.
Trino	Trino now includes a default event listener for retrieving audit logs.
Phoenix	Phoenix now supports Hive on Phoenix.

EMR-5.11.x

Release date

Version	Date
EMR-5.11.1	April 3, 2023
EMR-5.11.0 (not available for purchase)	February 28, 2023

Updates

EMR-5.11.1

Service	Changes
ClickHouse	Upgraded to version 22.8.14.53.
Trino	Added the odps.properties connector to query MaxCompute.
JindoData	Upgraded to version 4.6.5.
JindoSDK	Upgraded to version 4.6.5.
Flink Table Store	Upgraded to version 0.3-ali-2.
YARN	Added support for node labels.

EMR-5.11.0

Service	Changes
Iceberg	Upgraded to version 1.1.0.
Hudi	Upgraded to version 0.12.2. Added support for Change Data Capture (CDC).
Delta Lake	Upgraded to version 2.2.0. Now records vacuum operations to the transaction log.
Kudu	Upgraded to version 1.16.0.
ClickHouse	The ClickHouse installation now requires the ZooKeeper service.
Celeborn	Remote Shuffle Service (RSS) is renamed to Celeborn. The initial version is 0.2.0.
Presto	Added Presto as a new service. Its kernel is based on community Facebook PrestoDB 0.278.3. Default ports are 8889 (HTTP) and 7779 (HTTPS).
StarRocks	Upgraded to version 2.5.1.
Doris	Upgraded to version 1.2.1.
Kafka Manager	Upgraded to version 3.0.0.6.
Impala	Upgraded to version 4.2.0.
OpenLDAP	Upgraded to version 2.4.46.
HBase	Added support for JDK 11. Added support for ThriftServer 2. The default value for the hbase.block.data.cachecompressed parameter is now true.
Flink Table Store	Added Flink Table Store as a new service, based on community version 0.3.
JindoData	Upgraded to version 4.6.4.

EMR-5.10.x

Release date

EMR-5.10.0: December 1, 2022

Updates

Service	Description
Iceberg	Upgraded to version 0.14.1.
Flink	Upgraded to Flink 1.15-vvr-6.0.2, based on the Apache Flink 1.15 major version.
Kafka	Supports LDAP authentication and authorization. Supports user group authorization.
Trino	The EMR Presto component has been renamed to Trino to align with the official community name. Supports Ranger and DLF-Auth. Fixed an issue where connections to worker nodes failed after LDAP was enabled.
JindoSDK	Upgraded to version 4.6.2.
JindoData	Upgraded to version 4.6.2.
HBase	Supports Ranger. Fixed an issue where OSS-HDFS could not be selected as the storage mode when adding a service.
YARN	The Access Control List (ACL) is enabled by default in security mode.
StarRocks	Upgraded to version 2.4.1.
Doris	Upgraded to version 1.1.5.
Hudi	You can now configure hudi-defaults.conf in the console.
Ranger	Upgraded to version 2.3.0. Supports integration with Trino, YARN, HBase, and Kafka.
DLF-Auth	Upgraded to version 2.0.2. Supports Trino and Impala.
OpenLDAP	Now integrated with the Nslcd component.
Kudu	You can no longer install Kudu Tserver in task node groups.
Spark	Upgraded to version 3.3.1.
Tez	Upgraded to version 0.10.2.
Kyuubi	Upgraded to version 1.6.0.

EMR-5.9.x

Release date

Version	Date
EMR-5.9.1	November 8, 2022
EMR-5.9.0 (Not available for new purchases)	October 14, 2022

Updates

EMR-5.9.1

Service	Changes
Kerberos	Supports connecting to an external KDC.
Kafka	Adds a configuration option to customize service startup parameters.
JindoData	Upgraded to version 4.6.0. Supports rewriting OSS-HDFS access paths.
Flink	Upgraded to version 1.13_vvr_4.0.15.
RSS	Upgraded to version 0.1.4.

EMR-5.9.0

Service	Changes
Spark	Upgraded to version 3.3. Supports Kerberos authentication.
Hudi	Upgraded to version 0.12.0. Supports Spark 3.3. Lets you use a cloud-based metastore to host metadata and enable the acceleration feature. For more information, see Hudi Metastore Usage Instructions.
Flink	Supports Kerberos authentication. Supports automatic connection to Data Lake Formation (DLF).
Iceberg	Upgraded to version 0.14.0. Supports Spark 3.3. Supports Kerberos authentication.
JindoData	Upgraded to version 4.5.1. Supports accessing Alibaba Cloud resources without using an AccessKey pair.
Hadoop Common and HDFS	Supports Kerberos authentication. Addresses the CVE-2022-25168 vulnerability.
Knox	Integrates with Ranger, allowing access to the Ranger UI from the Access Links and Ports tab.
HBase	Upgraded to version 2.4.9. Supports Kerberos authentication. Supports node group configuration.
RSS	Upgraded to version 0.1.2. Supports Kerberos authentication.
Doris	Upgraded to version 1.1.2. Supports Kerberos authentication.
StarRocks	Upgraded to version 2.3.2. Supports Kerberos authentication.
Kafka	Upgraded to version 2.13_3.2.1. Supports Kerberos authentication.
Delta Lake	You can now upgrade to version 2.1.0. Supports Spark 3.3. Supports Kerberos authentication.
Impala	Supports creating views in Data Lake Formation (DLF). Supports Kerberos authentication.
Kudu	Added as a new component (version 1.14.0).
YARN, Ranger, Hive, Kyuubi, Tez, ZooKeeper, DLF-Auth, Phoenix, Sqoop, and Presto	Supports Kerberos authentication.

EMR-5.8.x

Release date

EMR-5.8.0: August 5, 2022

Updates

Service	Changes
Spark	Supports LDAP authentication.
Hive	Supports LDAP authentication.
Presto	Upgraded to community version 389. Uses standalone Delta Lake and Hudi connectors from the community. The Delta Lake connector does not support time travel or Z-Order. The Hudi connector does not support querying MOR tables. Supports LDAP authentication.
Delta Lake	Integrates with Data Lake Formation (DLF) for automated management of lake tables. Fixes an issue with the automatic synchronization of partition information in CTAS scenarios. The OPTIMIZE and VACUUM commands now return metrics.
Hudi	Upgraded to version 0.11.1.
Hadoop Common	Added the Hadoop Common component to resolve configuration override issues among HDFS, YARN, and JindoSDK.
YARN	Enhanced auto scaling.
Ranger	Supports both Spark 2 and Spark 3. Supports LDAP authentication for Ranger UserSync.
Kafka	Introduces Kafka 2.12-2.4.1 as a new component.
HBase	Introduces HBase 2.3.4 as a new component.
Phoenix	Introduces Phoenix 5.1.2 as a new component.
Doris	Upgraded to version 1.1.1.
StarRocks	Upgraded to version 2.3.0. The primary key model supports full `DELETE WHERE` syntax, persistent primary key indexes to reduce memory usage, and more.
ClickHouse	Upgraded to version 22.3.8.39. Fixes an out-of-memory (OOM) error when reading large files from Object Storage Service.

EMR-5.6.x

Release date

EMR-5.6.0: April 21, 2022

Updates

Service	Changes
JindoData	New component: 4.3.0.
JindoSDK	Upgraded to version 4.3.0.
Spark	Upgraded to version 3.2.1.
Hive	Resolved an issue where duplicate commits occurred in Hive on Tez when speculative execution was enabled.
Presto	Resolved an issue that prevented the Presto service from starting after it was added to an initialized Hadoop cluster.
Delta Lake	DML statements now support subqueries.
Hudi	Upgraded to version 0.10.1.
Iceberg	Upgraded to version 0.13.1.
YARN	Added a configuration to restrict ApplicationMasters to run only on core nodes.
HBase	Fixed a bug in the HBase 2.3.4 kernel.
ZooKeeper	Optimized JVM parameters.
Impala	Added support for JindoSDK 4.3.0.
Sqoop	Upgraded PostgreSQL.
Zeppelin	Resolved an issue that caused the JDBC interpreter to fail on startup.
Ranger	The Ranger 1.2.0 Spark plugin now supports Delta Lake and Hudi.
Flume	Added support for JindoSDK 4.3.0.
Oozie	Upgraded Log4j to version 2.17.2.
DLF-Auth	Upgraded to version 2.0.0.

EMR-5.5.x

Release date

Version	Date
EMR-5.5.1	March 25, 2022
EMR-5.5.0 (not available for new purchases)	February 15, 2022

Updates

EMR-5.5.1

Note

This version is supported only for OLAP clusters in the new console.

Service	Updates
ClickHouse	Updated default parameter values.
StarRocks	Upgraded to version 2.1.1.

EMR-5.5.0

Service	Updates
SmartData	Decommissioned.
Bigboot	Decommissioned.
RSS	Upgraded the EMR Remote Shuffle Service (ESS) to RSS. Enhanced functionality and stability.
JindoSDK	Upgraded the architecture to JindoData. Integrated JindoSDK 4.0, adding support for OSS and the Alibaba Cloud OSS-HDFS service.
Spark	Added support for IF expressions in COUNT DISTINCT and optimized CASE WHEN usage. Set the parameter spark.sql.optimizer.rewriteConditionalDistinctAggregates to true. Enabled fallback from Shuffle Hash Join to Sort Merge Join. Set the spark.sql.join.preferSortMergeJoin parameter to false and the spark.sql.join.enableShuffledHashJoinFallback parameter to true. Enabled automatic merging of small files for non-dynamic partitions. Set the spark.sql.adaptive.merge.output.small.files.enabled parameter to true. Automatically adjusts concurrency for scenarios such as GROUPING SETS and DISTINCT. Set the parameter spark.sql.execution.optimizeExpand to true. Optimized Hive on Spark. Added support for time travel syntax. Added support for JindoSDK.
Tez	Added support for JindoSDK.
Hive	Fixed a batch deletion issue in Hive Jindo. Fixed an out-of-memory (OOM) issue in HiveServer2. Optimized Hive on Spark. Added support for JindoSDK.
Presto	Upgraded Presto to community version 358. Added MySQL, Iceberg, Hudi, Phoenix, Kudu, and Delta connectors by default and updated their default configurations. Added support for data lake analysis. Added support for dynamic loading of UDFs. Added support for JindoSDK.
Delta Lake	Version upgrade Upgraded to version 1.1.0, which is compatible with Spark 3.2.0. Migrated all commercial features to version 1.1.0. Metadata management Optimized the synchronization of metadata modifications to the metastore. Automatically reports table statistics (data profiling) to the metastore. SQL Added support for time travel syntax. Added support for the DROP PARTITION SQL syntax. Added support for overwriting dynamic partitions with SQL. Added support for ADD COLUMN operations at specified positions (FIRST and AFTER). Enhanced table management capabilities Enabled dynamic file size adjustment based on table size. This feature is enabled by default. Enabled automatic and concurrent vacuuming. Automatic vacuuming is enabled by default. Optimized the automatic compaction logic. This feature is disabled by default. Introduced Z-order syntax and accelerated the Z-ordering process.
Hudi	Upgraded to version 0.10.0. Added support for Spark 3.2.0. Added support for JindoFS block mode.
HDFS	Added support for JindoSDK.
YARN	Added support for RSS memory configurations. Added support for JindoSDK.
Flume	Added support for JindoSDK.
Impala	Added support for JindoSDK.
Ranger	Added support for Spark 3.2.0. Added support for Presto 358.
HBase	Fixed incorrect default parameters. Fixed an issue with the date format in garbage collection (GC) logs.
ClickHouse	Added HDFS and OSS disk types to support data tiering. For more information, see Use HDFS for data tiering and Use OSS for data tiering. Engines of the Replicated*MergeTree family now support zero-copy operations for OSS, HDFS, and S3 disk types. Optimized the ClickHouse component shutdown process.
Iceberg	Upgraded to version 0.13.0. Added support for Presto 358.
DLF-Auth	Added support for Spark 3.2.0. Added support for Presto 358.

EMR-5.4.x

Release date

Version	Date
EMR-5.4.3	December 2021
EMR-5.4.2 (Not available for new purchases)	December 2021
EMR-5.4.1 (Not available for new purchases)	November 2021
EMR-5.4.0 (Not available for new purchases)	October 2021

Updates

EMR-5.4.3

This version fixes the Log4j security vulnerability in all related components. For more information, see Security Bulletin | Apache Log4j 2 Remote Code Execution Vulnerability.

Service	Changes
Presto	Fixed the Log4j vulnerability in the Elasticsearch connector.
DLF Metastore	Disabled metastore logging by default. Fixed an error caused by an excessively long getTableStats URI.
Delta Lake	Fixed an issue where schema changes failed to synchronize to the metastore.
Sqoop	Fixed a precision loss issue with the DECIMAL data type when importing data to HCatalog tables.

EMR-5.4.2

Service	Changes
SmartData	SmartData is updated to 3.8.0. For more information, see Introduction to SmartData 3.8.x. Authentication and authorization based on Kerberos and Ranger can be used to manage permissions on data in OSS.

EMR-5.4.1

Service	Changes
SmartData	Upgraded SmartData to version 3.7.3. For more information, see Introduction to SmartData 3.7.x.
Oozie	Fixed an issue that prevented the Oozie Jetty Server from starting in high availability (HA) environments due to dependency JAR conflicts.
Impala	Fixed a `no such method error` that occurred when querying DLF metadata tables.
DLF-Auth	Upgraded DLF-Auth to version 1.0.1.

EMR-5.4.0

Service	Changes
SmartData	Upgraded SmartData to version 3.7.2. For more information, see Introduction to SmartData 3.7.x.
Spark	Upgraded Spark to version 3.1.2. In Spark 3.x, the Distinct computing performance is optimized for Spark SQL. The optimization feature is triggered if an aggregation operator contains multiple `count(distinct case ... when ...)` methods. The array-index out of bounds error that is returned when some required statistics for Adaptive Query Execution (AQE) are missing is fixed. Errors related to AQE and data caching in specific scenarios are fixed.
Hive	Hive on JindoFS (Block) supports batch metadata optimization. This feature is disabled by default.
Presto	Added support for querying Delta Lake tables by using a storage handler.
Delta Lake	Upgraded Delta Lake to version 1.0.0. Unified the delta-connectors for Hive 2 and Hive 3. Fixed an error in delta-connectors when querying multi-level partitioned tables. Added SQL support for features such as Data Skipping, OPTIMIZE, and Z-Ordering. Added support for synchronizing metadata to a metastore.
Hudi	Hudi is updated to 0.9.0. The issue about the compatibility of sql.extension between Delta Lake and Hudi is fixed. Note Supports Spark 3.1.2.
HDFS	The default reserved space for a NameNode now increases adaptively to ensure the NameNode enters safe mode promptly when disk space is low.
Storm	Decommissioned the Storm component.
Zeppelin	Upgraded Zeppelin to community version 0.10.0.
Hue	Fixed an issue that occasionally prevented the YARN Job Browser from displaying or terminating jobs. Enabled the YARN Job Browser in the default configuration. Enabled the Presto protocol in the default configuration.
Druid	The following issue is fixed: After a server is unexpectedly shut down, the related node fails to restart because a PID file is not deleted.
ClickHouse	Updated the default configuration. Added support for cluster scale-out. Added support for the MetaChecker feature. Added support for reading data by using the OSS table engine and OSS table function.
Iceberg	Upgraded Iceberg to version 0.12.0-1.0.1. Fixed Hive runtime dependency errors.
Knox	The issue that the first access to the Spark UI fails is fixed.
DLF-Auth	The service is added. The permissions of using Hive or Spark to access DLF can be configured. The service version is 1.0.0.

EMR-5.3.x

Release date

Version	Date
EMR-5.3.1	September 2021
EMR-5.3.0 (not available for new purchases)	August 2021

Updates

EMR-5.3.1

Service	Changes
SmartData	Upgraded SmartData to version 3.7.1.
Hue	Fixed an issue that prevented Impala from being used in high-security clusters.
Kudu	Added support for Kerberos.
HBase	Fixed an issue that caused excessive restart times for HBase in high-security clusters. Resolved an integration failure with Spark 3.1.1. Optimized the graceful stop process.

EMR-5.3.0

Service	Changes
SmartData	Upgraded SmartData to version 3.7.0.
Spark	Fixed a compatibility issue with Delta Lake.
Hive	Hive on JindoFS (Block Storage Mode) supports batch metadata optimization. This feature is disabled by default.
Delta Lake	Added support for Delta Lake partitioning. Fixed a compatibility issue between the `desc detail` command and Spark 3.1.1.
YARN	Added appId, CPU, and memory resource usage information to the node containers REST API. Fixed an issue that made Application Master (AM) logs inaccessible on nodes released by auto scaling. Fixed an issue where historical data in the State Store caused cluster unavailability. Added support for cleaning up nodes released during an auto scaling decommission. Improved the graceful decommission logic for auto scaling. A decommission is now marked as complete only after the NodeManager (NM) process exits.
ZooKeeper	Upgraded ZooKeeper to community version 3.6.3.
Flink	Added the SmartData component to Flink. Fixed an issue with password-free access to Object Storage Service (OSS) for jobs submitted to a Dataflow Flink cluster via SSH.
Impala	Fixed an infinite loop caused by directly deleting an Object Storage Service (OSS) partition directory.
Hue	Fixed UI display issues with the Hue and Oozie integration.
Kudu	Upgraded Kudu to community version 1.14.0.
ClickHouse	Added the ClickHouse component, version 21.3.13.9.
Iceberg	Added the Iceberg component, version 0.12.0.

EMR-5.2.x

Release date

EMR-5.2.1: July 16, 2021

Updates

Service	Update
SmartData	Upgraded SmartData to version 3.6.1. For more information, see Introduction to SmartData 3.6.x.
Hive	Fixed an issue where the `show create table` command returned incorrect results when you use DLF metadata. Optimized the default Hive parameters to improve job performance. Capitalized configuration item names on the hive-env tab of the Hive service's Configure page in the EMR console for improved usability. Fixed a memory leak in HiveServer2 caused by UDFs (user-defined function). Improved the error message for write operations to a Hive table when the file system and metastore are inconsistent.
HDFS	Adds support for the Zstandard compression format.
Delta Lake	Upgraded Delta Lake to version 0.8.0. Adds support for Spark 3.
Flink	Upgraded Flink to version 1.12-vvr-3.0.2.
Hudi	Upgraded Hudi to version 0.8.0. Adds support for integration with Spark SQL.
Spark	Important In EMR-5.2.1, Spark (3.1.1) is incompatible with Kudu (1.11.1). Adds support for the Delta Lake and Hudi data lake formats. Adds support for Remote Shuffle Service. Adds support for Livy. Improved configuration item names on the spark-defaults tab of the Spark service's Configure page in the EMR console. Optimized features such as cost-based optimization (CBO), dynamic partition pruning (DPP), and Z-order, improving performance by 50% compared with open source Spark 3. Adds support for data sources such as Log Service, DataHub, and Message Queue for Apache RocketMQ (ONS).
Tez	Optimized the default Tez parameters to improve job performance.
Ranger	Fixed warning messages in Spark logs when Ranger is enabled. Fixed user synchronization failures when connected to an LDAP server.
Knox	Adds support for the Kudu component. Adds support for the HBase component.
Kafka	Adds support for the Cruise Control component for Kafka cluster balancing. Disks for Kafka clusters are now hot-swappable. You can replace a damaged disk without stopping the Kafka broker. Changed the default values of some parameters.
Phoenix	Fixed a "JDBC driver not found" error that occurred when you accessed Phoenix tables through Hive or Spark SQL.
EMR Remote Shuffle Service (ESS)	Adds support for Spark 3.