Release notes for EMR V5.2.X

For more information, see SmartData 3.6.X overview.

• The issue that the output of the show create table command based on Data Lake Formation (DLF) metadata is inaccurate is fixed.
• The default parameters of Hive are optimized to improve the performance of Hive jobs.
• In the EMR console, the parameter names on the hive-env tab of the Configure tab for the Hive service are changed to uppercase. This facilitates the use of the parameters.
• The issue that user-defined functions (UDFs) cause HiveServer2 memory leak is fixed.
• The error message that is reported because of the incompatibility between the file system and Hive metastore when you write data to a Hive table is optimized.

• Delta Lake is updated to 0.8.0.
• Spark 3 is supported.

• Hudi is updated to 0.8.0.
• Hudi can be integrated with Spark SQL.

• Delta Lake and Hudi are supported.
• Remote Shuffle Service is supported.
• Livy is supported.
• In the EMR console, the parameter names on the spark-defaults tab of the Configure tab for the Spark service are optimized.
• The cost-based optimization (CBO), dynamic partition pruning, and Z-order features are optimized. The performance of these features is 50% higher than in Spark 3.
• Log Service, DataHub, and Message Queue for Apache RocketMQ can be used as data sources.

• The warning error contained in logs about starting Spark in Ranger is fixed.
• The issue that user information fails to be automatically synchronized after Ranger is connected to a Lightweight Directory Access Protocol (LDAP) server is fixed.

• Knox is adapted to Kudu.
• Knox is adapted to HBase.

• The Cruise Control component can be used to provide the balance feature for Kafka clusters.
• Disks for Kafka clusters are hot-swappable. You can replace a damaged disk without the need to stop the Kafka broker of your cluster.
• The default values of some parameters are changed.