All Products
Search
Document Center

E-MapReduce:Release notes for EMR V3.34.X

Last Updated:Apr 27, 2023

This topic describes the release notes for E-MapReduce (EMR) V3.34.X, including the release date and updates.

Release date

March 15, 2021 for EMR V3.34.0

Updates

ComponentDescription
SmartDataSmartData is updated to 3.4.0.

For more information, see SmartData 3.4.X.

Spark
  • Some default configurations are optimized.

  • Performance is optimized. Window-based top-k queries can be pushed down.

  • The capability of reading data from and writing data to Hive tables in the CSV or JSON format is enhanced.

  • All the column names of a table can be omitted in the ANALYZE statement.

  • LDAP authentication can be enabled or disabled with a click.

  • Spark Beeline is easier to use.

Hive
  • Some default configurations are optimized.
  • Performance is optimized. The cost-based optimization (CBO) feature is enhanced.
  • LDAP authentication can be enabled or disabled with a click.
  • Calcite is updated to 1.12.0.
  • The hive.security.authorization.sqlstd.confwhitelist.append parameter is added.
PrestoLDAP authentication can be enabled or disabled with a click.
YARNThe risk caused by unauthorized access from a Hadoop cluster to the YARN web UI is fixed. If you access the YARN web UI by using SSH Tunnel, you no longer need to explicitly specify user.name in the URL.
ZooKeeperZooKeeper is updated to 3.6.2.
FlinkThe config.sh file is updated during initialization to fix the issue related to the HADOOP_CLASSPATH parameter.
Impala
  • Impala is updated to 3.4.0.
  • Shiro is updated to 1.7.0.
  • Data Lake Formation (DLF) metadata is supported.
  • Data in the Delta format can be queried.
  • LDAP authentication can be enabled or disabled with a click.
TezSome default configurations are optimized.
HASThe issue that prevents the admin.keytab file from being initiated again if a HAS installation error is reported is fixed.
Ranger
  • The issue caused by filter pushdown in Spark is fixed.

  • The issue that prevents Presto from being enabled after you disable Presto in Ranger is fixed.

  • LDAP authentication can be enabled or disabled with a click.

KnoxThe link to access Druid 0.20.0 by using Knox is fixed.
HueLDAP authentication can be enabled or disabled with a click.
Hudi
  • SQL statements can be executed to query data in Hudi tables.

  • The issue that causes the query results on some data to be inaccurate is fixed.

  • Partition pruning is supported if you query data in a Copy on Write table of Hudi by using Spark.

  • The bucket-based index mechanism is supported to improve write performance.

Delta Lake
  • The issue that prevents metadata from being synchronized to a Hive metastore based on existing Delta tables is fixed.

  • The issue that prevents the MERGE statement from parsing asterisks (*) in data is fixed.

  • The issue that causes an error to be reported when you convert data in the Parquet format into a Delta table and create table metadata is fixed.

  • The issue that causes the OPTIMIZE command to fail if no files need to be compacted is fixed.

  • A subquery can be used as the source in the MERGE statement.

  • Data can be cached if you use Presto to query data in a Delta table. This improves query efficiency.

  • Impala can be used to query data in Delta tables.

Superset
  • The issue that prevents the admin user from logging on to the web UI is fixed.

  • Datasets are compatible with Druid clusters.

  • Spark SQL datasets are no longer supported.

SqoopFiles in the Parquet format can be imported to OSS.
AlluxioAlluxio is updated to 2.4.1.
PhoenixField settings are supported by Hive on Phoenix.
PigPig is removed.