This topic describes the release notes for E-MapReduce (EMR) V3.34.X, including the release date and updates.

Release date

March 15, 2021 for EMR V3.34.0

Updates

Component Description
SmartData SmartData is updated to 3.4.0.

For more information, see SmartData 3.4.X.

Spark
  • Some default configurations are optimized.
  • Performance is optimized. Window-based top-k queries can be pushed down.
  • Hive is compatible with CSV and JSON files that are created in Spark.
  • All the column names of a table can be omitted in the ANALYZE statement.
  • LDAP authentication can be enabled or disabled with a click.

    For more information, see Manage LDAP authentication.

  • Spark Beeline is easier to use.
Hive
  • Some default configurations are optimized.
  • Performance is optimized. The cost-based optimization (CBO) feature is enhanced.
  • LDAP authentication can be enabled or disabled with a click.

    For more information, see Manage LDAP authentication.

  • Calcite is updated to 1.12.0.
  • The hive.security.authorization.sqlstd.confwhitelist.append parameter is added.
Presto LDAP authentication can be enabled or disabled with a click.

For more information, see Manage LDAP authentication.

YARN The risk caused by unauthorized access from a Hadoop cluster to the YARN web UI is fixed. If you access the YARN web UI by using SSH Tunnel, you no longer need to explicitly specify user.name in the URL.
ZooKeeper ZooKeeper is updated to 3.6.2.
Flink The config.sh file is updated during initialization to fix the issue related to the HADOOP_CLASSPATH parameter.
Impala
  • Impala is updated to 3.4.0.
  • Shiro is updated to 1.7.0.
  • Data Lake Formation (DLF) metadata is supported.
  • Data in the Delta format can be queried.
  • LDAP authentication can be enabled or disabled with a click.

    For more information, see Manage LDAP authentication.

Tez Some default configurations are optimized.
HAS The issue that prevents the admin.keytab file from being initiated again if a HAS installation error is reported is fixed.
Ranger
  • The issue caused by filter pushdown in Spark is fixed.
  • The issue that prevents Presto from being enabled after you disable Presto in Ranger is fixed.
  • LDAP authentication can be enabled or disabled with a click.

    For more information, see Manage LDAP authentication.

Knox The link to access Druid 0.20.0 by using Knox is fixed.
Hue LDAP authentication can be enabled or disabled with a click.

For more information, see Manage LDAP authentication.

Hudi
  • SQL statements can be executed to query data in Hudi tables.
  • The issue that causes the query results on some data to be inaccurate is fixed.
  • Partition pruning is supported if you query data in a Copy on Write table of Hudi by using Spark.
  • The bucket-based index mechanism is supported to improve write performance.
Delta Lake
  • The issue that prevents metadata from being synchronized to a Hive metastore based on existing Delta tables is fixed.
  • The issue that prevents the MERGE statement from parsing asterisks (*) in data is fixed.
  • The issue that causes an error to be reported when you convert data in the Parquet format into a Delta table and create table metadata is fixed.
  • The issue that causes the OPTIMIZE command to fail if no files need to be compacted is fixed.
  • A subquery can be used as the source in the MERGE statement.
  • Data can be cached if you use Presto to query data in a Delta table. This improves query efficiency.
  • Impala can be used to query data in Delta tables.
Superset
  • The issue that prevents the admin user from logging on to the web UI is fixed.
  • Datasets are compatible with Druid clusters.
  • Spark SQL datasets are no longer supported.
Sqoop Files in the Parquet format can be imported to OSS.
Alluxio Alluxio is updated to 2.4.1.
Phoenix Field settings are supported by Hive on Phoenix.
Pig Pig is removed.