Release notes for EMR V4.8.X

For more information, see SmartData 3.4.X.

• Some default configurations are optimized.
• Performance is optimized. Window-based top-k queries can be pushed down.
• The capability of reading data from and writing data to Hive tables in the CSV or JSON format is enhanced.
• All the column names of a table can be omitted in the ANALYZE statement.
• LDAP authentication can be enabled or disabled with a click.

  For more information, see Manage LDAP authentication.

• Spark Beeline is easier to use.

• Some default configurations are optimized.
• Performance is optimized. The cost-based optimization (CBO) feature is enhanced.
• LDAP authentication can be enabled or disabled with a click.

  For more information, see Manage LDAP authentication.

• The issue caused by filter pushdown in Spark is fixed.
• The issue that prevents Presto from being enabled after you disable Presto in Ranger is fixed.
• LDAP authentication can be enabled or disabled with a click.

  For more information, see Manage LDAP authentication.

For more information, see Manage LDAP authentication.

• Impala is updated to 3.4.0.
• Shiro is updated to 1.7.0.
• Metadata stored in Alibaba Cloud Data Lake Formation (DLF) is supported.
• Data in the Delta format can be queried.
• LDAP authentication can be enabled or disabled with a click.

  For more information, see Manage LDAP authentication.

• The exception that occurs when you use the INSERT OVERWRITE statement to overwrite data stored in OSS is fixed.

• SQL statements can be executed to query data in Hudi tables.
• The issue that causes the query results on some data to be inaccurate is fixed.
• Partition pruning is supported if you query data in a Copy on Write table of Hudi by using Spark.
• The bucket-based index mechanism is supported to improve write performance.

• The issue that prevents metadata from being synchronized to a Hive metastore based on existing Delta tables is fixed.
• The issue that prevents the MERGE statement from parsing asterisks (*) in data is fixed.
• The issue that causes an error to be reported when you convert data in the Parquet format into a Delta table and create table metadata is fixed.
• The issue that causes the OPTIMIZE command to fail if no files need to be compacted is fixed.
• A subquery can be used as the source in the MERGE statement.
• Data can be cached if you use Presto to query data in a Delta table. This improves query efficiency.
• Impala can be used to query data in Delta tables.

• Exceptions in the shuffle read stage, such as ClosedChannelException, IndexOutOfBoundsException, and excessive off-heap memory usage, are fixed.
• The issue that causes NullPointerException (NPE) to be reported after metric monitoring is enabled is fixed.

For more information, see Manage LDAP authentication.

• HBase is updated to 2.2.6.
• Access control based on Ranger is no longer supported.

• The issue that prevents the admin user from logging on to the web UI is fixed.
• Datasets are compatible with Druid clusters.
• Spark SQL datasets are no longer supported.

• Access to Presto by using Knox is supported.
• The issue that causes the Druid web UI to be inaccessible is fixed.
• The limit that you can access the Ranger web UI based on HTTP only by using Knox in high security mode is removed.