All Products
Search

This Product

    E-MapReduce:Release notes for EMR Serverless Spark - January 26,2026

    Document Center

    E-MapReduce:Release notes for EMR Serverless Spark - January 26,2026

    Last Updated:Mar 26, 2026

    2026-01-26 Version

    This topic describes the release notes for E-MapReduce (EMR) Serverless Spark on January 26, 2026.

    Platform updates

    Feature Description References
    Managed file directories NAS (Network-Attached Storage) file directories are now supported. After you mount a NAS file directory, Notebooks and Data Development tasks can read from and write to files in that directory. A single workspace can manage multiple file directories. Manage files
    You can now mount managed file directories to Spark Executors. All Executors access the mounted files using local paths. Manage Notebook sessions, Develop batch or streaming tasks
    Resource management Switch from pay-as-you-go to subscription billing at the queue level to match your business needs. Switch from pay-as-you-go to subscription
    Reduce pay-as-you-go quotas directly from the console. Edit pay-as-you-go quota limit
    Data catalog After you add a Data Lake Formation (DLF) Catalog to the data catalog, all table formats under that Catalog are natively supported, including Apache Paimon tables and Apache Iceberg tables. Manage data catalog, Use Paimon, Use Iceberg
    Resource monitoring Filter CU\*Hours, CU, and CPU & Memory metrics globally by queue. Download CU\*Hours metric data for resource monitoring.
    Cloud Monitor events Cloud Monitor now supports subscribing to session resource events and task status change events submitted through the Gateway. Subscribe to system event notifications
    Kyuubi Gateway After you add multiple Catalogs to the data catalog, tasks submitted through Kyuubi Gateway can access all added Catalogs by default. Manage data catalog, Manage Kyuubi Gateway
    Best practices Use DuckDB in Notebooks to access DLF data. Use DuckDB in Notebooks

    Engine updates

    The following engine updates apply to esr-5.1.0 (Spark 4.0.1, Scala 2.13), esr-4.7.0 (Spark 3.5.2, Scala 2.12), and esr-3.6.0 (Spark 3.4.4, Scala 2.12).

    Fusion

    • Added support for map_from_entries, shiftrightunsigned, and make_timestamp_ntz.

    • str_to_map now supports last_win.

    • Optimized window last and map_from_entries.

    • Optimized Parquet Write, Commit, Sort operator, and Format Table.

    • Reduced memory usage for the csv/JSON Fusion Reader in compression scenarios.

    • Added support for TimestampNTZType.

    • Improved memory stability.

    • Supports Apache Spark 4.0.

    Lake format

    DLF

    • Optimized table read and write.

    • Added support for OmniCatalog.

    • Added support for passwordless access to pvfs (private virtual file system).

    • Added support for the Lance file format.

    Apache Paimon

    • Added support for passwordless Parquet access.

    • Added support for row-level lineage.

    • Optimized MERGE INTO, compaction, predicate pushdown, and Split.

    • Improved detection of format table configurations.

    • Added support for text format tables.

    Apache Spark framework

    • Added support for Apache Spark 4.0.

    • Added support for Python UDFs.

    • Variant data type is now supported.

    • MC Connector supports max_pt and map_agg.

    • Added support for Fast Fail.

    • Improved Hive compatibility.

    • Added distcp functionality.

    • Added support for SeaTunnel.

    • Optimized OSS restore.

    DataWorks

    • Added support for RDD lineage.

    DuckDB

    • Added support for reading and writing OSS.

    • Added support for reading DLF.

    Apache Celeborn

    Stability improvements

    • Optimized the shuffle read retry mechanism.

    • Optimized shuffle resource allocation.

    • Optimized throttling.

    • Optimized memory fragmentation.