E-MapReduce:Release notes for EMR Serverless Spark on March 3, 2025

Feature

Description

References

Sales

Computing resource plans are provided for pay-as-you-go Spark workspaces. This helps reduce costs. The first time you purchase a resource plan with a capacity of 3,000 CU-hours, you can enjoy a 50% discount.

Notice on the first-purchase discount for EMR Serverless Spark resource plans

Subscription quotas are provided. Subscription quotas are cost-effective and are suitable for scenarios where long-term and stable resource usage is required.

Subscription

Free trial

New users can apply for a resource plan with a capacity of 1,000 CU-hours for free and use the resource plan to get started with extract, transform, and load (ETL) development, task scheduling, and data query and analysis.

Free trial of a 1,000 CU-hour resource plan

Integration with other ecosystems

• Batch and streaming jobs can be connected to the external Ranger service to implement fine-grained permission management.

• Batch jobs can be connected to the external Kerberos service to provide identity authentication and security.

• Configure Ranger authentication for a Spark Thrift Server

• Enable Kerberos authentication

• Kyuubi gateways are supported. You can use a Kyuubi gateway to submit SQL jobs to EMR Serverless Spark by using Beeline and Thrift drivers.

• The details of a submitted Spark job can be viewed on the Applications tab of the specific Kyuubi gateway.

Manage gateways

OpenAPI

APIs for creating workspaces and sessions are available.

• CreateWorkspace

• CreateSessionCluster

Engine version

Description

esr-2.5.1 (Spark 3.3.1, Scala 2.12)

esr-3.1.1 (Spark 3.4.3, Scala 2.12)

esr-4.1.1 (Spark 3.5.2, Scala 2.12)

The ClassNotFound issue and issues related to stack overflow are fixed.

Celeborn

• The push, merge, and split operations are optimized to improve data processing efficiency.

• The Map Range Read feature is optimized to improve data reading performance.