EMR on ACK lets you run open-source big data workloads on Alibaba Cloud Container Service for Kubernetes (ACK). ACK manages container lifecycle and service deployment, so your team can focus on big data jobs instead of cluster infrastructure.
Who this is for
If you already use EMR on ECS, create an EMR on ACK cluster to run Spark and Presto jobs alongside your existing workloads. Different applications share a single ACK cluster, and computing resources can be shared across zones.
If you already run big data jobs on ACK, EMR on ACK automates cluster deployment and management for Spark and Presto workloads. Combine EMR on ACK with EMR Shuffle Service to significantly improve Spark job performance.
Feature comparison
Alibaba Cloud E-MapReduce (EMR) offers two deployment models to meet different infrastructure needs.
| Type | Description |
|---|---|
| EMR on ECS | EMR deploys open source Hadoop ecosystem components on Elastic Compute Service (ECS) instances and runs them as services. Manage ECS instances and services in the EMR console, then submit big data jobs to the cluster. |
| EMR on ACK | Deploy an ACK cluster first. EMR then deploys big data service components on ACK resources and runs them in containers. |
Advantages
Cost-effective
Run big data jobs on an existing ACK cluster with minimal configuration—no separate clusters required for big data services. Reuse idle cluster resources to run EMR workloads such as Spark and Presto jobs with a single click. In mixed online and offline workload scenarios, big data and online applications share cluster resources, which enables peak-load shifting and maximizes resource utilization.
Simplified O&M
A single O&M and cluster management system covers both big data and online application services. This simplifies O&M.
Optimized user experience
EMR supports two Infrastructure as a Service (IaaS) models—EMR on ECS and EMR on ACK—with seamless switching between them. ACK and Elastic Container Instance (ECI) deliver elastic computing resources quickly to handle peak computing demands. Adjust the Spark version at the job level to test new features or meet different business requirements without cluster-wide changes.
Deep integration
EMR on ACK uses a cloud-native data lake architecture:
-
Computing: Alibaba Cloud ACK provides unlimited scaling of computing resources.
-
Storage: Alibaba Cloud Object Storage Service (OSS) decouples storage from compute.
-
Metadata: Data Lake Formation (DLF) manages metadata to support your data lake.