Managing production data pipelines at scale is complex — scheduling interdependent tasks, handling failures, and maintaining open-source schedulers all require significant operational overhead. EMR Workflow is a fully managed scheduling service that eliminates this overhead. Fully compatible with open source Apache DolphinScheduler, it provides a serverless environment where you can orchestrate workflows and tasks without deploying or managing any infrastructure.
EMR Workflow is currently in public preview. For details, see Public preview of EMR Workflow.
Benefits
-
No deployment or O&M. EMR Workflow is ready to use out of the box — no scheduler cluster to provision, patch, or maintain.
-
Cloud-native reliability. The service runs on Alibaba Cloud's serverless infrastructure, providing stable and resilient execution without capacity planning.
-
Enterprise security and isolation. EMR Workflow integrates with Resource Access Management (RAM) for role-based access control and enterprise-level isolation.
-
Native big data integration. Connect directly to EMR clusters and popular compute engines — Hive, Spark, and Flink — without custom connectors.
-
Compatible API and low-effort migration. EMR Workflow exposes API operations compatible with Apache DolphinScheduler, so you can migrate existing DolphinScheduler workflows and legacy EMR jobs with a few clicks.
Features
| Feature | Description |
|---|---|
| Multiple task types | Schedule tasks across a range of types, including SHELL, SPARK, and HIVECLI. |
| Visual workflow editor | Build and connect tasks in a visualized manner. See Create a workflow for details. |
| Real-time monitoring | Track task status and view or download logs as workflows run. |
| Scheduled task management | Define cron-based schedules and manage them from a single interface. See Configure a scheduled workflow. |
| Exception-based alerting | Receive automatic notifications when tasks fail or encounter exceptions. See Create an alert instance. |
| Resource management | Create and upload resource files through a visual interface for use in your workflows. See Resource management. |