This topic describes the features and benefits of fully managed Flink of Realtime Compute for Apache Flink and compares this Alibaba Cloud service with Apache Flink.
Category | Item | Description | Benefit |
Performance and cost | Core performance |
| Fully managed Flink provides better engine performance and finer-grained resource allocation, which reduces the total cost of ownership (TCO) compared with that of Apache Flink. Fully managed Flink also supports multiple billing methods and intelligent auto scaling. This helps you use resources in a more fine-grained manner. |
Resource utilization | Allows you to automatically scale your clusters in response to business loads. | ||
Supports the Autopilot feature. After this feature is enabled, the system can automatically monitor and adjust deployment resources and apply different resource plans for different scenarios. This helps you scale to meet the requirements of business peaks while keeping costs within a reasonable range. | |||
Allows you to manage resources in a fine-grained manner and supports fine-grained resource configurations (CPU cores and memory) at the SQL operator level. This improves the resource utilization of large-scale deployments by 100%. | |||
Billing method | Provides the subscription and pay-as-you-go billing methods, which you can choose based on your business requirements. | ||
Featured capabilities | Real-time data ingestion into data lakes or data warehouses | Supports real-time synchronization of data from a database, real-time synchronization of data from tables in shared databases, and real-time synchronization of schema changes. | This feature allows you to ingest data in business databases or message-oriented middleware that uses table sharding into data lakes or data warehouses in real time. |
Real-time fraud detection | Supports enterprise-class complex event processing (CEP), and allows you to dynamically configure rules for a deployment without having to restart the deployment. This delivers uninterrupted production-level capabilities in mission-critical scenarios, such as online real-time fraud detection. | This feature is suitable for mission-critical scenarios, such as real-time marketing, real-time fraud detection, Threat Detection Service (TDS), to improve development efficiency and large-scale data processing capabilities, while ensuring business continuity. | |
Upstream and downstream connectors |
| You do not need to develop or connect to various upstream and downstream ecosystems. The connectors help ensure system stability and performance. | |
Development efficiency | Develop a draft | Programming languages: Fully managed Flink provides an end-to-end development and management platform, which supports various programming languages such as SQL, Java, Scala, and Python. | You do not need to build a development environment or connect to Apache Flink. Flink SQL is easy to use in the overall development environment. |
Multi-version support: Fully managed Flink supports mainstream Flink versions. You can compare deployment code between different versions and perform rollback operations on the code. | |||
Metadata management: You can create catalogs to connect fully managed Flink to common upstream and downstream components, such as MySQL, Hive, Hologres, Data Lake Formation (DLF), and Kafka. This helps you manage and use metadata in a unified manner. | |||
User-defined functions (UDFs): You can easily manage and use UDFs. | |||
Code templates: More than 20 templates that are used in common scenarios of Flink SQL are provided to help you quickly understand how to use Flink SQL to build deployment code. | |||
Code debugging | Test data management: Fully managed Flink supports online sampling and management of mock testing data to help you build test processes. | Programmers and even data analysts can debug and publish drafts. This significantly reduces the costs for debugging and testing, and reduces the time required to publish drafts. | |
Fast deployment and debugging: Fully managed Flink allows you to start or cancel deployments in session clusters within seconds. This makes deployment debugging more efficient. | |||
Display of intermediate results: Intermediate results can be displayed. This improves the efficiency of debugging complex SQL statements. | |||
Environment isolation: The development environment is isolated from the production environment. This way, deployments and data in the production environment are not affected during the debugging process. | |||
Deployment operations | Monitoring and alerting | Provides various metrics and aggregate dimensions to help you resolve issues, such as deployment delays, data skew, and backpressure. | These features significantly improve system stability, reduce the O&M workload, and simplify tuning operations. They allow you to manage resources in a fine-grained manner to significantly reduce costs. Furthermore, Alibaba Cloud provides high availability assurances for the service. |
Sends alerts to recipients over DingTalk or by email, text message, or phone at the earliest opportunity. You can also connect fully managed Flink to your internal unified monitoring and alerting system Prometheus. | |||
Issue analysis and diagnostics | Allows you to dynamically modify the configuration of a deployment without the need to cancel the deployment. For example, you can change the log level and enable or disable the flame graph without canceling the deployment. | ||
Provides intelligent diagnostics for common issues, such as backpressure, deployment exceptions, and TaskManager discontinuity. It is also capable of quickly identifying issues based on logs, and providing suggestions for tuning and modifications. You can also enable Autopilot to automatically locate issues. | |||
High availability | Delivers Service Level Agreement (SLA)-guaranteed service availability of 99.9% for the maintenance service. | ||
Supports end-to-end automatic fault recovery and fault-tolerant JobManager to prevent single points of failure. This improves service stability. | |||
Supports fast single-node fault recovery to balance between data consistency and service continuity. | |||
Status management | Provides complete system checkpoints and savepoint lifecycle management of deployments, and supports state data compatibility checks and state data migration to maximize the reuse of original state data. | ||
Enterprise security | Isolation | Supports tenant-level and project-level resource isolation and code isolation to meet the data security requirements when different teams collaborate on projects. | These features support the collaboration between multiple departments within an enterprise and meet the internal and external audit requirements of the enterprise. |
Access control | Uses the Alibaba Cloud account system to support access control of multiple roles. |