Realtime Compute for Apache Flink:What is Realtime Compute for Apache Flink?

Item

Apache Flink

Realtime Compute for Apache Flink

Why us?

Performance and cost

• No built-in elastic scalability.

• Resource utilization depends on manual tuning.

• Higher performance: Through SQL operator optimization and the self-developed GeminiStateBackend, Realtime Compute for Apache Flink offers performance twice as high as Apache Flink in Nexmark benchmark tests.

• Intelligent performance tuning: Features Autopilot, which automatically monitors and adjusts job resource allocation, solving various performance issues like insufficient job throughput, backpressure in the entire pipeline, and resource waste without manual intervention.

• Higher resource utilization: Supports fine-grained resource (CPU/Mem) configuration at the operator level, improving resource utilization by up to 100% for large-scale jobs.

• Flexible billing: Supports Subscription, Pay-as-you-go and Hybrid billing models for compute unit (CU) consumption.

• Flexible billing models and better performance, helping enterprises reduce costs.

• Intelligent CU scaling improves resource utilization.

Compatibility and integration capabilities

• Native Flink SQL and DataStream APIs.

• Requires manual integration with systems such as MySQL, Kafka, Paimon, with potential compatibility issues caused by frequent version updates.

• Fully compatible with mainstream Flink APIs (SQL, DataStream, PyFlink, and Flink CDC).

• Provides over 30 built-in connectors (such as MySQL, Kafka, Hologres, and Paimon) of various storage types, including databases, message queues, data warehouses, data lakes, and file systems.

• Supports custom connectors to integrate with external systems.

• Lower entry barriers.

• Improved ecosystem integration efficiency.

• Smooth business migration.

Development efficiency and debugging experience

• Lacks a one-stop development management platform.

• Limited debugging tools.

• Supports syncing an entire database, merging and syncing tables, schema evolution, and reading the snapshot from database replicas and incremental data from the master database (CDAS/CTAS).

• Manages draft versions (code comparison and rollback), supporting integration with remote Git repositories (such as GitHub, GitLab, or Gitee).

• Supports native Apache Flink functions and custom functions; provides over 20 Flink SQL script templates for common real-world use cases.

• Supports test data management, quick debugging, intermediate result display, and development-production isolation; integrates with Visual Studio Code's local development tool.

• Easier development.

• Reduced debugging and testing costs.

• Faster job deployment and higher quality.

Operations and management capabilities

• No comprehensive built-in monitoring and alerting system.

• Lacks Graphical User Interface (GUI).

• Requires manually scaling resources; complex resource scheduling.

• Provides a GUI for job management, state monitoring, and log tracking, improving O&M efficiency.

• Supports monitoring metrics in multiple dimensions and intelligent metric aggregation, letting you quickly identify issues such as high latency, skew, and backpressure.

• Offers built-in real-time alerting mechanism, supporting multiple notification methods, including DingTalk, emails, and text messages, and can integrate with enterprise-level monitoring systems like Prometheus.

• Supports dynamic configuration modification without the need for job restarts, and provides intelligent diagnostics to help quickly identify job abnormalities, such as TaskManager disconnections and log locationing and tuning suggestions.

• Lower O&M overhead and optimization difficulty.

• Fine-grained resource management and reduced costs.

• Improved job observability and response efficiency.

Stability and reliability

• Flink cluster deployment has regional limitations.

• Fault tolerance must be manually configured.

• Supports the cross-zone high availability feature in multiple regions to ensure business stability.

• Supports automatic end-to-end fault tolerance, JobManager high availability, eliminating single-points-of-failure.

• Supports checkpoint and savepoint management, state compatibility checks, and data migration to maximize the use of existing states.

Ensures stable operation of large-scale jobs, meeting enterprise-level production environment requirements.

Enterprise-level services

• Users rely on the Apache Flink documentation, community, and other unofficial support.

• No dedicated technical support team.

• Provides 24/7 professional technical support, backed by Realtime Compute for Apache Flink engineers, with a 99.9% SLA.

• Offers rapid response and supports customized features.

• Provides continuous update, ongoing maintenance, and long-term version support.

Professional, trustworthy technical support to accelerate problem resolution and business deployment.

Security and access control

• Basic authentication mechanisms like Kerberos.

• Access control requires integration with external systems.

• Implements Alibaba Cloud's role-based access control.

• Supports tenant-level and project-level isolation of resources and code files for cross-team collaboration.

• Enhances credential security through variable management.

• Supports comprehensive action auditing, which tracks all changes in the production environment.

Offers a unified identity authentication system, ensuring data asset security and compliance.

Extensibility and ecosystem

• Extends functionality through plug-ins.

• Ecosystem depends on the Apache Flink community or developers.

• Supports emerging use cases such as AI and intelligent data analytics.

• Supports integration with data lakes (Iceberg and Hudi) and data warehouses (ClickHouse, Hologres, and MaxCompute).

• Provides SDKs and Restful APIs for secondary development.

Provides a flexible, extensible platform for diverse real-time use cases.