Realtime Compute for Apache Flink:AI assistant (Beta)

Core capabilities

• Intelligent code generation
  Interprets natural language descriptions of business needs and generates Flink SQL code snippets or complete jobs. Supports declaring source tables, joining dimension tables, performing complex window calculations, and writing to sink tables.
  
  

• Code logic explanation
  For complex or legacy SQL logic, provides line-by-line parsing and translates technical implementations into clear business language, helping you quickly understand data flow and computation logic.
  
  

• Intelligent fault diagnosis
  When a job fails due to runtime exceptions or syntax errors, analyzes error logs, identifies root causes, and provides specific remediation suggestions.
  
  

• Performance optimization suggestions
  Detects potential performance bottlenecks—such as data skew or oversized state—and recommends parameter tuning or SQL rewrites based on Flink best practices.
  
  

Agent mode and Chat mode

The AI Assistant offers two interaction modes, switchable below the input box:

• Agent mode—The assistant actively invokes skills to perform operations. Use this mode for data queries, diagnostics, or other workspace actions.

• Chat mode—Pure conversational Q&A. Use this mode for conceptual questions or SQL syntax explanations that don’t involve system operations.

Resource planning consultation

Before scaling or configuring Flink compute units (CUs), the AI Assistant provides capacity estimates based on your business scale. More precise questions yield better recommendations.

• Data throughput: Specify message rate (TPS/QPS) or data volume (MB/s).

• Computation complexity: Describe key operations (e.g., simple filtering, multi-stream join, large window aggregation).

• Latency tolerance: State acceptable delay (e.g., sub-second, minute-level).

Example:

I need to read from Kafka at approximately 50,000 records/second, with each record sized 1 KB. The logic involves field renaming and simple filtering before writing to Hologres. Please recommend suitable concurrency and CU count.

The AI Assistant returns a two-part response: a Conclusion with recommended concurrency and CU count, and a Rationale covering QPS handling capacity, concurrency settings, external system bottlenecks, and window function impact.

Job drafting assistance

The AI Assistant generates Flink-compliant SQL code as needed.

1. Define source and sink: Clearly describe your input source (Source) and output destination (Sink). Specify the connector type (e.g., Kafka, JDBC, HBase) along with field names and data types.

2. Describe processing logic: Explain required data transformations, including filter conditions, aggregate operations, window types (tumbling, sliding, or session windows), and any necessary joins.

Example:

Write a Flink SQL job. The source is the Kafka topic user_behavior, containing user_id and timestamp. Calculate unique visitors (UV) per 5-minute tumbling window and write results to the MySQL table uv_stats.

The AI Assistant generates a complete Flink SQL job template, including CREATE TABLE statements for source and sink tables with WITH clause configurations. Copy the output directly into the SQL editor.

Job development error analysis

If your job contains invalid statements or errors during development, click the analysis button in the upper-right corner of the error message.

Runtime error log analysis

1. Job fails immediately with no logs.

   In Operation Center > Job Operations, an AI diagnostics icon appears next to failed jobs.

2. Error logs are already available.

   If the job hasn’t failed or the error is a non-fatal exception, click the AI diagnostics button in the upper-right corner of the job logs to analyze them.

Parameter query

1. Identify the connector type.

2. Describe the desired functionality.

Example:

What parameters enable partial column writes in Hologres?

The AI Assistant lists relevant parameters (e.g., sink.partial-insert.enabled) with their purpose, valid values, applicable scenarios, and links to reference documentation.

Resource configuration and parameter tuning

Proper parameter settings are critical for stable job operation. The AI Assistant generates optimal parallelism and memory configurations based on your resource limits (total CUs) and job characteristics.

• Resource budget: Specify your total CU count or core count.

• Job type: Is it compute-intensive (complex UDFs/JSON parsing) or memory-intensive (large windows/deduplication)?

• Tuning objective: Prioritize high throughput, low latency, or job stability.

Example:

I have 10 CUs available for a standard dual-stream join job using two MySQL tables. Recommend TaskManager count, memory allocation, and parameter tuning to ensure high throughput.

The AI Assistant provides a resource allocation plan for TaskManager and JobManager, parameter tuning suggestions (including Mini-batch, state TTL, and Checkpoint settings), and optimization recommendations.