You can view the performance of a running deployment, including the CPU, memory, and thread usage of its JobManager and running TaskManagers. This helps you identify code-level issues, such as slow class initializations or classes that consume excessive resources.
Prerequisites
If you share a workspace with other Alibaba Cloud accounts or RAM users, you must grant them the necessary permissions to view deployment performance. For more information, see Development console authorization.
Limitations
-
Viewing deployment performance is supported only for deployments that run on Ververica Runtime (VVR) 4.0.11 or later.
-
This feature is available only for running deployments. You cannot view the performance of historical deployments.
Analysis tool usage
Flame Graph
Because a Flame Graph is generated from samples, it may not reflect the entire program execution. Always use it with other analysis tools and tailor your analysis to your specific use case. You can typically check the following aspects:
-
CPU time: Functions that occupy a wider area at the top of the graph usually consume more CPU time, indicating a potential performance issue.
-
Memory allocation: View the memory allocated by different classes.
-
Lock contention: Check for performance degradation caused by lock contention or deadlocks.
-
ITimer: View the CPU consumption across all threads within the same time interval.
On the Running TaskManagers tab of the target deployment, click a target TaskManager, and then select the Debug tab. Here, you can select a Flame Graph type, set a sampling interval, and click Query to generate the graph.
-
Understand the structure of a Flame Graph
Each layer in a Flame Graph represents a level in the call stack. The bottom layer is the application's entry point, while higher layers represent deeper function calls.
-
Focus on the width and frequency of frames
In a Flame Graph, the width of each frame represents the percentage of execution time the function took during sampling. A wider frame indicates higher CPU consumption and a potential bottleneck. A frequently appearing frame, even if not the widest, indicates a function called many times, which can also cause a cumulative performance issue.
-
Analyze the position of frames
The vertical position of a frame indicates the level in the call stack where a bottleneck occurs. A wide frame at the bottom usually indicates a problem in the program's early stages or main logic, while a wide frame at the top might point to a specific processing function or library function.
-
Analyze in the context of your code
After identifying a potential bottleneck in the Flame Graph, analyze the corresponding code. Look for optimization opportunities in these hot spot functions, such as reducing loop iterations, improving data structures, or minimizing synchronization.
-
Run performance tests
After optimizing the code, run performance tests to verify the improvements. Compare the Flame Graphs before and after your changes to confirm the bottleneck is resolved.
If parts of the Flame Graph display as "UnKnown", it indicates native method usage, where Java method names are not expected to appear. For more information, see Unknown.
Threads
-
Go to the Debug tab.
-
JobManager performance
On the Logs tab, click the JobManager tab, and then click Debug.
-
Running TaskManagers performance
On the Logs tab, click the Running TaskManagers tab, click a value in the Path, ID column, and then click Debug.
-
-
On the Threads tab, find the target operator and click Sample in the Actions column. After a sampling period, observe its thread stack. For example, the following is a thread stack for Gemini State access.
The Single Thread Sampling dialog box shows the results, including a CPU usage line chart and a stack sample table. Here, you can view the method call stack, such as
java.lang.Object.wait, at each sampling point. Click Pause Sampling to stop data collection.
Thread Dump
-
On the Logs tab, click the Running TaskManagers tab, and then click a value in the Path, ID column.
-
On the Thread Dump tab, search by operator name and observe the thread stack to see if it is accessing state. A stack that consistently shows calls in the Gemini or RocksDB access path indicates this.
To find the name of an operator to use on the Thread Dump tab, go to the details page of the target TaskManager. You can find the target operator name on the Status tab.
The Detail panel on the right shows the execution plan for each operator, providing names like
TableSourceScan,Calc, andSinkfor your Thread Dump search.
Related documents
-
The intelligent deployment diagnostics service helps you monitor the health of your deployments and ensures the stability and reliability of your business. For more information, see intelligent deployment diagnostics.
-
You can improve Flink SQL deployment performance by optimizing configurations and queries. For more information, see High-performance Flink SQL optimization techniques.