Retrieves the list of GPU cluster alerts.
Operation description
After installing the component on the target ACK cluster:
When a cluster is managed for the first time, the component is installed on all ECS instances currently in the cluster. If the cluster has more than 50 nodes, only the first 50 nodes are processed in the first batch.
The operating system console periodically checks the scaling status of managed clusters. When a new ECS instance is added to a cluster, the console automatically installs the component on it without user intervention.
Try it now
Test
RAM authorization
Request syntax
GET /api/v1/aiTopology/topology/listAlarmRecord HTTP/1.1Request parameters
|
Parameter |
Type |
Required |
Description |
Example |
| alarmTime |
string |
No |
Query by a specific point in time, such as "2025-08-22 18:16:25". Query by a time range using the "start,end" format, such as "2025-08-22 18:16:25,2025-08-23 18:16:25". |
"2025-08-22 18:16:25,2025-08-23 18:16:25" |
| alarmCluster |
string |
No |
Query by a specific cluster, such as "cluster1". To query multiple clusters, separate them with commas: "cluster1,cluster2,cluster3". |
"cluster1" |
| alarmTask |
string |
No |
Query by a specific task, such as "task1". To query multiple tasks, separate them with commas: "task1,task2,task3". |
task id |
| current |
integer |
Yes |
The current page number for pagination. |
1 |
| pageSize |
integer |
Yes |
The number of entries per page for pagination. |
10 |
| status |
string |
No |
Filter by task status, such as blocking, running, exited, or restart. |
Running |
| alarmType |
string |
No |
Filter by anomaly type, such as GPU memory access out-of-bounds or task hang. |
cuda error |
| alarmExtInfo |
string |
No |
Filter anomalies by instance ID or container. |
instance id |
Response elements
|
Element |
Type |
Description |
Example |
|
object |
Schema of Response |
||
| code |
string |
The value is "Success" or "Sysom.ServerError". |
Success or Sysom.ServerError |
| data |
array<object> |
The list of alert data. |
|
|
object |
The data node. |
||
| alarmCluster |
string |
The cluster where the alert was triggered. |
cluster1 |
| alarmExtInfo |
any |
The additional information about the alert. |
{ error_node_gpu, error_pod, task_pod_list, error_type, kernel, } |
| alarmId |
string |
The unique ID of the alert. |
746d13f8-9662-4441-b76b-a8ed8d485195 |
| alarmLog |
string |
The event log associated with the alert. |
cuda OOM：xxxxx |
| alarmTask |
string |
The ID of the task where the anomaly occurred. |
task job id |
| alarmTime |
string |
The date and time when the alert was triggered. |
2025-08-22 18:16:25 |
| alarmTopology |
any |
The topology snapshot. |
topology data，e.g. "alarmTopology": { "nodes": [ { "gpu_list": [ { "values": { "Basic": [ { "value": "1", "key": "Gpu Number\n" } ], "Metric": [ { "value": "50", "key": "gpu_util(%)\n" } ], "Exception": [ { "value": "NCCL HANG", "key": "error0" } ] }, "name": "GPU1", "style": { "fill": "#e94242\n", "opacity": "0.1" }, "id": "6cc37fbc" } ], "values": { "Basic": [ { "value": "i-xxxxxx", "key": "instance" } ], "Metric": [ { "value": "value of metrics 1", "key": "name of metric 1" } ], "Exception": [ { "value": "NCCL HANG", "key": "error0" } ] }, "name": "i-xxxxxx\n", "style": { "fillOpacity": "0.5", "fill": "#e94242\n" }, "id": "975fbcb0" } ], "edges": [ { "src_id": "00534a88", "values": { "Basic": [ { "value": "v2.22.3\n", "key": "NCCL Version\n" } ], "Metric": [ { "value": "10000", "key": "Recv Msg Count" } ], "Exception": [ { "value": "NCCL HANG", "key": "error0" } ] }, "dst_id": "260525e9", "style": { "opacity": "0.1", "lineWidth": "1", "selectedStyle": "\"{\\\"opacity\\\": 1, \\\"lineWidth\\\": 1}\"\n", "stroke": "#e94242" }, "type": "GPU" } ] }, |
| alarmType |
string |
The Alarm Metric for alerting. |
cuda OOM |
| status |
string |
The task status. |
running、blocking、restart、exited、failed |
| uid |
string |
The UUID of the cloud account. |
1234123412352311 |
| message |
string |
When code is not "Success", message contains the error information. |
server error |
| requestId |
string |
Id of the request |
real request ID |
| total |
string |
The total number of returned records. |
10 |
Examples
Success response
JSON format
{
"code": "Success or Sysom.ServerError",
"data": [
{
"alarmCluster": "cluster1",
"alarmExtInfo": "{\nerror_node_gpu, \nerror_pod, \ntask_pod_list, \nerror_type, \nkernel, \n}",
"alarmId": "746d13f8-9662-4441-b76b-a8ed8d485195",
"alarmLog": "cuda OOM：xxxxx",
"alarmTask": "task job id",
"alarmTime": "2025-08-22 18:16:25",
"alarmTopology": "topology data，e.g.\n \"alarmTopology\": {\n \"nodes\": [\n {\n \"gpu_list\": [\n {\n \"values\": {\n \"Basic\": [\n {\n \"value\": \"1\",\n \"key\": \"Gpu Number\\n\"\n }\n ],\n \"Metric\": [\n {\n \"value\": \"50\",\n \"key\": \"gpu_util(%)\\n\"\n }\n ],\n \"Exception\": [\n {\n \"value\": \"NCCL HANG\",\n \"key\": \"error0\"\n }\n ]\n },\n \"name\": \"GPU1\",\n \"style\": {\n \"fill\": \"#e94242\\n\",\n \"opacity\": \"0.1\"\n },\n \"id\": \"6cc37fbc\"\n }\n ],\n \"values\": {\n \"Basic\": [\n {\n \"value\": \"i-xxxxxx\",\n \"key\": \"instance\"\n }\n ],\n \"Metric\": [\n {\n \"value\": \"value of metrics 1\",\n \"key\": \"name of metric 1\"\n }\n ],\n \"Exception\": [\n {\n \"value\": \"NCCL HANG\",\n \"key\": \"error0\"\n }\n ]\n },\n \"name\": \"i-xxxxxx\\n\",\n \"style\": {\n \"fillOpacity\": \"0.5\",\n \"fill\": \"#e94242\\n\"\n },\n \"id\": \"975fbcb0\"\n }\n ],\n \"edges\": [\n {\n \"src_id\": \"00534a88\",\n \"values\": {\n \"Basic\": [\n {\n \"value\": \"v2.22.3\\n\",\n \"key\": \"NCCL Version\\n\"\n }\n ],\n \"Metric\": [\n {\n \"value\": \"10000\",\n \"key\": \"Recv Msg Count\"\n }\n ],\n \"Exception\": [\n {\n \"value\": \"NCCL HANG\",\n \"key\": \"error0\"\n }\n ]\n },\n \"dst_id\": \"260525e9\",\n \"style\": {\n \"opacity\": \"0.1\",\n \"lineWidth\": \"1\",\n \"selectedStyle\": \"\\\"{\\\\\\\"opacity\\\\\\\": 1, \\\\\\\"lineWidth\\\\\\\": 1}\\\"\\n\",\n \"stroke\": \"#e94242\"\n },\n \"type\": \"GPU\"\n }\n ]\n },",
"alarmType": "cuda OOM",
"status": "running、blocking、restart、exited、failed",
"uid": "1234123412352311"
}
],
"message": "server error",
"requestId": "real request ID",
"total": "10"
}Error codes
|
HTTP status code |
Error code |
Error message |
Description |
|---|---|---|---|
| 400 | Sysom.ServerError | Server error, please correct the code according to the error message. %s. | Server error, please correct the code according to the error message |
See Error Codes for a complete list.
Release notes
See Release Notes for a complete list.