获取GPU集群告警列表
接口说明
为目标 ACK 集群安装组件后:
首先,初次纳管集群,会为集群当前包含的所有 ECS 安装组件(如果集群节点超过 50 台,第一批次只会安装 50 台);
其次,操作系统控制台会周期性的检查被纳管集群的扩缩容情况,一旦有新的 ECS 被添加到集群中,操作系统控制台会为其自动安装组件无需用户干预。
调试
您可以在OpenAPI Explorer中直接运行该接口,免去您计算签名的困扰。运行成功后,OpenAPI Explorer可以自动生成SDK代码示例。
调试
授权信息
请求语法
GET /api/v1/aiTopology/topology/listAlarmRecord HTTP/1.1
请求参数
|
名称 |
类型 |
必填 |
描述 |
示例值 |
| alarmTime |
string |
否 |
指定时刻查询,如:"2025-08-22 18:16:25" 指定时间范围查询为"start,end"格式,如: "2025-08-22 18:16:25,2025-08-23 18:16:25" |
"2025-08-22 18:16:25,2025-08-23 18:16:25" |
| alarmCluster |
string |
否 |
指定集群查询,如:"cluster1",如查询多个集群用 逗号分隔:"cluster1,cluster2,cluster3" |
"cluster1" |
| alarmTask |
string |
否 |
指定任务查询,如:"task1",如 ch 查询多个任务用 逗号分隔:"task1,task2,task3 |
task id |
| current |
integer |
是 |
分页参数,当前页 |
1 |
| pageSize |
integer |
是 |
分页参数,页大小 |
10 |
| status |
string |
否 |
根据任务状态过滤,如 blocking、running、exited、restart |
Running |
| alarmType |
string |
否 |
过滤指定异常类型,如显存访问越界、任务 HANG |
cuda error |
| alarmExtInfo |
string |
否 |
按照 instance id、容器过滤异常 |
instance id |
返回参数
|
名称 |
类型 |
描述 |
示例值 |
|
object |
Schema of Response |
||
| code |
string |
code 为'Success'或'Sysom.ServerError' |
Success or Sysom.ServerError |
| data |
array<object> |
告警数据列表 |
|
|
object |
数据节点 |
||
| alarmCluster |
string |
告警集群 |
cluster1 |
| alarmExtInfo |
any |
告警其他信息 |
{ error_node_gpu, error_pod, task_pod_list, error_type, kernel, } |
| alarmId |
string |
唯一告警 ID |
746d13f8-9662-4441-b76b-a8ed8d485195 |
| alarmLog |
string |
告警事件日志 |
cuda OOM:xxxxx |
| alarmTask |
string |
发生异常的任务 ID |
task job id |
| alarmTime |
string |
告警时间日期 |
2025-08-22 18:16:25 |
| alarmTopology |
any |
拓扑快照 |
topology data,e.g. "alarmTopology": { "nodes": [ { "gpu_list": [ { "values": { "Basic": [ { "value": "1", "key": "Gpu Number\n" } ], "Metric": [ { "value": "50", "key": "gpu_util(%)\n" } ], "Exception": [ { "value": "NCCL HANG", "key": "error0" } ] }, "name": "GPU1", "style": { "fill": "#e94242\n", "opacity": "0.1" }, "id": "6cc37fbc" } ], "values": { "Basic": [ { "value": "i-xxxxxx", "key": "instance" } ], "Metric": [ { "value": "value of metrics 1", "key": "name of metric 1" } ], "Exception": [ { "value": "NCCL HANG", "key": "error0" } ] }, "name": "i-xxxxxx\n", "style": { "fillOpacity": "0.5", "fill": "#e94242\n" }, "id": "975fbcb0" } ], "edges": [ { "src_id": "00534a88", "values": { "Basic": [ { "value": "v2.22.3\n", "key": "NCCL Version\n" } ], "Metric": [ { "value": "10000", "key": "Recv Msg Count" } ], "Exception": [ { "value": "NCCL HANG", "key": "error0" } ] }, "dst_id": "260525e9", "style": { "opacity": "0.1", "lineWidth": "1", "selectedStyle": "\"{\\\"opacity\\\": 1, \\\"lineWidth\\\": 1}\"\n", "stroke": "#e94242" }, "type": "GPU" } ] }, |
| alarmType |
string |
告警类型 |
cuda OOM |
| status |
string |
任务状态 |
running、blocking、restart、exited、failed |
| uid |
string |
云账户 uuid |
1234123412352311 |
| message |
string |
code 不为'Success'时,message 为出错信息 |
server error |
| requestId |
string |
Id of the request |
real request ID |
| total |
string |
返回记录数量 |
10 |
示例
正常返回示例
JSON格式
{
"code": "Success or Sysom.ServerError",
"data": [
{
"alarmCluster": "cluster1",
"alarmExtInfo": "{\nerror_node_gpu, \nerror_pod, \ntask_pod_list, \nerror_type, \nkernel, \n}",
"alarmId": "746d13f8-9662-4441-b76b-a8ed8d485195",
"alarmLog": "cuda OOM:xxxxx",
"alarmTask": "task job id",
"alarmTime": "2025-08-22 18:16:25",
"alarmTopology": "topology data,e.g.\n \"alarmTopology\": {\n \"nodes\": [\n {\n \"gpu_list\": [\n {\n \"values\": {\n \"Basic\": [\n {\n \"value\": \"1\",\n \"key\": \"Gpu Number\\n\"\n }\n ],\n \"Metric\": [\n {\n \"value\": \"50\",\n \"key\": \"gpu_util(%)\\n\"\n }\n ],\n \"Exception\": [\n {\n \"value\": \"NCCL HANG\",\n \"key\": \"error0\"\n }\n ]\n },\n \"name\": \"GPU1\",\n \"style\": {\n \"fill\": \"#e94242\\n\",\n \"opacity\": \"0.1\"\n },\n \"id\": \"6cc37fbc\"\n }\n ],\n \"values\": {\n \"Basic\": [\n {\n \"value\": \"i-xxxxxx\",\n \"key\": \"instance\"\n }\n ],\n \"Metric\": [\n {\n \"value\": \"value of metrics 1\",\n \"key\": \"name of metric 1\"\n }\n ],\n \"Exception\": [\n {\n \"value\": \"NCCL HANG\",\n \"key\": \"error0\"\n }\n ]\n },\n \"name\": \"i-xxxxxx\\n\",\n \"style\": {\n \"fillOpacity\": \"0.5\",\n \"fill\": \"#e94242\\n\"\n },\n \"id\": \"975fbcb0\"\n }\n ],\n \"edges\": [\n {\n \"src_id\": \"00534a88\",\n \"values\": {\n \"Basic\": [\n {\n \"value\": \"v2.22.3\\n\",\n \"key\": \"NCCL Version\\n\"\n }\n ],\n \"Metric\": [\n {\n \"value\": \"10000\",\n \"key\": \"Recv Msg Count\"\n }\n ],\n \"Exception\": [\n {\n \"value\": \"NCCL HANG\",\n \"key\": \"error0\"\n }\n ]\n },\n \"dst_id\": \"260525e9\",\n \"style\": {\n \"opacity\": \"0.1\",\n \"lineWidth\": \"1\",\n \"selectedStyle\": \"\\\"{\\\\\\\"opacity\\\\\\\": 1, \\\\\\\"lineWidth\\\\\\\": 1}\\\"\\n\",\n \"stroke\": \"#e94242\"\n },\n \"type\": \"GPU\"\n }\n ]\n },",
"alarmType": "cuda OOM",
"status": "running、blocking、restart、exited、failed",
"uid": "1234123412352311"
}
],
"message": "server error",
"requestId": "real request ID",
"total": "10"
}
错误码
|
HTTP status code |
错误码 |
错误信息 |
描述 |
|---|---|---|---|
| 400 | Sysom.ServerError | Server error, please correct the code according to the error message. %s. | 服务端错误,请根据错误提示信息修正代码 |
访问错误中心查看更多错误码。
变更历史
更多信息,参考变更详情。