Check the network and hardware status of Lingjun nodes and run network tests based on multiple communication libraries and communication models.
Self-service diagnostics
Network diagnostics
Network diagnostics includes Static config check and Dynamic config check, with visualized results for Lingjun node network status.
-
Static config check run static configuration diagnostics for Lingjun nodes, including system software, network, and GPU checks.
-
Log on to the Lingjun console.
-
In the left-side navigation bar, choose Diagnostics > Self-service Diagnostics.
-
Click the Network Diagnostics tab.
-
Click Static config check.
-
In the Diagnostics Information section, select the target Cluster Name, Node ID, and other parameters from the dropdown lists.
-
Click Start Diagnostics.
-
-
Dynamic config check run dynamic diagnostics for Lingjun nodes, including TCP connectivity, TCP latency, and RDMA connectivity checks.
-
In the left-side navigation bar, choose Diagnostics > Self-service Diagnostics.
-
Click the Network Diagnostics tab.
-
Click Dynamic config check.
-
In the Diagnostics Information section, configure the target Cluster Name, Node ID, and other parameters from the dropdown list.
-
Click Start Diagnostics.
-
Server Diagnostics
Server diagnostics checks the hardware status of Lingjun nodes and provides visualized diagnostic results.
-
In the left-side navigation bar, choose Diagnostics > Self-service Diagnostics.
-
Click the Diagnose server tab.
-
Click Diagnose hardware.
-
In the Diagnostics Information section, configure the target Cluster Name, Node ID, and other parameters from the dropdown list.
-
Click Start Diagnostics.
Network Tests
Network tests include general network tests and communication library tests. General network tests cover latency and traffic. Communication library tests use the Alibaba Collective Communication Library (ACCL) and NVIDIA Collective Communications Library (NCCL) across multiple communication models.
General Network Tests
-
In the left-side navigation bar, choose Diagnostics > Network test.
-
Click the General Network Test tab.
-
In the Test Information area, select the Network Protocol and Test Type parameters, and then configure the corresponding parameters.
RDMA traffic test
Configuration Parameters
Description
Traffic Model
-
The traffic mode. Valid values: MtoN: tests the one-way connectivity between Client nodes and Server nodes, including the one-way connectivity between a single Client node and a single Server node and between multiple Client nodes and multiple Server nodes.
-
Fullmesh: tests the connectivity between each two nodes among all Lingjun nodes.
Test Duration
Select a fixed duration from the dropdown list. Unit: seconds.
QP
The number of test process streams, which affects the test bandwidth size.
GDR
Specifies whether to test the direct communication bandwidth between GPUs. If you set the GDR parameter to Enable, the system associates the Lingjun network interface controller (NIC) with the corresponding GPU for traffic testing.
Cluster Name
The name of the cluster to which the target Lingjun node belongs.
Clients
If you set the Traffic Model parameter to MtoN, you must select Client nodes.
Servers
If you set the Traffic Model parameter to MtoN, you must select Server nodes.
Select a destination node.
If you set the Traffic Model parameter to Fullmesh, you must select a destination node.
Node Port
The starting port used for the test.
RDMA protocol latency test
Configuration parameters
Description
Cluster Name
The name of the cluster to which the target Lingjun node belongs.
Test Nodes
The latency test checks the network latency between each pair of Test Nodes.
Node Port
The starting port used for the test.
-
-
Click Start Diagnostics.
Communication Library Test
-
In the left-side navigation bar, choose Diagnostics > Network test.
-
Click the Library Test tab.
-
Configure the test info in the Test Information section of the Library Test tab.
Configure Parameters
Description
Library Category
Only ACCL and NCCL are supported.
Communication Model
-
The communication model. Valid values: ALLReduce: This communication model aggregates data of multiple processes, reduces the data to a single value based on calculations, and then distributes the result to all processes.
-
ALLGather: This communication model gathers the data of all processes to a structure so that each process can access the data.
-
ALLGatherA: Data type parameters are added on the basis of the ALLGather model. This communication model can transfer a variety of data types, including big data types and custom data types.
-
ALLToALL: This communication model distributes the data of each process to other processes, and each process can obtain the data of all other processes.
-
ALLToALLA: Data type parameters and buffer parameters are added on the basis of the ALLToAllmodel. This communication model can transfer data of different sizes and types.
-
Broadcast: This communication model distributes the data of one process to all other processes.
Number of GPUs
Valid values: 1 to 8.
Cluster Name
The name of the cluster to which the target Lingjun node belongs.
Select a destination node.
When selecting the target edge zone, specify only the IP address. You do not need to specify the Lingjun node to start the test.
Node Port
The starting port used for the test.
-
-
Click Start Test.
View Report
The diagnostic history shows reports for Self-service Diagnostics and Network test, including the Report ID, Cluster Name, and more. Switch between tabs to view different diagnostic types. In the Actions column of the target Report ID, click an option to perform the desired operation.
-
View Report. View the diagnostic results and diagnostic list of the target diagnosis.
-
Diagnose Again to perform the diagnostics again.