Use the BCC toolset to track and analyze I/O latency - Alibaba Cloud Linux

The /proc/diskstats raw data source interface of the iostat I/O latency analysis tool is optimized in Alibaba Cloud Linux to calculate the amount of time consumed by read, write, and special I/O (discard) operations on devices. In addition, Alibaba Cloud Linux 2 provides the BPF Compiler Collection (BCC) toolset to track I/O latency. This topic describes the optimized /proc/diskstats interface and the BCC toolset in Alibaba Cloud Linux 2.

/proc/diskstats interface

The /proc/diskstats interface in Alibaba Cloud Linux 2 allows you to query the I/O information on a disk and the amount of time consumed by read, write, and discard operations on a device.

Run the following command to query the /proc/diskstats interface:

cat /proc/diskstats

The following command output is returned:

254       0 vda 6328 3156 565378 2223 1610 424 25160 4366 0 1358 5332 0 0 0 0 2205 3347 0

In the command output, the last three fields are new fields in Alibaba Cloud Linux 2. The following table describes all fields.

Fields

Field	Description
The first field	The total number of completed read operations.
The second field	The number of merged read operations.
The third field	The number of sectors from which data was read.
The fourth field	The amount of time consumed by all read operations. Unit: milliseconds.
The fifth field	The total number of completed write operations.
The sixth field	The number of merged read operations.
The seventh field	The number of sectors to which data was written.
The eighth field	The amount of time consumed by all write operations. Unit: milliseconds.
The ninth field	The number of ongoing I/O operations.
The tenth field	The amount of time consumed by I/O operations. Unit: milliseconds.
The eleventh field	The amount of weighted time consumed by I/O operations. Unit: milliseconds.
The twelfth field	The total number of completed discard operations.
The thirteenth field	The number of merged discard operations.
The fourteenth field	The total number of sectors from which data was discarded.
The fifteenth field	The amount of time consumed by all discard operations. Unit: milliseconds.
The sixteenth domain	The amount of time consumed by read operations on the device. Unit: milliseconds.
The seventeenth domain	The amount of time consumed by write operations on the device. Unit: milliseconds.
The eighteenth domain	The amount of time consumed by discard operations on the device. Unit: milliseconds.

Install and use the BCC toolset

Alibaba Cloud Linux 2 provides the BCC toolset to track I/O latency.

Run the following command to download the BCC toolset:
```
yum install -y bcc-tools
```

Run one of the following commands to query the description of the BCC toolset:

Run the following command to query the description of the BCC toolset:

/usr/share/bcc/tools/alibiolatency -h

Sample description:

usage: alibiolatency [-h] [-d DEVICE] [-i [DIS_INTERVAL]]
                     [-t [AVG_THRESHOLD_TIME]] [-T [THRESHOLD_TIME]] [-r]

Summarize block device I/O latency

optional arguments:
  -h, --help            show this help message and exit
  -d DEVICE, --device DEVICE
                        inspect specified device
  -i [DIS_INTERVAL], --dis_interval [DIS_INTERVAL]
                        specify display interval
  -t [AVG_THRESHOLD_TIME], --avg_threshold_time [AVG_THRESHOLD_TIME]
                        display only when average request process time is
                        greater than this value
  -T [THRESHOLD_TIME], --threshold_time [THRESHOLD_TIME]
                        dump request life cycle when single request process
                        time is greater than this value
  -r, --dump_raw        dump every io request life cycle

examples:
    ./alibiolatency          # summarize block I/O latency(default display interval is 2s)
    ./alibiolatency -d sda3  # inspect specified device /dev/sda3
    ./alibiolatency -i 2     # specify display interval, 2s
    ./alibiolatency -t 10    # display only when average request process time is greater than 10ms
    ./alibiolatency -T 20    # dump request life cycle when single request process time is greater than 20ms
    ./alibiolatency -r       # dump every io request life cycle

Run the man command to query the description of the BCC toolset.
```
man bcc-alibiolatency
```