Use the kdumpctl tool to view the boot logs for kernel crash dumps - Alibaba Cloud Linux

kdump is a kernel crash dumping mechanism in Linux that allows you to capture a memory image (crash dump or vmcore) when the kernel crashes. The memory image contains the contents of the system memory at the time of the crash to help you analyze and troubleshoot the issue. If kdump encounters an anomaly, kdump may be unable to generate memory images. In this case, you can use the kdumpctl tool to view the boot logs for kernel crash dumps and analyze the kdump anomaly and the cause of the memory image generation failure based on the logs. By default, the kdumpctl tool is pre-installed in Alibaba Cloud Linux 3. This topic describes how to use the kdumpctl tool to view the boot logs for kernel crash dumps on an Elastic Compute Service (ECS) instance that runs Alibaba Cloud Linux 3.

Prerequisites

Before you can use the kdumpctl tool to view the boot logs for kernel crash dumps on an ECS instance, make sure that the instance and the tool meet the following requirements:

The ECS instance has more than 2 GB of memory.
The ECS instance runs Alibaba Cloud Linux 3 or Anolis OS 8 with kernel version 5.10.134-14 or later.
The version of the kexec-tools package is 2.0.25.0.2 or later.

Procedure

Perform the following operations to view the boot logs for kernel crash dumps by using the kdumpctl tool:

Run the following command to check the status of kdump:
```
sudo kdumpctl status
```
Perform different steps based on the command output:
- If the kdump: Kdump is operational message appears in the command output, kdump runs as expected. Skip Step 2 and perform Step 3.
- If the kdump: Kdump is not operational message appears in the command output, perform Step 2 to start kdump.
Run the following command to start kdump:
```
sudo kdumpctl start
```
The following command output indicates that kdump is started:
```
kdump: kexec: loaded kdump kernel
kdump: Starting kdump: [OK]
```
Run the following command to trigger a Linux kernel crash.
If a kernel crash occurred after the last time the Linux operating system was restarted, skip this step and perform Step 4.
Warning
After you run the following command, the kernel immediately crashes. Kernel crashes cause system exceptions or service interruptions. Do not perform this operation in a production environment or on systems in which business runs.
```
echo c | sudo tee /proc/sysrq-trigger
```
Important
After you run the preceding command, the Linux terminal simulator that you use cannot communicate with the ECS instance. Reconnect to the ECS instance to perform subsequent operations.

Run the following command to view the boot logs for the kernel crash dump:

Important

Run the sudo kdumpctl readlog command only after crash-triggered kernel restart, but not system restart. The boot logs that you want to view by running the following command are stored in memory. If you run the command after the Linux operating system is restarted, the command output is empty.

sudo kdumpctl readlog

The following command output is returned:

console log: 
[    0.000000] Linux version 5.10.134-17.2.al8.x86_64 (mockbuild@h87c01383.na61) (gcc (GCC) 10.2.1 20200825 (Alibaba 10.2.1-3.8 2.32), GNU ld version 2.35-12.3.al8) #1 SMP Fri Aug 9 15:49:42 CST 2024
[    0.000000] Command line: BOOT_IMAGE=(hd0,gpt3)/boot/vmlinuz-5.10.134-17.2.al8.x86_64 ro rhgb cgroup.memory=nokmem spec_rstack_overflow=off vring_force_dma_api kfence.sample_interval=100 kfence.booting_max=0-2G:0,2G-32G:2M,32G-:32M preempt=none biosdevname=0 net.ifnames=0 console=tty0 console=ttyS0,115200n8 noibrs nvme_core.io_timeout=4294967295 nvme_core.admin_timeout=4294967295 cryptomgr.notests rcupdate.rcu_cpu_stall_timeout=300 vring_force_dma_api irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0 ramoops.mem_address=0xaf000000 ramoops.mem_size=0x60000 ramoops.console_size=0x48000 ramoops.ttyprobe_size=0x10000 ramoops.ecc=0 ramoops.dump_oops=0 elfcorehdr=3063156K
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'

.........
ttylog:

Welcome to Alibaba Cloud Linux 3.2104 U10 (OpenAnolis Edition) dracut-049-228.git20230802.0.1.al8 (Initramfs)!

[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Reached target Timers.
[  OK  ] Reached target Initrd Root Device.
[  OK  ] Reached target Local File Systems.
[  OK  ] Listening on Journal Socket.
[  OK  ] Started Memstrack Anylazing Service.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Journal Socket (/dev/log).
.........