Problem description
An Elastic Compute Service (ECS) instance occasionally experiences automatic stops even when no operations are performed. After you restart the instance, the instance runs for a period of time and then automatically stops again. Error messages similar to the following messages appear in the system log files such as
/var/log/messages:localhost auditd[607]: Audit daemon is low on disk space for logging
localhost auditd[607]: The audit daemon is now halting the systemCause
By default, the auditd service is enabled for ECS instances that run CentOS or RHEL operating systems. The auditd service writes a large number of audit log entries to the /var/log/audit/ directory while other services and applications continuously write data to disks. As a result, the disk space may become insufficient. You can configure auditd action parameters, such as space_left_action, admin_space_left_action, and disk_full_action, in the /etc/audit/auditd.conf configuration file to specify what action to take when the disk space is insufficient. If you set an auditd action parameter to halt, the auditd service triggers a system shutdown when the disk space is insufficient. As a result, the ECS instance is stopped.
Solution
- Connect to the ECS instance. For more information, see Connection method overview.
- Run the following command to modify the auditd configuration file:
vi /etc/audit/auditd.conf - Change the values of the auditd action parameters that respond to insufficient disk space. The parameters include space_left_action, admin_space_left_action, and disk_full_action. We recommend that you set the parameters to
SUSPEND, which specifies that the auditd service stops writing audit log entries to the /var/log/audit/ directory when the disk space is insufficient.space_left = 75 space_left_action = SUSPEND admin_space_left = 50 admin_space_left_action = SUSPEND disk_full_action = SUSPEND - Restart the auditd service for the new configurations to take effect.
systemctl restart auditd.service