All Products
Search

This Product

    Simple Application Server:What do I do if no disk space is available or the number of inodes exceeds the inode quota in a Linux simple application server?

    Document Center

    Simple Application Server:What do I do if no disk space is available or the number of inodes exceeds the inode quota in a Linux simple application server?

    Last Updated:Jul 23, 2026

    When you create a file or start an application on your Linux simple application server, a No space left on device error indicates the disk space is exhausted. If the disk should not be full given your expected usage, use this topic to identify the cause and resolve the issue.

    Note

    If the disk is full as expected and you simply need more capacity, upgrade your server, attach a data disk, or extend an existing data disk. See Upgrade a simple application server, Attach a data disk, and Extend the data disk.

    Symptom

    When you create a file or run an application on the Linux simple application server, the following error message appears, indicating that storage resources are exhausted: No space left on device.

    Diagnosis and solutions

    Important

    Before you make any changes, create a snapshot to back up your data and prevent data loss caused by mis-operations that could interrupt your services.

    Cause 1: Disk partition space is exhausted

    1. Connect to the Linux server by using the rescue feature. For more information, see Connect to a Linux server by using the rescue feature.

    2. Check disk usage.

      Run sudo df -h to view the disk usage of each mount point. If Use% shows 100%, the corresponding space is full.

    3. Clean up unnecessary files or directories.

      Run sudo du -sh <directory>/* to check the size of files and subdirectories in the specified directory. If needed, enter the directory and drill down level by level to identify which files or directories consume the most space, and then decide which ones can be deleted or moved.

      For example, run sudo du -sh /mnt/* to view the size of files and subdirectories in the /mnt directory.

    4. If space is still insufficient after cleanup, resolve the issue by upgrading the server, attaching a data disk, or extending the data disk.

    Cause 2: Inode resources are exhausted

    Every file consumes one inode. If a large number of small files exist on the disk, inodes may be exhausted even when disk space remains, which prevents you from creating new files.

    1. Connect to the Linux server by using the rescue feature. For more information, see Connect to a Linux server by using the rescue feature.

    2. Check inode usage.

      Run sudo df -i. If IUse% reaches 100%, inode resources are exhausted.

    3. Clean up unnecessary files or directories.

      Run sudo du -sh --inodes <directory>/* to view the number of inodes consumed by files and subdirectories in the specified directory. If needed, enter the directory and drill down level by level.

      For example, run sudo du -sh --inodes /mnt/* to view the number of inodes consumed by files and subdirectories in the /mnt directory.

    4. If inodes are still insufficient after cleanup, upgrade the server, attach a data disk, or extend the data disk.

    Cause 3: Deleted files still hold disk space

    Even after a file is deleted, as long as a process is still using it (that is, holding an open file handle), the system does not release the disk space it occupied. The space is reclaimed only after the process terminates or actively closes the file.

    1. Connect to the Linux server by using the rescue feature. For more information, see Connect to a Linux server by using the rescue feature.

    2. Install the lsof tool.

      Deleted files that still hold space cannot be observed by using df or du. Use the lsof tool to list them.

      Alibaba Cloud Linux, CentOS

      sudo yum install -y lsof

      Debian, Ubuntu

      sudo apt install -y lsof

    3. List deleted files whose disk space has not been released.

      sudo lsof | grep delete | sort -k7 -rn | more

      The 7th column of the output shows the file size in bytes. Sum the sizes to calculate the total amount of space that has not been released.

      tail      619544                        root    3r   REG    253,1   50000000    136507 /home/test_file (deleted)
aliyun-se 347980                        root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347997 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347996 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347995 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347994 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347983 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347982 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)
aliyun-se 347980 347981 aliyun-se       root    7uW  REG    253,1          0    262160 /tmp/AliyunAssistClientSingleLock.lock (deleted)

    4. Record the name and PID of the process that holds the file.

      Run sudo lsof | grep delete. Get the process name from the COMMAND column and the process ID from the PID column.

    5. Restart or stop the related service.

      Run sudo ps -ef | grep <PID> to confirm the purpose of the process. After evaluating the impact, restart or stop the related service.

      Important

      Restarting or stopping the service may affect your business. Evaluate the impact carefully and choose an appropriate time to perform the operation.

    Cause 4: A mount point is overwritten

    When another device is mounted on a non-empty directory, the data under that directory is hidden, but processes that have already opened the directory can still write to and consume the underlying space. This "hidden" space usage cannot be observed by using the df command, and can easily cause unexpected space exhaustion.

    1. Connect to the Linux server by using the rescue feature. For more information, see Connect to a Linux server by using the rescue feature.

    2. Check for duplicate mount point information.

      Run sudo lsblk, view the MOUNTPOINT column, and record any duplicate mount point names.

      sudo lsblk
      NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    253:0    0   40G  0 disk 
├─vda1 253:1    0    2M  0 part 
├─vda2 253:2    0  200M  0 part /boot/efi
└─vda3 253:3    0 39.8G  0 part /
vdb    253:16   0   40G  0 disk 
└─vdb1 253:17   0   40G  0 part /mnt
vdc    253:32   0   40G  0 disk 
└─vdc1 253:33   0   40G  0 part /mnt

      In this example, both vdb1 and vdc1 are mounted on /mnt, which indicates a risk of mount point overwriting.

    3. Unmount the file system.

      Important

      Unmounting the file system may interrupt services that depend on the path. Evaluate the risk and choose an appropriate time to perform the operation.

      Obtain <duplicate mount point> from the previous step.

      sudo umount <duplicate mount point>
      In this example, /mnt is the duplicate mount point. Run sudo umount /mnt to unmount the most recently mounted device vdc1.

    4. Identify the device name of the overwritten mount point.

      Run sudo df -h to locate the device name of the overwritten mount point.

      sudo df -h
      Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.7G     0  3.7G   0% /dev
tmpfs           3.7G     0  3.7G   0% /dev/shm
tmpfs           3.7G  524K  3.7G   1% /run
tmpfs           3.7G     0  3.7G   0% /sys/fs/cgroup
/dev/vda3        40G  4.5G   33G  12% /
/dev/vda2       200M  5.8M  194M   3% /boot/efi
/dev/vdb1        40G   40G     0  100% /mnt
tmpfs           747M     0  747M   0% /run/user/0

      In this example, the partition currently mounted at /mnt is vdb1, so the device name of the overwritten mount point is vdb1.

    5. Resolve the disk space issue.

      1. Clean up unnecessary files or directories in the overwritten space.

        In this example, clean up the /mnt directory that is mounted by vdb1.

      2. If space is still insufficient after cleanup, extend the data disk and mount it on another empty directory.

        In this example, the target device to extend is vdb1.
    Important

    Do not mount multiple devices on the same directory.

    When multiple devices are mounted on the same directory, the space of the earlier-mounted device is hidden and data may be written to the wrong device. Make sure that different devices are mounted on different empty directories.

    Cause 5: Docker-related files occupy large amounts of disk space

    During Docker operation, a large number of intermediate images, stopped containers, and build cache are generated. These objects accumulate over time and occupy disk space.

    1. Check the disk space usage of Docker files.

      Run sudo df -h. If Use% of the entry whose Filesystem is overlay reaches 100%, Docker is the source of the pressure.

    2. Determine the resource usage inside Docker.

      Run sudo docker system df and check the Size and RECLAIMABLE columns to determine which files occupy the most space.

      sudo docker system df
      TYPE            TOTAL      ACTIVE     SIZE       RECLAIMABLE
Images          21         9          13.94GB    10.66GB (76%)
Containers      9          5          30.09MB    0B (0%)
Local volumes   6          6          259.9MB    0B (0%)
Build Cache     0          0          0B         0B

      In this example, Docker images occupy 13.94 GB, of which 10.66 GB is reclaimable. Give priority to cleaning up unused images.

    3. Clean up unnecessary files.

      If Docker files cannot be cleaned up, follow Cause 1: Disk partition space is exhausted to resolve the issue.

      • Remove all stopped containers: run sudo docker container prune.

      • Remove all dangling images (images without tags): run sudo docker image prune.

      • Remove unused build cache: run sudo docker builder prune.

    Cause 6: The inotify watches limit is reached

    When you run a command similar to sudo tail -f and receive the message tail: cannot watch '...': No space left on device, the disk is not full. Instead, the inotify watches limit — which is used to track file and directory changes — has been reached and must be raised.

    1. Check the current inotify watches limit.

      Run sudo cat /proc/sys/fs/inotify/max_user_watches to view the current upper limit of inotify watches.

    2. Increase the inotify watches limit.

      Raising the limit may cause inotify to consume more system memory. Evaluate carefully before modifying. In general, a <new limit> higher than 524288 is not recommended.

      sudo sh -c "echo fs.inotify.max_user_watches=<new limit> >> /etc/sysctl.conf"

    3. Reload the configuration.

      Run sudo sysctl --system to reload the configuration and apply it.

    4. Verify the configuration.

      Run sudo cat /proc/sys/fs/inotify/max_user_watches again to confirm that the value is updated to your expected inotify watches limit.