What do I do if a large amount of disk space is occupied by the EditLog directory of the NameNode or JournalNode? - E-MapReduce

This topic describes how to troubleshoot the issue that a large amount of disk space is occupied by the EditLog directory of the NameNode or JournalNode.

Cause

Hadoop Distributed File System (HDFS) depends on the checkpoints of the FsImage files to merge the EditLog files. If a checkpoint fails to be created for the FsImage files, the EditLog files cannot be merged. In most cases, this is because the FsImage directory is full or an exception occurs in disks. In this case, the NameNode takes an extended period of time to restart. After a checkpoint fails to be created for the FsImage files on the NameNode, the FsImage directory is not used again by default. You must enable the system to automatically restore the FsImage directory.

Solution

Manually enable the system to automatically restore the FsImage directory.
```
su - hdfs
hdfs dfsadmin -restoreFailedStorage true
```
Note After you manually enable the system to automatically restore the FsImage directory, this setting takes effect until you restart the NameNode.
After this setting, when a checkpoint fails to be created for the FsImage files on the NameNode, the system tries to restore the FsImage directory first.
```
restoreFailedStorage is set to true
```
Optional:Manually create a checkpoint for the NameNode. For more information, see Manually perform a checkpoint on a NameNode.
After you manually create a checkpoint for the FsImage files, the EditLog files are immediately merged. This reduces the disk usage that is occupied by the EditLog files. However, to perform this operation, the NameNode must enter the safe mode, which makes HDFS unavailable for a short period of time.
Enable the system to automatically restore the FsImage directory by default.
Log on to the E-MapReduce (EMR) console. Go to the details page of your cluster. On the Configure tab of the HDFS service, click the hdfs-site.xml tab. Add the dfs.namenode.name.dir.restore configuration item and set the value to true. Next time you restart the NameNode, the system is enabled to automatically restore the FsImage directory by default.