When you use the trash feature of OSS-HDFS to delete specified files, the client moves the files to a specific trash directory. Then, the server deletes data in the specific trash directory based on the preset policy.
Usage notes of the trash feature
When you delete a file from a bucket for which OSS-HDFS is enabled, the file is not immediately deleted. Instead, the file is moved to the
/user/<username>/.Trash/Currentdirectory.After 30 minutes, the file is moved from the Current directory to the
/user/<username>/.Trash/<timestamp>directory.NoteFiles that are deleted within a specific period of time are moved to a directory that has a timestamp. The timestamp indicates the time when the files were deleted and functions as a checkpoint.
The directory is permanently deleted after three days.
Within three days after you delete a file, you can find the file in the .Trash directory based on the time when the file was deleted. Then, you can restore the file by moving the file out of the .Trash directory.
The trash feature depends on the cooperation between the client and the server. The client moves the files that you delete to the .Trash directory. By default, the server periodically deletes the files from the /user/<username>/.Trash directory.
Use the trash feature in Hadoop FileSystem Shell
Sample command:
hadoop fs -rm oss://bucket/a/b/cIn Hadoop FileSystem Shell, the trash feature is enabled for the client by default. When you run the preceding remove command, the client automatically converts the command into the move command hadoop fs -mv oss://bucket/a/b/c /user/<username>/.Trash/Current/a/b/c. This way, you do not need to manage the trash feature. The server will periodically clear files in the /user/<username>/.Trash directory.
If you want to immediately delete a file to free up storage space, add the -skipTrash parameter to the remove command. In this case, the file is directly deleted from the file system instead of being moved to the trash directory. Sample command:
hadoop fs -rm -skipTrash oss://bucket/a/b/cUse the trash feature in Hadoop ecosystem services
Services such as Hive, Spark, and Flink are not aware of the trash feature of OSS-HDFS. When you call the delete interface of HDFS to delete a file, the file is immediately deleted.
OSS-HDFS adopts a similar policy to open source Hadoop. To use the trash feature in Hadoop ecosystem services, you must explicitly call the rename interface of HDFS to move the files that you want to delete to the /user/<username>/.Trash/Current directory. Then, the OSS-HDFS server periodically clears files from the trash directory.