All Products
Search
Document Center

E-MapReduce:Use the trash feature of OSS-HDFS (JindoFS)

Last Updated:Sep 29, 2024

When you use the trash feature of OSS-HDFS to delete specified files, the client moves the files to a specific trash directory. Then, the server deletes data in the specific trash directory based on the preset policy.

Usage notes of the trash feature

  1. When you delete a file from a bucket for which OSS-HDFS is enabled, the file is not immediately deleted if you do not forcefully delete the file. Instead, the file is moved to the /user/<username>/.Trash/Current directory.

  2. After 30 minutes, the file is moved from the Current directory to the /user/<username>/.Trash/<timestamp> directory.

    Note

    Files that are deleted within a specific period of time are moved to a directory that has a timestamp. The timestamp indicates the time when the files were deleted and functions as a checkpoint.

  3. The directory is permanently deleted after three days.

Within three days after you delete a file, you can find the file in the .Trash directory based on the time when the file was deleted. Then, you can restore the file by moving the file out of the .Trash directory.

Note

The trash feature depends on the cooperation between the client and the server. The client moves the files that you delete to the .Trash directory. By default, the server periodically deletes the files from the /user/<username>/.Trash directory.

Use the trash feature in Hadoop FileSystem Shell

In Hadoop FileSystem Shell, the trash feature is enabled for the client by default.

Note

The trash feature is available only for OSS-HDFS.

  • Run the following command to move a file to the trash directory. Then, the server will periodically clear files in the directory.

    hadoop fs -rm  oss://examplebucket.cn-hangzhou.oss-dls.aliyuncs.com/a/b/c

    When you run the preceding command, the client automatically converts the command into the move command hadoop fs -mv oss://examplebucket.cn-hangzhou.oss-dls.aliyuncs.com/a/b/c /user/<username>/.Trash/Current/a/b/c. This way, you do not need to manage the trash feature. The server will periodically clear files in the /user/<username>/.Trash directory.

  • Use one of the following methods to permanently delete a file to free up storage space. In this case, the file is directly deleted instead of being moved to the trash directory.

    • Method 1: Run the following command to forcefully delete a file

      hadoop fs -rm -f oss://examplebucket.cn-hangzhou.oss-dls.aliyuncs.com/a/b/c
    • Method 2: Add the -skipTrash parameter to the following command

      hadoop fs -rm -skipTrash  oss://examplebucket.cn-hangzhou.oss-dls.aliyuncs.com/a/b/c

Use the trash feature in Hadoop ecosystem services

Services such as Hive, Spark, and Flink are not aware of the trash feature of OSS-HDFS. When you call the delete interface of HDFS to delete a file, the file is immediately deleted.

OSS-HDFS adopts a similar policy to open source Hadoop. To use the trash feature in Hadoop ecosystem services, you must explicitly call the rename interface of HDFS to move the files that you want to delete to the /user/<username>/.Trash/Current directory. Then, the OSS-HDFS server periodically clears files from the trash directory.