Export OSS-HDFS object metadata - Object Storage Service

You can export metadata of objects in OSS-HDFS to a specific path as a JSON file. This facilitates object metadata management and analysis.

Prerequisites

Use the latest version of JindoSDK. For more information, visit GitHub.
A valid AccessKey pair is configured. The AccessKey pair is used to access the bucket for which OSS-HDFS is enabled.
For more information, see Use Jindo CLI commands to access OSS or OSS-HDFS.

Procedure

Export object metadata
1. Log on to the OSS console.
2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.
3. In the left-side navigation tree, choose Data Lake > OSS-HDFS.
4. In the Object Metadata section, click Export.
  The amount of time that is required to export object metadata ranges from several minutes to several hours and varies based on the amount of metadata. By default, the metadata files are exported to the oss://<hdfs_bucket>.<dls_endpoint>/.sysinfo/inventory/ path. Custom output paths are not supported.
  Important
  The .sysinfo/inventory/ directory cannot be deleted. You can access and delete the metadata files in this directory.

Download the metadata file.

Run the following command to download the metadata file:

jindo fs -get oss://<hdfs_bucket>.<dls_endpoint>/.sysinfo/inventory/1666584461201.2ce40fba-5704-45c4-8720-d92a891d**** /tmp/

Run the vi or vim command to open the metadata file.

Sample metadata file:

{"id":163**,"path":"/","type":"directory","size":0,"user":"admin","group":"supergroup","atime":1666581702921,"mtime":1666581702933,"permission":511}
{"id":624668410678950****,"path":"/dls-1000326249","type":"directory","size":0,"user":"hadoop","group":"supergroup","atime":1666581702923,"mtime":1660889124590,"permission":511}
{"id":624668410678950****,"path":"/dls-1000326249/benchmark","type":"directory","size":0,"user":"hadoop","group":"supergroup","atime":166658170293325,"mtime":1660889124590,"permission":511}
{"id":624668410678950****,"path":"/dls-1000326249/benchmark/n1","type":"directory","size":0,"user":"hadoop","group":"supergroup","atime":1666581702930,"mtime":1660889124590,"permission":511}
{"id":624668410678950****,"path":"/dls-1000326249/benchmark/n1/490747449","type":"directory","size":0,"user":"hadoop","group":"supergroup","atime":1666581702931,"mtime":1660895613953,"permission":511}

The following table describes the fields in a metadata file.

Field	Description
id	The ID of the object or directory.
path	The path to the object or directory.
type	The type of the resource. Valid values: directory file
size	The data size in bytes. If the value of the type field is file, the value of the size field is the size of the object. If the value of the type field is directory, the value of the size field is 0.
user	The owner of the object or directory.
group	The user group to which the object or directory belongs.
atime	The time when the object or directory was accessed. The value is a timestamp.
mtime	The time when the object or directory was modified. The value is a timestamp.
permission	The permissions on the object or directory.

(Optional) Delete the metadata file.
An OSS-HDFS metadata file consumes the storage space and incurs a storage fee. If a metadata file is no longer required, we recommend that you delete the metadata file by using HDFS commands.
Important
When you delete a metadata file, make sure that you specify the correct path to the metadata file to avoid deleting system data in the .dlsdata and .sysinfo paths. For example, to delete the metadata file exported in Step 1, specify the exact path to the metadata file.
The following command provides an example on how to delete a metadata file:
```
jindo fs -rm -skipTrash oss://<hdfs_bucket>.<dls_endpoint>/.sysinfo/inventory/1666584461201.2ce40fba-5704-45c4-8720-d92a891d****
```