Resource Management - Data Lake Formation - Alibaba Cloud Documentation Center

This topic describes how to view the resource overview in Data Lake Formation (DLF). The resource overview provides a comprehensive view of your current resource usage. This helps you plan and manage your resources more effectively.

View the resource overview

Data catalog level

Log on to the DLF console.
On the Data Catalog list page, click the name of the target catalog to view its details.
Click the Resource Overview tab. You can view the metrics for the current data catalog. Metrics include resource usage (billing-related metrics), quota monitoring, and resource request details.

Table level

On the Data Catalog list page, click the name of the target catalog. Then, click the target Paimon table in the destination database.
Click the Resource Overview tab to view the metrics and details for the current Paimon table.

Metric description (Data catalog)

Resource usage

Metric	Description
Total storage usage	The latest stored value.
Total storage usage	The change in total storage usage over the selected time period.
Compute resource usage in the last 24 hours (CU × Hours)	The total compute resource usage in CU × Hours for lake table optimization in the data catalog during the 24 hours before the selected end time.
Storage usage details	The raw and billable usage for each storage class over the selected time period. The storage classes are Standard, IA (IA storage class), Archive (Archive Storage), and Cold Archive.
Lake table optimization CU × Hours usage	The hourly compute resource usage in CU × Hours for lake table optimization in the data catalog over the selected time period.
Number of file requests	The number of file requests per hour over the selected time period. Write requests include billable PUT, POST, and HEAD requests. Read requests include billable GET requests.

Quota monitoring

Metric	Description
QPS	The average number of file requests per second in the one minute before the selected end time.
Download bandwidth	The average download bandwidth in the one minute before the selected end time.
Upload bandwidth	The average upload bandwidth in the one minute before the selected end time.
QPS trend	The average number of file requests per second within the last minute.
Bandwidth trend	The change trends of the average upload bandwidth and average download bandwidth within the last minute.

Resource request details

Metric	Description
CU × Hours usage trend by category	The CU usage for various task types within the selected time range.
CU × Hours usage by category	The CU usage for various task types within the selected time range.
Catalog read traffic	Total (total traffic) Compaction (traffic generated by data compaction tasks) RequestLogs (traffic generated by system request logs) DLF (internal system management traffic, such as lifecycle management and file cleanup)
Catalog write traffic
Catalog read traffic by category (sampled by file type)	Manifest (manifest file) Snapshot (snapshot file) Schema (metadata file)
Catalog write traffic by category (sampled by file type)
Catalog read requests	Compaction (requests generated by data compaction tasks) RequestLogs (requests generated by system request logs) DLF (internal system management requests, such as lifecycle management and file cleanup)
Catalog write requests
Catalog read requests by category	Manifest (manifest file) Snapshot (snapshot file) Schema (metadata file) `Bucket-` (categorized by data format)
Catalog write requests by category
Number of IO requests	The number of file requests within one minute.
Number of GetObject requests	The number of GetObject requests within one minute.
GetObject request latency	The average latency of GetObject requests within one minute.
Number of Upload requests	The number of Upload requests within one minute.
Upload request latency	The average latency of Upload requests within one minute.
Number of Delete requests	The number of Delete requests within one minute.
Delete request latency	The average latency of Delete requests within one minute.
Number of ObjectMeta requests	The number of ObjectMeta requests within one minute.
ObjectMeta request latency	The average latency of ObjectMeta requests within one minute.
Number of ListObjects requests	The number of ListObjects requests within one minute.
ListObjects request latency	The average latency of ListObjects requests within one minute.
Number of metadata requests	The number of metadata requests within one minute.
Number of TableRead requests	The number of TableRead requests within one minute.
TableRead request latency	The average latency of TableRead requests within one minute.
Number of TableWrite requests	The number of TableWrite requests within one minute.
TableWrite request latency	The average latency of TableWrite requests within one minute.
Number of List requests	The number of List requests within one minute.
List request latency	The average latency of List requests within one minute.
Number of other metadata requests	The number of other metadata requests within one minute.
Other metadata request latency	The average latency of other metadata requests within one minute.
Top tables by CU × Hours	For operations on multiple tables or at the database level, the ID is displayed as `default_id` by default. Note The data is generated through sampling and aggregation. The values are for reference only.
Top tables by read traffic
Top tables by write traffic
Top tables by read requests
Top tables by write requests

Metric description (Table)

Note

Data collection and processing are subject to a delay. Values from the last three hours may fluctuate and are for reference only.
Percentage data is generated by sampling and aggregation. These values are for reference only.

Metric	Description
Table CU × Hours (hourly)	The trend of CU usage for the current table. This metric only includes tables that use independent compaction.
Table CU × Hours (daily)
Current table CU × Hours trend by category	The CU usage of different task types for the current table. This metric only includes tables that use independent compaction.
Current table CU × Hours by category
Current table/Catalog total CU usage percentage trend	The percentage of the data catalog's total CU usage that the current table consumes. This metric only includes tables that use independent compaction.
Current table/Catalog total CU usage percentage
Table read traffic	The percentage of total table traffic attributed to each origin request.
Table write traffic
Table read traffic by category (sampled by origin request)	The percentage of table traffic from each origin request category.
Table write traffic by category (sampled by origin request)
Table read requests	The percentage of total table requests attributed to each origin request.
Table write requests
Table read requests by category (sampled by origin request)	The percentage of table requests from each origin request category.
Table write requests by category (sampled by origin request)
Table read requests by category (sampled by file type)	The percentage of table requests for each file type.
Table write requests by category (sampled by file type)	The percentage of table requests for each file type.
Table/Catalog read request percentage	The percentage of the data catalog's total requests that are for the current table.
Table/Catalog write request percentage
Number of GetObject requests	The total number of GetObject requests.
GetObject request latency	The average latency of GetObject requests.
Number of Upload requests	The total number of Upload requests.
Upload request latency	The average latency of Upload requests.
Number of Delete requests	The total number of Delete requests.
Delete request latency	The average latency of Delete requests.
Number of ObjectMeta requests	The total number of ObjectMeta requests.
ObjectMeta request latency	The average latency of ObjectMeta requests.
Number of ListObjects requests	The total number of ListObjects requests.
ListObjects request latency	The average latency of ListObjects requests.