Table Struct Metadata and ID Format Explained - DataWorks - Alibaba Cloud - DataWorks

Parameter	Type	Description	Example
	object	The table object.
Id	string	The ID. For more information, see Metadata entity concepts. The format is `${EntityType}:${instance ID or encoded URL}:${DataFolderIdentity}:${DatabaseName}:${PatternName}:${TableName}`. Use an empty character as a placeholder for levels that do not exist. Note For maxcompute and dlf types, use an empty string as a placeholder for the instance ID. For the maxcompute type, the database name is the MaxCompute project name. Projects with the three-layer model enabled require a schema name. For projects without the three-layer model enabled, use an empty string as a placeholder for the schema name. Note For the starrocks type, the data catalog identifier is the catalog name. For the dlf type, the data catalog identifier is the catalog ID. Other types do not support the catalog level. Use an empty string as a placeholder. The following examples show the ID formats for common types: `maxcompute-table:::project_name:[schema_name]:table_name` `dlf-table::catalog_id:database_name::table_name` `hms-table:instance_id::database_name::table_name` `holo-table:instance_id::database_name:schema_name:table_name` `mysql-table:(instance_id\|encoded_jdbc_url)::database_name::table_name` Note Where `instance_id`: The instance ID. This is required when the data source is registered in instance mode. `encoded_jdbc_url`: The URL-encoded JDBC connection string. This is required when the data source is registered by using a connection string. `catalog_id`: The DLF catalog ID. `project_name`: The MaxCompute project name. `database_name`: The database name. `schema_name`: The schema name. For the maxcompute type, this is required only when the three-layer model is enabled for the project. If the three-layer model is not enabled, use an empty string as a placeholder. `table_name`: The table name.	dlf-table::catalog_id:database_name::table_name
Name	string	The name.	table_name
Comment	string	The comment.	测试表
TableType	string	The table type. The valid values depend on the metadata collector type.	TABLE
PartitionKeys	array	The list of partition keys. This is empty for non-partitioned tables.
	string	The partition key.	ds
ParentMetaEntityId	string	The parent-level metadata entity ID. For more information, see Metadata entity concepts. For types that support schemas (`maxcompute/holo/postgresql/sqlserver/hybriddb_for_postgresql/oracle, where the maxcompute type requires the Layer 3 model to be enabled for the project`), ParentMetaEntityId is the database pattern to which the table belongs. The format is `${EntityType}:${instance ID or encoded URL}:${DataFolderIdentity}:${DatabaseName}:${PatternName}`. Use an empty character as a placeholder for levels that do not exist. For other types, ParentMetaEntityId is the database to which the table belongs. The format is `${EntityType}:${instance ID or encoded URL}:${DataFolderIdentity}:${DatabaseName}`. Use an empty character as a placeholder for levels that do not exist. Note For maxcompute and dlf types, use an empty string as a placeholder for the instance ID. For the maxcompute type, the database name is the MaxCompute project name. Note For the starrocks type, the data catalog identifier is the catalog name. For the dlf type, the data catalog identifier is the catalog ID. Other types do not support the catalog level. Use an empty string as a placeholder. The following examples show the ParentMetaEntityId formats for common types: `maxcompute-project:::project_name` `maxcompute-schema:::project_name:schema_name` (only when the three-layer model is enabled for the project) `dlf-database::catalog_id:database_name` `hms-database:instance_id::database_name` `holo-schema:instance_id::database_name:schema_name` `mysql-database:(instance_id\|encoded_jdbc_url)::database_name` Note Where `instance_id`: The instance ID. This is required when the data source is registered in instance mode. `encoded_jdbc_url`: The URL-encoded JDBC connection string. This is required when the data source is registered by using a connection string. `catalog_id`: The DLF catalog ID. `project_name`: The MaxCompute project name. `database_name`: The database name. `schema_name`: The schema name.	dlf-database::catalog_id:database_name
CreateTime	integer	The creation time, in millisecond-level timestamp.	1736852168000
ModifyTime	integer	The modification time, in millisecond-level timestamp.	1736852168000
TechnicalMetadata	object	The technical metadata.
Owner	string	The owner.	test_user
Location	string	The storage location.	oss://test-bucket/test_tbl
Compressed	boolean	Indicates whether the table is compressed.	false
InputFormat	string	The input format.	org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
OutputFormat	string	The output format.	org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
SerializationLibrary	string	The class used by SerDe.	org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
Parameters	object	The parameter information.
	string	The parameter information.	value
BusinessMetadata	object	The business metadata related to DataWorks, including usage instructions, tags, categories, upstream production nodes, and extended information.
Readme	string	The usage instructions.	## 使用说明
CustomAttributes	object	The custom attribute values, where key is the custom attribute identifier and value is the list of attribute values.
	array	The list of attribute values.
	string	The attribute value.	张三
Tags	array	The list of tags.
	object	The tag information.
Key	string	The tag key. This value cannot be empty.	tag_key
Value	string	The tag value. This can be an empty string.	tag_value
Categories	array	The list of categories.
	array	The hierarchical list of a single category (multi-level category).
	object	The category information.
Id	string	The category ID.	CATEGORY.456
Name	string	The category name.	测试类目
ParentId	string	The parent category's ID. This can be an empty string.	CATEGORY.123
UpstreamTasks	array	The list of upstream nodes.
	object	The upstream node.
Id	integer	The task ID.	123456
Name	string	The task name.	test_task
Extension	object	The extension information. Currently only supported for MaxCompute type.
ProjectId	integer	The workspace ID.	234
EnvType	string	The environment type. Valid values: Prod: The production environment. Dev: The development environment.	Dev
ViewCount	integer	The number of views.	0
ReadCount	integer	The number of reads.	0
FavorCount	integer	The number of times the table was favorited.	0
StatisticsInfos	object
	string

object

The table object.

Id

string

The ID. For more information, see Metadata entity concepts.

The format is ${EntityType}:${instance ID or encoded URL}:${DataFolderIdentity}:${DatabaseName}:${PatternName}:${TableName}. Use an empty character as a placeholder for levels that do not exist.

Note

For maxcompute and dlf types, use an empty string as a placeholder for the instance ID. For the maxcompute type, the database name is the MaxCompute project name. Projects with the three-layer model enabled require a schema name. For projects without the three-layer model enabled, use an empty string as a placeholder for the schema name.

Note

For the starrocks type, the data catalog identifier is the catalog name. For the dlf type, the data catalog identifier is the catalog ID. Other types do not support the catalog level. Use an empty string as a placeholder.

The following examples show the ID formats for common types:

maxcompute-table:::project_name:[schema_name]:table_name

dlf-table::catalog_id:database_name::table_name

hms-table:instance_id::database_name::table_name

holo-table:instance_id::database_name:schema_name:table_name

mysql-table:(instance_id|encoded_jdbc_url)::database_name::table_name

Note

Where
instance_id: The instance ID. This is required when the data source is registered in instance mode.
encoded_jdbc_url: The URL-encoded JDBC connection string. This is required when the data source is registered by using a connection string.
catalog_id: The DLF catalog ID.
project_name: The MaxCompute project name.
database_name: The database name.
schema_name: The schema name. For the maxcompute type, this is required only when the three-layer model is enabled for the project. If the three-layer model is not enabled, use an empty string as a placeholder.
table_name: The table name.

dlf-table::catalog_id:database_name::table_name

Name

string

The name.

table_name

Comment

string

The comment.

测试表

TableType

string

The table type. The valid values depend on the metadata collector type.

TABLE

PartitionKeys

array

The list of partition keys. This is empty for non-partitioned tables.

string

The partition key.

ds

ParentMetaEntityId

string

The parent-level metadata entity ID. For more information, see Metadata entity concepts.

For types that support schemas (maxcompute/holo/postgresql/sqlserver/hybriddb_for_postgresql/oracle, where the maxcompute type requires the Layer 3 model to be enabled for the project), ParentMetaEntityId is the database pattern to which the table belongs. The format is ${EntityType}:${instance ID or encoded URL}:${DataFolderIdentity}:${DatabaseName}:${PatternName}. Use an empty character as a placeholder for levels that do not exist.
For other types, ParentMetaEntityId is the database to which the table belongs. The format is ${EntityType}:${instance ID or encoded URL}:${DataFolderIdentity}:${DatabaseName}. Use an empty character as a placeholder for levels that do not exist.

Note

For maxcompute and dlf types, use an empty string as a placeholder for the instance ID. For the maxcompute type, the database name is the MaxCompute project name.

Note

For the starrocks type, the data catalog identifier is the catalog name. For the dlf type, the data catalog identifier is the catalog ID. Other types do not support the catalog level. Use an empty string as a placeholder.

The following examples show the ParentMetaEntityId formats for common types:

maxcompute-project:::project_name

maxcompute-schema:::project_name:schema_name (only when the three-layer model is enabled for the project)

dlf-database::catalog_id:database_name

hms-database:instance_id::database_name

holo-schema:instance_id::database_name:schema_name

mysql-database:(instance_id|encoded_jdbc_url)::database_name

Note

Where
instance_id: The instance ID. This is required when the data source is registered in instance mode.
encoded_jdbc_url: The URL-encoded JDBC connection string. This is required when the data source is registered by using a connection string.
catalog_id: The DLF catalog ID.
project_name: The MaxCompute project name.
database_name: The database name.
schema_name: The schema name.

dlf-database::catalog_id:database_name

CreateTime

integer

The creation time, in millisecond-level timestamp.

1736852168000

ModifyTime

integer

The modification time, in millisecond-level timestamp.

1736852168000

TechnicalMetadata

object

The technical metadata.

Owner

string

The owner.

test_user

Location

string

The storage location.

oss://test-bucket/test_tbl

Compressed

boolean

Indicates whether the table is compressed.

false

InputFormat

string

The input format.

org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat

OutputFormat

string

The output format.

org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat

SerializationLibrary

string

The class used by SerDe.

org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe

Parameters

object

The parameter information.

string

The parameter information.

value

BusinessMetadata

object

The business metadata related to DataWorks, including usage instructions, tags, categories, upstream production nodes, and extended information.

Readme

string

The usage instructions.

## 使用说明

CustomAttributes

object

The custom attribute values, where key is the custom attribute identifier and value is the list of attribute values.

array

The list of attribute values.

string

The attribute value.

张三