Data structures - Data Lake Formation - Alibaba Cloud Documentation Center

This topic describes the data structures related to the metadata API.

GetViewResponse

Name	Type	Description	Example
-	object	The details of the view.
id	string	The view ID.	1
name	string	The view name.	view_demo
schema	ViewSchema	The view structure.

GetTableResponse

Name	Type	Description	Example
-	object	The details of the table.
id	string	The UUID of the table.	1
name	string	The table name.	table_test
path	string	The table path.	oss://clg-paimon-xxx/db-xxx/tbl-xxxx
isExternal	string	External.	true
schemaId	string	The ID of the table schema.	1
schema	Schema	The table schema.
owner	string	The owner of the table.	ALIYUN$1234567
createdAt	long	The time when the table was created.	1741266704867
createdBy	string	The user who created the table.	acs:ram:1234567:root
updatedAt	long	The time when the table was last updated.	1741266704867
updatedBy	string	Updater	acs:ram:1045689747920334:root

Partition

Name	Type	Description	Example
-	object	The details of the partition.
spec	map	The key-value pairs of the partition values.	{"year":"2025"}
recordCount	long	The number of records.	1
fileSizeInBytes	long	The file size in bytes.	1024
fileCount	long	The number of files.	1
lastFileCreationTime	long	The time when the latest file was created.	1741762385537

TableSnapshot

Name	Type	Description	Example
-	object	The information about the table snapshot.
snapshot	Snapshot	The snapshot information.
recordCount	long	The number of records.	table_test
fileSizeInBytes	long	The file size in bytes.	1024
fileCount	long	The number of files.	24
lastFileCreationTime	long	The time when the latest file was created.	1741701564261

Identifier

Name	Type	Description	Example
-	object	The table identifier.
databaseName	string	The database name.	db_test
tableName	string	The table name.	table_test

Schema

Name	Type	Description	Example
-	object	The table schema.
fields	array	The list of table fields.
dataField	DataField	The table field.
partitionKeys	array	The list of partition keys.	["partition_key1", "partition_key2"]
partitionKey	string	The partition key.	partition_key1
primaryKeys	array	The list of primary keys.	["pk1", "pk2"]
primaryKey	string	The primary key.	pk1
options	map	The table configuration.	{"manifest.merge-min-count": "10"}
comment	string	The table description.	test_comment

DataField

Name	Type	Description	Example
-	object	The table field.
id	integer	The field ID.	1
name	string	The field name.	table_test
type	DataType	The data type.

DataType

Name	Type	Description	Example
-	object	The data type. This can be one of the following specific data types.	ArrayType
primitiveType	string	The primitive data type. Valid values include BIGINT, BINARY, BOOLEAN, CHAR, DATE, DECIMAL, DOUBLE, FLOAT, INT, TIMESTAMP, SMALLINT, TINYINT, VARBINARY, VARCHAR, and VARIANT.	BIGINT
arrayType	ArrayType	The array type.
multisetType	MultisetType	The collection type.
mapType	MapType	The key-value pair type.
rowType	RowType	The composite row type.

SchemaChange

Name	Type	Description	Example
-	object	The information about the schema evolution. This can be one of the following data structures.
setOption	SetOption	Updates the table option.
RemoveOption	RemoveOption	Deletes the table option.
UpdateComment	UpdateComment	Updates the table description.
AddColumn	AddColumn	Adds a column.
RenameColumn	RenameColumn	Renames a column.
DropColumn	DropColumn	Deletes a column.
UpdateColumnComment	UpdateColumnComment	Updates the column description.
UpdateColumnType	UpdateColumnType	Updates the column type.
UpdateColumnPosition	UpdateColumnPosition	Updates the column position.
UpdateColumnNullability	UpdateColumnNullability	Updates the nullability of the column.

ViewSchema

Name	Type	Description	Example
-	object	The view structure.
fields	array	The list of view fields.
dataField	DataField	The view field.
query	string	The view search statement.
dialects	map<string, string>	The dialect.	{ "flink": "xxxx", "spark": "xxxx" }
comment	string	The comment.
options	map	The properties.

Snapshot

Name	Type	Nullable	Description	Example
-	object		The snapshot information.
version	integer	Yes	The version of the snapshot file. The current version is 3.	3
id	long	No	The snapshot ID.	6
schemaId	long	No	The ID of the table schema.	0
baseManifestList	string	No	The base manifest list.	manifest-list-0c6fcaa1-b6bc-413d-9634-6471cce4f37e-2
baseManifestListSize	long	Yes	The size of the base manifest list.	845
deltaManifestList	string	No	The manifest list that records all new changes in this snapshot.	manifest-list-0c6fcaa1-b6bc-413d-9634-6471cce4f37e-3
deltaManifestListSize	long	Yes	The size of the manifest list that records all new changes in this snapshot.	759
changelogManifestList	string	Yes	The manifest list that records all changelogs generated in this snapshot.	manifest-list-0c6fcaa1-b6bc-413d-9634-6471cce4f37e-4
changelogManifestListSize	long	Yes	The size of the manifest list that records all changelogs generated in this snapshot.	759
indexManifest	string	Yes	The manifest that records all index files of this table.	index-manifest-7cbdaa47-9578-43ac-ae6e-a7ec0486bba8-0
commitUser	string	No	Usually a UUID, used to resume stream writing.	446aac61-bd5c-4fa0-8b14-846fa8810651
commitIdentifier	long	No	The transaction ID for stream writing.	9223372036854775807
commitKind	string	No	The type of stream writing. Valid values include APPEND, COMPACT, OVERWRITE, and ANALYZE.	COMPACT
timeMillis	long	No	The commit time in milliseconds.	1741701564261
logOffsets	Map<Integer, Long>	Yes	The commit log offset.	{0: 1L, 1: 3L}
totalRecordCount	long	Yes	The total number of records for all changes in this snapshot.	1024
deltaRecordCount	long	Yes	The number of records for all new changes in this snapshot.	10
changelogRecordCount	long	Yes	The number of records for all changelogs generated in this snapshot.	100
watermark	long	Yes	The watermark.	-9223372036854775808
statistics	string	Yes	The name of the statistics file for this table.

Instant

Name	Type	Description	Example
-	object	One of the following data structures.
snapshotInstant	SnapshotInstant	The snapshot constant.	1
tagInstant	TagInstant	The tag constant.	test_tag

SnapshotInstant

Name	Type	Description	Example
-	object	The snapshot information constant.
snapshotId	long	The snapshot ID.	1

TagInstant

Name	Type	Description	Example
-	object	The tag information constant.
tagName	string	The tag name.	1

ArrayType

Name	Type	Description	Example
-	object	The array data type.
type	string	Pattern: ^ARRAY.*	ARRAY
element	DataType	The data type.	INT

MultisetType

Name	Type	Description	Example
-	object	The collection data type.
type	string	Pattern: ^MULTISET.*	MULTISET
element	DataType	The data type.	BIGINT

MapType

Name	Type	Description	Example
-	object	The data type.
type	string	Pattern: ^MAP.*	MAP
key	DataType	The data type.	STRING
value	DataType	The data type.	BOOLEAN

RowType

Name	Type	Description	Example
-	object	The row data type.
type	string	Pattern: ^ROW.*	ROW
fields	array	The array of fields.
field	DataField	The field.

SetOption

Name	Type	Description	Example
-	object	Sets a table option.
type	string	Constant: setOption.	setOption
key	string	The key in the table option key-value pair.	'manifest.delete-file-drop-stats'
value	string	The value in the table option key-value pair.	'true'

RemoveOption

Name	Type	Description	Example
-	object	Deletes a table option.
type	string	Constant: removeOption.	removeOption
key	string	The key in the table option key-value pair.	'manifest.delete-file-drop-stats'

UpdateComment

Name	Type	Description	Example
-	object	Updates the table description.
type	string	Constant: updateComment.	updateComment
comment	string	The table description.	comment_test

AddColumn

Name	Type	Description	Example
-	object	Adds a column.
type	string	Constant: addColumn.	addColumn
fieldNames	array	The names of the table fields.
dataType	DataType	The data type.
comment	string	The description.	comment_test
move	Move	The move information.

RenameColumn

Name	Type	Description	Example
-	object	You can add a column.
type	string	Constant: renameColumn.	renameColumn
fieldNames	array	The array of table field names.
newName	string	The new field name.	new_col_test

DropColumn

Name	Type	Description	Example
-	object	You can add a column.
type	string	Constant: dropColumn.	dropColumn
fieldNames	array	The array of table field names.

UpdateColumnComment

Name	Type	Description	Example
-	object	You can add a column.
type	string	Constant: updateColumnComment.	updateColumnComment
fieldNames	array	The array of table field names.
newComment	string	The new field description.	col_comment_test

UpdateColumnType

Name	Type	Description	Example
-	object	You can add a column.
type	string	Constant: updateColumnType.	updateColumnType
fieldNames	array	The array of table field names.
newDataType	DataType	The new field data type.
keepNullability	boolean	Indicates whether the column is nullable.	true

UpdateColumnPosition

Name	Type	Description	Example
-	object	Updates the column position.
type	string	Constant: updateColumnPosition.	updateColumnPosition
fieldNames	array	The array of table field names.
move	Move	The move information.

UpdateColumnNullability

Name	Type	Description	Example
-	object	Update the table location.
type	string	Constant: updateColumnNullability.	updateColumnNullability
newNullability	boolean	Indicates whether the column is nullable.	true

Move

Name	Type	Description	Example
-	object	You can add a column.
fieldName	string	The field name.	col_test
referenceFieldName	string	The name of the reference field.	other_col_test
type	string	The move type. Valid values are FIRST, AFTER, BEFORE, and LAST.	FIRST