Metadata describes a database. Metadata is also called a data dictionary. Metadata includes the database name, the database character set, table names, table sizes, the number of rows in each table, the character set of each table, table columns, table indexes, the description of each table, the data type of each column, the precision of the columns, and the description of the columns. Data Management (DMS) collects metadata from databases such as the information_schema database. The collected metadata may not be accurate and is used only for reference.

Background information

Metadata is used to retrieve and display data in DMS. Metadata is the basis for automatic identification based on sensitive data rules. Metadata can also be used to implement fine-grained access control on databases, tables, columns, rows, and programmable objects. DMS supports different metadata collection scopes and methods based on the control mode of each database instance.

Collection scopes and methods

Collection scope and method Security Collaboration Stable Change and Flexible Management
Metadata collection scope DMS collects and stores all metadata of an instance. For more information, see the Collected metadata section of this topic. DMS collects and stores all metadata of a database. Other metadata is loaded in real time after logon.
Whether to trigger full metadata collection when a database is added to DMS Yes, full metadata collection is triggered when a database is added to DMS. Yes, full metadata collection is triggered when a database is added to DMS.
Full metadata collection at specified points in time By default, DMS collects metadata from all instances of a tenant at six points in time each day.
Note DMS synchronizes the metadata every four hours at 02:00:00, 06:00:00, 10:00:00, 14:00:00, 18:00:00, and 22:00:00.
Full metadata collection at specified points in time is not supported.
On-demand incremental metadata collection
  • In the top navigation bar of the DMS console, click Data Assets. In the left-side navigation pane, click Instance. On the Instance List tab, find the instance that you want to manage, move the pointer over More in the Actions column, and then select Refresh/Sync dictionary to trigger incremental metadata collection. For more information, see Synchronize the data dictionary for an instance.
    Note If you use the previous version of the DMS console as a database administrator (DBA) or DMS administrator, choose System > Instance in the top navigation bar. On the Instance List tab, find the instance that you want to manage, move the pointer over More in the Actions column, and then select Refresh/Sync dictionary to trigger incremental metadata collection.
  • In the left-side instance list, right-click the instance that you want to manage and select Refresh / Sync dictionary to trigger incremental metadata collection.
  • On the SQLConsole tab, click the Refresh button to trigger incremental metadata collection.
  • Execute DDL statements on a table in DMS to trigger incremental metadata collection.
  • In the top navigation bar of the DMS console, click Data Assets. In the left-side navigation pane, click Instance. On the Instance List tab, find the instance that you want to manage, move the pointer over More in the Actions column, and then select Refresh/Sync dictionary to trigger incremental metadata collection. For more information, see Synchronize the data dictionary for an instance.
    Note If you use the previous version of the DMS console as a DBA or DMS administrator, choose System > Instance in the top navigation bar. On the Instance List tab, find the instance that you want to manage, move the pointer over More in the Actions column, and then select Refresh/Sync dictionary to trigger incremental metadata collection.
  • In the left-side instance list, right-click the instance that you want to manage and select Refresh / Sync dictionary to trigger incremental metadata collection.
  • On the SQLConsole tab, click the Refresh button to trigger incremental metadata collection.
  • Execute DDL statements on a table in DMS to trigger incremental metadata collection.
Others N/A. When you log on to a database, DMS collects information about the database based on the permissions of the account that you use to log on.

Use metadata

Metadata can be used to retrieve data and manage permissions. You can export metadata by calling API operations to provide support for operations such as compliance audit.

You can export only the metadata of instances that are managed in Security Collaboration mode by calling API operations.

The following table describes the API operations that are related to metadata.

API Description
GetDatabase Queries details of a database.
GetPhysicalDatabase You can call the GetPhysicalDatabase to obtain the details of a physical library.
ListColumns Queries details of columns in a table.
ListIndexes Queries details of indexes of a table.
GetLogicDatabase Queries details of a logical database.
ListDatabases Queries details of databases in an instance.
ListLogicDatabases Queries details of logical databases.
ListLogicTables Queries details of logical tables in a logical database.
SearchDatabase Queries details of databases.
SearchTable Queries details of tables.
ListTables Queries details of tables in a database.
GetMetaTableDetailInfo Queries the details of a table. These details include columns and indexes.
GetTableDBTopology Queries the topology of a table.
GetTableTopology You can call the GetTableTopology to obtain the topology information of a data table.
GetDBTopology You can call this operation to obtain the topology information of a logical database and its corresponding physical database shards.
GetMetaTableColumn Queries the details about columns in a table.
SyncDatabaseMeta Synchronizes metadata of a database.
SyncInstanceMeta Synchronizes the metadata of all databases in an instance.

Collected metadata

Object MySQL series (1) PostgreSQL series (2) SQL Server Oracle MongoDB Redis
Database
  • Name
  • Character set
 Name Name Name N/A Name
Table
  • Name
  • Description
  • Character set
  • Number of rows
  • Table size
  • Name
  • Description
  • Number of rows
  • Table size
  • Name
  • Description
  • Number of rows
  • Table size
  • Name
  • Description
  • Number of rows
  • Table size
 N/A N/A
Column
  • Name
  • Data type
  • Whether the value of the column can be null
  • Length
  • Precision
  • Description
  • Name
  • Data type
  • Whether the value of the column can be null
  • Length
  • Precision
  • Description
  • Name
  • Data type
  • Whether the value of the column can be null
  • Length
  • Precision
  • Description
  • Name
  • Data type
  • Whether the value of the column can be null
  • Length
  • Precision
  • Description
 N/A N/A
Index
  • Name
  • Type
  • Indexed column
  • Name
  • Type
  • Indexed column
  • Name
  • Type
  • Indexed column
  • Name
  • Type
  • Indexed column
  • Name
  • Indexed column
 N/A
Programmable object
  • Name
  • Type
  • Name
  • Type
  • Name
  • Type
 N/A N/A N/A
Pattern N/A Name N/A N/A N/A N/A
Collection N/A N/A N/A N/A Name N/A
Key N/A N/A N/A N/A N/A
  • Name
  • Type
Note

(1) Native MySQL, PolarDB for MySQL, PolarDB-X, AnalyticDB for MySQL, Data Lake Analytics (DLA), ApsaraDB for ClickHouse, ApsaraDB OceanBase for MySQL, native MariaDB, and ApsaraDB RDS for MariaDB

(2) Native PostgreSQL, PolarDB-O, PolarDB for PostgreSQL, AnalyticDB for PostgreSQL, ApsaraDB OceanBase for Oracle, and ApsaraDB RDS for PPAS