Query data archived to OSS buckets or dedicated storage - Data Management

Query historical data archived to Object Storage Service (OSS) buckets or dedicated storage directly from the DMS console, without restoring the data to a live database.

Prerequisites

Before you begin, make sure that:

Data has been archived to OSS buckets or dedicated storage
You are the creator or a stakeholder of the data archiving ticket

An old user refers to a user who used the data archiving feature before the archived data query feature was released on September 14, 2022.

A new user refers to a user who uses the data archiving feature after the archived data query feature is released.

Billing

Querying archived data is free during public preview.

Limitations

Only the creator or stakeholders of a data archiving ticket can query its archived data.
Queries must go through a logical data warehouse. Direct queries against OSS buckets or dedicated storage are not supported.

Query data archived to OSS buckets

To query OSS-archived data, find your archiving ticket, authenticate with OSS credentials on first access, and run the auto-generated SQL query in the logical data warehouse.

Log on to the DMS console V5.0.
In the top navigation bar, choose Solution > Data Archiving.
If you use the DMS console in simple mode, move the pointer over the icon in the upper-left corner and choose All functions > Solution > Data Archiving.
On the Archive DataTickets page, set View to Archive View.
Enter the ticket number, database name, or table name in the search box, then click Search.
Find the target ticket and click Query in the Operation column.
Important
If this is the first time you are querying a specific OSS bucket, you must authenticate with the bucket's credentials. In the dialog box, enter the database account (your AccessKey ID) and password (your AccessKey secret), then click OK. The credentials must match the AccessKey pair of the OSS bucket exactly — mismatched credentials prevent the query from running. To create an AccessKey pair, see Create an AccessKey pair.
On the Logical data warehouse page, an SQL query statement is automatically filled in the command execution area.
If a blank page appears, you do not have permission to access the logical data warehouse. Join DingTalk group 24005004020 to request access.
Click Execute. Results appear on the Execution History tab at the bottom of the SQL Console page.
By default, the system returns up to 20 records. To return more records, modify the limit in the query settings before running the query.

Query data archived to dedicated storage

To query data in dedicated storage, find your archiving ticket and run the auto-generated SQL query in the logical data warehouse.

Log on to the DMS console V5.0.
In the top navigation bar, choose Solution > Data Archiving.
On the Archive DataTickets page, set View to Archive View.
Enter the ticket number, database name, or table name in the search box, then click Search.
Find the target ticket and click Query in the Operation column.
On the Logical data warehouse page, a SQL query statement is automatically filled in the statement execution area.
If a blank page appears, you do not have permission to access the logical data warehouse. Join DingTalk group 24005004020 to request access.
Click Execute. Results appear on the Execution History tab at the bottom of the page.
By default, the system returns up to 20 records. To return more records, modify the limit in the query settings before running the query.

FAQ

Are tables in a logical database merged into a single physical table when I query archived data?

Yes. When data in a logical database is archived, all tables are merged into one physical database table. On the Archive DataTickets page in Archive View, click a logical table to query all data across the tables in that logical database.

Can I query archived data across two data archiving tickets?

Yes. Logical data warehouses in Data Management (DMS) support cross-ticket queries, including data from different OSS buckets or the same OSS bucket. Go to the Logical data warehouse page, log on to the relevant OSS buckets, and run LEFT JOIN, INNER JOIN, or RIGHT JOIN statements.

Can I query data archived during a specific time period if periodic archiving is configured?

Yes. With periodic archiving, data is archived in separate batches based on the scheduling cycle. Query the data for a specific period by filtering on the time when the data archiving task was created.