如何进行数据归档 - Data Management - Alibaba Cloud Documentation Center

In the use of an online database, an increasing amount of data is stored in the database. The data that is infrequently accessed occupies much storage space and affect the query performance and business operation. In this case, you can use the data archiving feature provided by Data Management (DMS) to periodically archive data of specific tables in the database to other databases or storage space. You can also perform other operations to reduce the online storage cost. For example, you can delete the archived data from the source tables and optimize the storage usage of the source tables.

Background information

On e-commerce platforms, a large amount of order data is generated every day, and the need for access to these order data diminishes over time. To improve database performance and free up storage space, you can use the data archiving feature of DMS to periodically archive data.

Usage notes

The business that is running may be affected when you use the data archiving feature.
Important
To minimize the impact, we recommend that you use the data archiving feature during off-peak hours.
The data archiving feature is supported only in the Singapore and Indonesia (Jakarta) regions.

Comparison between data archiving of DMS and data migration of DTS

Features
- Data archiving of DMS allows you to configure filter conditions to archive infrequently accessed data to other storage space. This reduces the load on the primary database and improves database performance.
- Data migration of Data Transmission Service (DTS) allows you to migrate data between homogeneous and heterogeneous data sources. For more information, see Overview of data migration scenarios.
Scenarios
- Data archiving of DMS allows you to archive infrequently accessed data to other databases or storage space on a regular basis or at a time.
- Data migration of DTS is applicable to scenarios such as data migration to Alibaba Cloud, data migration between databases within Alibaba Cloud, and database splitting or scale-out.

Comparison among different categories of archiving destinations

Databases

Item	Lindorm	AnalyticDB for MySQL V3.0	AnalyticDB for PostgreSQL	ApsaraDB RDS for MySQL	PolarDB for MySQL
Supported source database types	MySQL: ApsaraDB RDS for MySQL, PolarDB for MySQL, and AnalyticDB for MySQL V3.0 PostgreSQL: ApsaraDB RDS for PostgreSQL and PolarDB for PostgreSQL PolarDB for Xscale (PolarDB-X) Note Both physical and logical databases are supported. A physical database is an actual database. A logical database consists of one or more physical databases to facilitate sharding. For more information, see Logical database. If the source database is a MySQL database, the database account of the source database must have the REPLICATION CLIENT permission.
Billing	You can use the data archiving feature free of charge. However, you may be charged when you purchase and use the destination instance.
Deletion of archived data from sources	After you archive data to a destination database, DMS can automatically delete the archived data from the source tables based on your settings. You no longer need to delete the source data after the data is archived. This reduces data risks. The source table data is deleted without the lock of source tables. The process is stable and fast and does not involve large transactions. The source data is cached in a local temporary table. You can restore the cached data if necessary.
Query of archived data	You can query the archived data in the DMS console. You can also query the archived data in the corresponding database engine.
Configuration of the data archiving destination	You must specify a destination instance. You do not need to specify a destination database or destination table. DMS automatically creates a destination database and a destination table based on the names of the source database and source table.
Change of the table schema	A column is added to the archived tables in the destination to store information such as the ticket ID and archiving time. Data in the tables can be used as usual.
Usage	This engine type supports the analysis of archived data. This engine type can reduce online storage costs.	These engine types support the analysis of archived data and deliver better performance in the analysis. These engine types can reduce online storage costs in an efficient manner.
Data control	These engine types provide the strongest data control. DMS can archive data to a database instance that you manage. You can flexibly process data by interacting within databases.

Storage space

Item	Dedicated storage space (recommended)	Custom OSS bucket	OSS bucket connected to DBS
Supported source database types	MySQL: ApsaraDB RDS for MySQL, PolarDB for MySQL, and AnalyticDB for MySQL V3.0 PostgreSQL: ApsaraDB RDS for PostgreSQL and PolarDB for PostgreSQL MongoDB 3.6 and later Note Both physical and logical databases are supported. If the source database is a MySQL database, the database account of the source database must have the REPLICATION CLIENT permission.	MySQL: ApsaraDB RDS for MySQL, PolarDB for MySQL, and AnalyticDB for MySQL V3.0 PostgreSQL: ApsaraDB RDS for PostgreSQL and PolarDB for PostgreSQL MongoDB 3.6 and later PolarDB-X	MySQL: ApsaraDB RDS for MySQL and PolarDB for MySQL PolarDB for PostgreSQL PolarDB-X Note Only physical databases are supported.
Billing	You are not charged for archiving data to dedicated storage space during public preview.	You can use the data archiving feature free of charge. However, you may be charged when you purchase and use Object Storage Service (OSS) buckets.	DMS creates a backup schedule in Database Backup (DBS). You are charged for data backup and storage based on the amount of data that is backed up. For more information about the billing, see Billing overview. Note By default, a backup schedule of the xlarge type is created in DBS for data archiving. If you want to view archived data, you must activate Data Lake Analytics (DLA). For more information, see Billing overview.
Deletion of archived data from sources	After you archive data to a destination database, DMS can automatically delete the archived data from the source tables based on your settings. You no longer need to delete the source data after the data is archived. This reduces data risks. The source table data is deleted without the lock of source tables. The process is stable and fast and does not involve large transactions. The source data is cached in a local temporary table. You can restore the cached data if necessary.		The archived data cannot be automatically deleted from the source database. You must create a regular data change ticket to manually delete the archived data from the source database.
Query of archived data	You can query the archived data by using the logical data warehouses in DMS. You can also query the archived data by using the file merging feature to achieve higher efficiency.	You can query the archived data by using the logical data warehouses in DMS.	To query the archived data, you must activate DLA in DBS. Then, you can query the archived data in DLA.
Configuration of the data archiving destination	You do not need to focus on the archiving destination.	You must specify an OSS bucket.	You do not need to specify an OSS bucket. DMS automatically creates a backup schedule in DBS to archive data to OSS.
Change of the table schema	The table schema is not changed.		The table schema is not changed. Note If you query the archived data in DLA, the sequences of columns in the tables may be changed.
Usage	This type of storage space supports the analysis of archived data. This type of storage space can reduce some online storage costs.	This type of storage space can reduce the most online storage costs.	This type of storage space can reduce many online storage costs.
Data control	This type of storage space provides strong data control. Data is archived and stored in DMS. You do not need to focus on the archiving destination.	This type of storage space provides strong data control. DMS can archive data to an OSS bucket that you purchase.	This type of storage space provides moderate data control. DMS can archive data to an OSS bucket connected to DBS.

References

FAQ

Q: If I select Clean up the archived data of the original table (delete-No Lock) when I configure parameters for data archiving, is the historical data still available for retrieval?

A: Yes. The data archiving feature regularly archives the data of large tables to other databases. You can query archived historical data in the destination instances to which the data is archived.