This topic describes how to use the data archiving feature in Data Management (DMS).

Background information

As business develops, online databases are continuously growing in size. As a result, table queries and business performance are affected.

To resolve the preceding issues, DMS provides the data archiving feature in addition to database sharding and table partitioning. This feature allows you to periodically archive the data in large tables to Object Storage Service (OSS). Then, the archived data is deleted from the original tables. This enhances data query performance and reduces storage costs.

Billing

DMS creates a backup schedule in Database Backup (DBS) to periodically archive data to OSS. You are not charged for data archiving in DMS but charged for data backup or storage in DBS. The backup or storage fees depend on the amount of data to be backed up. For more information, see Billing overview in DBS documentation.

Prerequisites

  • An ApsaraDB RDS for MySQL database or a PolarDB for MySQL database is used.
  • DBS is activated. For more information about how to activate DBS, see Activate DBS.
  • Data Lake Analytics (DLA) is activated. For more information about how to activate DLA, see Activate DLA.
  • You are a DMS administrator, database administrator (DBA), or regular user. A regular user must have the export permission on the database. For more information about permissions, see Permission management.

Procedure

  1. Log on to the DMS console.
  2. In the top navigation bar, move the pointer over the More icon and choose Data Plans > Data archiving.
  3. On the New Data Archive Work Order tab, set the parameters that are described in the following table.
    Data archiving configurations
    Parameter Description
    Task Name The name of the data archiving task. Clarify the task name to reduce unnecessary communication.
    Filing method Set the value to OSS.
    Database The database to be archived.
    Archive configuration The configurations for data archiving, including the one or more tables to be archived and the filter conditions. Enter a WHERE clause without the WHERE keyword in the Filter Condition field.
    Variable configuration Optional. The variable that you want to use for data archiving. Create a variable as needed. After you create the variable, you can use the variable in the ${Variable name} format to specify a point in time or time period when you set the Archive configuration parameter. For more information about the rules of variables, see Configure variables.

    For example, create a variable named 6_month_ago in the yyyy-MM-01 format and set the offset to -6 Month.

    The value of this variable is accurate to the day and indicates the first day of the month that is six months ahead of the current month. For example, today is January 10, 2021. The variable value indicates July 1, 2020.

    Operation mode The method to be used for running the task. Valid values:
    • Single execution: The task is immediately run after the ticket is approved. The task can be run only once.
    • Cyclic scheduling: After the ticket is approved, the task is scheduled to be run based on the scheduling cycle that you specify. You can change the scheduling cycle in the Apply step of the ticket. For more information, see the Scheduling properties table in the Task flow topic.
  4. Click Submit.
    DMS checks whether your configurations are valid.
  5. Click Submit for Approval.
    After the ticket is approved, DMS runs the task based on the configurations.

What to do next

  1. Create a data lake in DBS for backup sets. For more information, see Create a data lake.
  2. Register a DLA instance with DMS. For more information, see Register an ApsaraDB instance.

    After you register the DLA instance with DMS, you can query data on the SQLConsole tab and perform data analysis and data development by using the task orchestration feature.