This topic describes how to use the data archiving feature in Data Management (DMS).

Background information

As business develops, online databases are continuously growing in size. As a result, table queries and business performance are affected.

To resolve the preceding issues, DMS provides the data archiving feature in addition to database sharding and table partitioning. This feature allows you to periodically archive the data in large tables to Object Storage Service (OSS). Then, you can submit tickets for clearing the historical data in DMS to delete the archived data from the original tables. This enhances data query performance and reduces storage costs.

Billing

DMS creates a backup schedule in Database Backup (DBS) to periodically archive data to OSS. You are not charged for data archiving in DMS but charged for data backup or storage in DBS. The backup or storage fees depend on the amount of data to be backed up. For more information, see Billing overview in DBS documentation.

Prerequisites

  • An ApsaraDB RDS for MySQL database or a PolarDB for MySQL database is used.
  • DBS is activated. For more information about how to activate DBS, see How do I activate DBS?.
  • Data Lake Analytics (DLA) is activated. For more information about how to activate DLA, see Activate DLA.
  • You are a DMS administrator, database administrator (DBA), or regular user. A regular user must have the export permissions on the database. For more information about permissions, see Permission management.

Procedure

  1. Log on to the DMS console V5.0.
    Note To switch to the previous version of the DMS console, click the 5租户头像 icon in the lower-right corner of the page. For more information, see Switch to the previous version of the DMS console.
  2. On the New Data Archive Work Order page, set the parameters that are described in the following table.
    Data archiving configurations
    Parameter Description
    Task Name The name of the data archiving task. Clarify the task name to reduce unnecessary communication.
    Archiving Destination The place to which data is archived. Set the value to OSS Buckets Connected to DBS.
    Database The database to be archived.
    Archive configuration The configurations for data archiving, including the one or more tables to be archived and the filter conditions. Enter a WHERE clause without the WHERE keyword in the Filter Condition field.
    Variable configuration Optional. The variable that you want to use for data archiving. Create a variable as needed. After you create the variable, you can use the variable in the ${Variable name} format to specify a point in time or time period when you set the Archive configuration parameter. For more information about the rules of variables, see Configure variables.
    Note All custom variables are offset from the data timestamp, which is one day before the current time. In other words, the value of a custom variable indicates the day before the date to which the specified offset points.

    For example, you have created a variable named 6_month_ago in the yyyy-MM-dd format and set the offset to -6 Month. In this case, if the current date is January 10, 2021, the value of the 6_month_ago variable indicates July 9, 2020.

    Operation mode The method to be used for running the task. Valid values:
    • Single execution: The task is immediately run after the ticket is approved. The task can be run only once.
    • Cyclic scheduling: After the ticket is approved, the task is scheduled to be run based on the scheduling cycle that you specify. You can change the scheduling cycle in the Apply step of the ticket. For more information, see the Scheduling properties table in the Configure a task flow topic.
  3. Click Submit.
    DMS checks whether your configurations are valid.
  4. Click Submit.
    After the ticket is approved, DMS runs the task based on the configurations.

What to do next

  1. Create a data lake in DBS for backup sets. For more information, see Create a data lake.
  2. Register a DLA instance with DMS. For more information, see Register an ApsaraDB instance.

    After you register the DLA instance with DMS, you can query data on the SQLConsole tab and perform data analysis and data development by using the task orchestration feature. For more information, see Configure a task flow.