All Products
Search
Document Center

ApsaraDB RDS:Download backup files

Last Updated:Sep 25, 2024

This topic describes how to download the backup files of an ApsaraDB RDS for MySQL instance. You can use the backup files in the scenarios such as long-term retention of backup files, backup queries, data migration, and audit.

For more information about how to download the backup files of an RDS instance that runs a different database engine, see the following topics:

Before you begin

Before you download the backup files of your RDS instance, you must select a download method based on your business requirements.

Note

The data of the backup files that you download cannot be directly restored to your RDS instance. For more information about to restore an RDS instance, see Overview of data restoration methods.

Scenario

Suggestion

  • Migrate the data of your RDS instance to a different RDS instance or to a self-managed MySQL instance.

  • Restore the data of your RDS instance from a backup file.

Use one of the following methods:

  • Query the data that exists in your RDS instance at a specific point in time.

  • Query the data of a backup file of your RDS instance.

Use one of the following methods:

  • Restore the backup file to a new pay-as-you-go or serverless RDS instance. After you verify that the restoration is complete, release the pay-as-you-go or serverless RDS instance. For more information, see Restore full data.

  • Download the backup file, restore the backup file to a self-managed MySQL instance, and then query the data from the self-managed MySQL instance based on the descriptions in this topic.

  • Save the backup file of your RDS instance to your computer.

  • Download the backup file of your RDS instance and use the backup file for auditing.

Use one of the following methods:

  • Upload the backup file of your RDS instance to an Object Storage Service (OSS) bucket.

Backup files do not consume the storage capacity of your RDS instance. Backup files are stored in the provisioned backup storage. You can use one of the following methods to store the backup file of your RDS instance in an OSS bucket:

  • If your RDS instance that uses cloud disks meets relevant requirements, you can create an advanced download task and configure the task to download the backup files of your RDS instance to an OSS bucket. If your RDS that uses cloud disks does not meet relevant requirements or uses local disks, you can download the backup files of your RDS instance by following the instructions provided in this topic and then upload the backup files to an OSS bucket. For more information, see How do I upload large objects to OSS?

  • Use Database Backup (DBS) to back up your RDS instance. Make sure that you select an OSS bucket as your storage. For more information, see Configure a backup schedule.

  • Retain the backup files of your RDS instance for a long period of time.

  • Configure the system to automatically download the backup files of your RDS instance.

Billing rules

If your RDS instance uses local disks and you download the backup files of the RDS instance, you are charged fees for Internet traffic generated for the download. If your RDS instance uses cloud disks and you download the backup files of the RDS instance, you are charged fees for file conversion and Internet traffic generated for the download.

Traffic fee

  • Downloads over an internal network: You are not charged for the traffic that you consume.

  • Downloads over the Internet: After the free quota of 500 GB per instance-month is exhausted, you are charged for the excess traffic that you consume based on the pay-as-you-go billing method. Fees are deducted from your account on a daily basis. For more information, see Network traffic fees.

Note
  • When you start a task to download the snapshot backup file of an RDS instance that uses cloud disks, the system converts the backup file and generates a download task. If the task is not complete, the download URL cannot be obtained. In this case, you are not charged for Internet traffic. In-progress tasks and failed tasks are considered not complete. After the task is complete, you are charged for the traffic generated when you download the backup file by using the URL.

  • When you download the physical backup file of an RDS instance that uses local disks, the system does not convert the backup file and directly provides you with the download URL. You are charged for the traffic generated when you download the backup file by using the URL.

  • To view the volume of traffic that you consumed to download backup files over the Internet, you can log on to the ApsaraDB RDS console, find your RDS instance, and then click the ID of the instance to go to the Basic Information page of the RDS instance. Then, you can view the Backup Downloads parameter in the Usage Statistics section of the page.

  • We recommend that you purchase a subscription network plan to offset the fee for the Internet traffic that is used to download the backup files. A large capacity of a network plan indicates a high discount. For more information, see Use network plans.

Backup set conversion fee

If you use the advanced download feature, the snapshot backup files are converted into CSV files with or without headers, SQL, or Parquet files, and you are charged for conversions. No free quota is provided for backup set conversion. The following table describes the unit prices for backup set conversion in different regions.

Region

Unit price (USD per GB)

Public cloud

0.03125

Note

When you start a task to download the snapshot backup file of an RDS instance that uses cloud disks, the system generates a task to convert the backup file. If the conversion task fails, you are not charged for backup set conversion.

Download methods

RDS instance that uses cloud disks

You can create an advanced download task by point in time or backup set. You can set the Download Destination parameter to URL or directly upload the downloaded data to your OSS bucket to facilitate data analysis and offline archiving.

Prerequisites

  • Your RDS instance meets the following requirements:

    • The RDS instance runs MySQL 8.0 or MySQL 5.7. Serverless RDS instances are supported.

    • The RDS instance uses cloud disks.

    Note
    • You can go to the Basic Information page of the RDS instance to view the preceding information.

    • If you enable the data archiving feature of general Enterprise SSDs (ESSDs) for your RDS instance and archive the data of the RDS instance to Object Storage Service (OSS) buckets, the advanced download feature is not supported for the RDS instance. As a result, the backup download tasks fail. For more information, see Use the data archiving feature of general ESSDs.

  • Your RDS instance resides in one of the following regions(only for public cloud): China (Chengdu), China (Guangzhou), China (Qingdao), China (Beijing), China (Shanghai), China (Zhangjiakou), China (Hangzhou), China (Shenzhen), China (Hong Kong), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo), Singapore, US (Silicon Valley), US (Virginia), and Germany (Frankfurt).

    Note

    The feature will be available in other regions soon.

  • The minor engine version of your RDS instance must be a later version than 20201031. For more information about how to update the minor engine version, see Update the minor engine version.

    Important

    If the minor engine version of your RDS instance meets the requirements but you cannot use the advanced download feature in the ApsaraDB RDS console, we recommend that you check your hardware. If the Download Instance Backup button is dimmed, the advanced download feature is unavailable. The advanced download feature is not supported for some RDS instances due to hardware limits. If you want to use the advanced download feature, you must check whether the database proxy feature is enabled for your RDS instance and then update the minor engine version of the instance to resolve this issue. For more information, see Enable the database proxy feature. The following list describes the details:

    • If the database proxy feature is disabled, directly update the minor engine version. After the update is complete, go to the ApsaraDB RDS console to check whether the Download Instance Backup button is available. For more information, see Update the minor engine version.

    • If the database proxy feature is enabled, you cannot directly update the minor engine version to resolve this issue.

      In this case, disable the database proxy feature and then update the minor engine version. After the update is complete, go to the ApsaraDB RDS console to check whether the Download Instance Backup button is available. For more information, see Disable the dedicated proxy feature and Update the minor engine version. For subsequent use, you can enable the feature based on your business requirements.

  • The disk encryption feature is disabled for your RDS instance. If the RDS instance uses cloud disks and cloud disk encryption is enabled for the RDS instance, the advanced download feature is not supported.

  • The Resource Access Management (RAM) user that you want to use to log on to your RDS instance is granted the permissions to download backup files. For more information about how to grant permissions to a RAM user, see Grant backup file download permissions to a RAM user with read-only permissions.

    RAM user permissions

    The authentication capability of the advanced download feature is provided based on the policy management feature of RAM. You can attach a policy to the RAM user to allow the RAM user to obtain the download URLs.

    • If you cannot use the advanced download feature, such as creating and querying advanced download tasks, check whether the AliyunDBSFullAccess policy is attached to the RAM user that you use. For more information, see Grant permissions to the RAM user.

    • If you want the RAM user to use DBS but do not want the RAM user to obtain URLs for advanced downloads, you can create a custom policy to prohibit the RAM user from obtaining URLs for advanced downloads. For more information, see Create a custom policy.

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Deny",
                  "Action": "dbs:DescribeDownloadBackupsetStorageInfo",
                  "Resource": "*"
              }
          ]
      }

      After you create the custom policy, you must attach the policy to the RAM user. For more information, see Grant permissions to a RAM user. Then, the RAM user cannot obtain the download URLs for advanced downloads.

Limits

  • The advanced download feature allows you to export most information about schemas.

    • You can export column information, primary key indexes, non-primary key indexes, unique indexes, partition table information, table engines, and table-level or database-level character sets and character collations.

    • You cannot export expression indexes, foreign keys, generated columns, hidden columns, views, functions, stored procedures, system variables, or triggers.

  • The advanced download feature does not support fields of spatial data types. If your RDS instance contains fields of the following spatial data types, backup file conversions fail:

    GEOMETRY, POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, and GEOMETRYCOLLECTION

  • The CSV files that you export from your RDS instance do not contain the following system databases:

    information_schema, mysql, performance_schema, sys, and __recycle_bin__

  • If you set the Download Destination parameter to OSS, only the Standard storage class is supported. For more information about how to change the storage class, see Convert storage classes.

Procedure

  1. Log on to the ApsaraDB RDS console and go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the instance ID.

  2. In the left-side navigation pane, click Backup and Restoration.

  3. On the page that appears, click the Base Backups tab and then the Data Backup tab.

  4. Find the backup set that you want to download and click Download Instance Backup File in the Actions column.

    Note
    • By default, the ApsaraDB RDS console displays the backup sets that are generated over the most recent eight days. If you want to view the backup sets that are generated eight days ago, you must change the default time range.

    • If Download Instance Backup File is not displayed, check whether the major engine version or region of the RDS instance meets the requirements that are described in Prerequisites.

  5. In the Point in Time and Backup Set step of the wizard, select Download by Point in Time or Download by Backup Set and then click Next.

    Note

    The Download by Point in Time option is available only when the log backup feature is enabled for your RDS instance. For more information about how to enable the log backup feature, see Use the log backup feature.

  6. In the Instance, Database, and Table step of the wizard, click Next.

    Note

    By default, Instance Download is selected.

  7. In the Destination and Format step of the wizard, configure the Download Destination parameter and complete the relevant configurations.

    Important
    • We recommend that you set the Download Destination parameter to OSS to ensure efficiency.

    • The backup download task cannot be canceled after it is started.

    Set the Download Destination parameter to OSS (recommend)

    Note

    If you set the Download Destination parameter to OSS, data is directly downloaded to your OSS bucket. You can use and delete the data based on your business requirements.

    1. Configure the Select Bucket and Directory Prefix parameters.

    2. Configure the Download Format parameter.

      Note
      • You can download the backup files in one of the following formats: CSV without headers, CSV with headers, SQL, and Parquet.

      • If you do not have the permissions to access OSS resources, follow the instructions in the console and click Authorize and then Confirm Authorization Policy. After the permissions are granted, configure the parameters for the advanced download task.

    3. Read and select the required content and click Complete.

    4. Go to the Backup Download tab.

      Wait until the Status parameter of the task changes from Running to Finished. You can view and use the downloaded file in the specified OSS bucket.

      Important
      • Computing resources are competed to convert the downloaded backup sets. As a result, temporary resource allocation may fail, and the download task fails. The download task may also fail due to unsupported special data formats. If your download task fails, try again or contact DBS technical support.

      • No fees are generated for failed tasks.

    Set the Download Destination parameter to URL

    Note

    If you set the Download Destination parameter to URL, the downloaded data is temporarily stored in the built-in storage of DBS. No storage fees are generated. For more information, see Built-in storage and OSS.

    1. Configure the Download Format parameter, read and select the required content, and then click Complete.

      Note

      You can download the backup files in one of the following formats: CSV without headers, CSV with headers, SQL, and Parquet.

    2. Wait until the Status parameter of the task changes from Running to Finished.

      Important
      • Computing resources are competed to convert the downloaded backup sets. As a result, temporary resource allocation may fail, and the download task fails. The download task may also fail due to unsupported special data formats. If your download task fails, try again or contact DBS technical support.

      • No fees are generated for failed tasks.

    3. Click Generate Link in the Download Destination column.

    4. In the Generate Link dialog box, configure the Validity Period parameter and click Generate Link. Then, you can download backup data by using the generated URL. For more information, see Appendix: Download commands.

      Important
      • You can obtain the download URL within three days after the download task is complete. The validity period of a download URL can be set to 5 minutes to 1 day. The default validity period is 2 hours.

        After a download task is complete, the download task and the download URL of the task expire three days later. After the expiration, the data of the download task is automatically cleared within a specific period of time. If you do not retain the data within the specified period of time but want to use the data, you must initiate a new download task and obtain a new download URL.

      • We recommend that you save the URL at the earliest opportunity and keep the URL confidential.

      • If you use a third-party download tool to download the backup set, additional download traffic may be generated. As a result, you may be charged additional fees. Proceed with caution.

      • When you perform an advanced download, backup file conversion fees and traffic fees are generated. For more information, see Billing.

RDS instances that uses local disks

Prerequisites

  • The RDS instance uses local disks.

    Note

    You can go to the Basic Information page of the RDS instance to view the preceding information.

  • The RAM user that you want to use to log on to your RDS instance is granted the permissions to download backup files. For more information about how to grant permissions to a RAM user, see Grant backup file download permissions to a RAM user with read-only permissions.

Procedure

  1. Log on to the ApsaraDB RDS console and go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the instance ID.

  2. In the left-side navigation pane, click Backup and Restoration. On the page that appears, click the Base Backups tab.

  3. Click the Data Backup tab or the Log Backup tab.

  4. Select a time range. This step is required if you want to view the backup files that are generated eight days ago. The default time range spans the most recent eight days.

  5. Find the backup set that you want to download. In the Actions column, click Download Instance Backup File on the Data Backup tab or Download on the Log Backup tab.

  6. In the dialog box that appears, click I have learned the billing rules for backup file and I want to download backup files over the Internet or copy the URL from which you can download the backup file.

    • Copy Internal URL: If you want to connect your virtual private cloud (VPC)-type ECS instance to the RDS instance over an internal network, you can log on to your Elastic Compute Service (ECS) instance and use the internal URL to download the backup file. This method is faster and more secure.

    • Copy Public URL: If the RDS instance cannot be connected over an internal network, you can use the public URL to download the log backup file.

    Note
    • Select I have learnt the billing rules for backup file download. If you use the public URL to download the backup file, you are charged for the excess Internet traffic that you consume. For more information, see Billing.

    • The URLs that can be used to download backup files are valid for only 1 hour. We recommend that you download the URLs at the earliest opportunity. If the download URLs expire, you can refresh the page to obtain the latest download URLs.

    • The download tasks that are run before the URLs expire are not affected after the URLs expire.

    • For more information about how to use the internal or public URL to download backup files, see Appendix: Download commands.

    • If you use the internal URL to download the backup files, make sure that the server to which you log on and the RDS instance reside in the same VPC. If the server and the RDS instance reside in VPCs of different regions or if the server resides in the classic network and the RDS instance resides in a VPC, you cannot download the backup files by using the internal URL on the server.

    • The log backup files that are described in this section are the log backup files that are stored on the RDS instance. For more information about how to obtain the log files, see How do I use the mysqlbinlog command to view the binary logs of an ApsaraDB RDS for MySQL instance?.

    • If the RDS instance runs RDS High-availability Edition or RDS Enterprise Edition, both the primary RDS instance and the secondary RDS instances generate log backup files. You can view the IDs of the primary RDS instance and the secondary RDS instances on the Service Availability page.

    • If you want to restore the data of the RDS instance to a self-managed MySQL instance, the point in time at which the used data backup file is generated must be within the restorable time range that is supported by the log backup file. In addition, the log backup file and the data backup file must have the same instance ID.

Appendix: Download commands

This section describes the commands that are commonly used to download backup files.

Note
  • If the speed of a download is lower than 64 KB per second, the download may be interrupted. When you download a backup file, make ensure that the network status is optimal.

  • If you want to download a backup file to a disk that is attached by using ossfs, you must adjust the value of the multipart_size parameter for ossfs. The maximum value of this parameter is 100 GB. If the size of the backup file that you want to download exceeds 100 GB, the download fails. For more information about ossfs and its parameter settings, see ossfs and Options supported by ossfs.

  • We recommend that you use the commands, such as wget and curl, that are provided in this topic to download a data backup file. If you use a third-party tool to download a backup file, the backup file may be downloaded multiple times. As a result, the amount of data that is downloaded is greater than the size of the backup file, and you may be charged for the traffic that is consumed to download the excess amount of data over the Internet.

wget

Command: nohup wget -c -t 0 "The URL of the backup file to download" -O The destination path and file name > The file to which the downloaded data is saved &
Example: nohup wget -c -t 0 "https://example.aliyundoc.com/examplebackup.qp.xb" -O /backup/examplebackup.qp.xb > /tmp/download.log &
The following list provides the meanings of the options in the preceding command:
-t 0: enables an unlimited number of retries. 
-c: enables resumable uploads. 
-O: specifies the save path and name of the backup file that you want to download. 
nohup: prevents the download from being interrupted when you accidentally replicate data or disconnect your database client during the download. If you specify this option, the download process automatically exits after the download is completed.

curl

Command: nohup curl -C - --retry 10 "The URL that is used to download the backup file" -o The user-defined name of the downloaded backup file> The file to which the downloaded data is saved&
Example: nohup curl -C - --retry 10 "https://example.aliyundoc.com/examplebackup.qp.xb" -o backup.qp.xb > /tmp/download.log &
The following list provides the meanings of the options in the preceding command:
--retry 10: enables up to 10 retries if the download fails. 
-C -: enables resumable uploads. 
-o: specifies the save path and name of the backup file that you want to download. 
nohup: prevents the download from being interrupted when you accidentally replicate data or disconnect your database client during the download. If you specify this option, the download process automatically exits after the download is completed.

Note

For more information about how to obtain the download URL of a backup file, see Download methods.

References

FAQ

Can I upload the backup files of my RDS instance to an OSS bucket?

Yes, you can create an advanced download task for your RDS instance that uses cloud disks by point in time or backup set. Then, you can directly upload the downloaded backup files to your OSS bucket. For more information, see Download methods. You can also download the backup files of your RDS instance that uses cloud disks or local disks to your computer. Then, you can upload the downloaded backup files to your OSS bucket. For more information, see How do I upload large objects to OSS?

Does ApsaraDB RDS limit bandwidth when I download the backup files of my RDS instance over the Internet?

ApsaraDB RDS does not impose limits on bandwidth. The speed of downloading backup files over the Internet depends on the bandwidth of your server. If the bandwidth of your server is limited, the speed of downloading backup files may be affected. If the amount of bandwidth that you consume to download backup files over the Internet exceeds the free quota, you are charged for the excess bandwidth that you consume. For more information, see Billing rules.

How do I use the data backup files and log backup files that I downloaded?

If your RDS instance uses local disks, you can restore the data of the backup files that you downloaded by using the standard download feature to a self-managed instance. For more information, see Restore the data of an ApsaraDB RDS for MySQL instance from a physical backup file to a self-managed MySQL database or Restore the data of an ApsaraDB RDS for MySQL instance from a logical backup file to a self-managed MySQL instance.

Can I change the default format of the physical backup files of an RDS instance that uses lock disks from qp.xb to the .gz format?

  • If the restoration feature for individual databases and tables is enabled for the RDS instance, the backup files of the RDS instance must be compressed in the qp.xb format. For more information, see Restore individual databases and tables.

  • If the restoration feature for individual databases and tables is disabled for the RDS instance, you can call the ModifyBackupPolicy operation to change the compression format of the backup files of the RDS instance from qp.xb to tar.gz. For more information, see Restore individual databases and tables and ModifyBackupPolicy. The operation changes only the compression method of the backup files of the RDS instance.

My RDS instance uses cloud disks. What do I do if the ERROR 1148 (42000): The used command is not allowed with this MySQL version error message appears when I restore the data of a backup file that I downloaded by using the advanced download feature to a self-managed MySQL instance?

Execute the SHOW VARIABLES LIKE 'local_infile'; statement on the self-managed MySQL instance. If OFF is displayed in the output, execute the SET statement to set the global local_infile parameter to 1 to enable file import. After you perform the preceding operations, rerun the import script.

After I download a backup file, why is the size of the backup file that is displayed in the ApsaraDB RDS console inconsistent with the size of the downloaded backup file?

The downloaded backup file is compressed. As a result, the size of the downloaded backup file is smaller than the file size that is displayed in the ApsaraDB RDS console. You can check the data integrity after the data is restored from the backup file. For more information, see Restore the data of an ApsaraDB RDS for MySQL instance to a self-managed MySQL instance by using snapshot backup files.

Can I directly restore the backup files that are downloaded from the ApsaraDB RDS console to another RDS instance?

No, you cannot directly restore the backup files that are downloaded from the ApsaraDB RDS console to another RDS instance. If you want to restore the backup files to another RDS instance, perform the following operations: Restore the backup files to a self-managed MySQL instance and use DTS or mysqldump to migrate data from the self-managed MySQL instance to the required RDS instance. For more information, see Restore the data of an ApsaraDB RDS for MySQL instance from a physical backup file to a self-managed MySQL database, Migrate data from a self-managed MySQL database to an ApsaraDB RDS for MySQL instance, and Use the mysqldump extension to migrate data from a self-managed MySQL instance to an ApsaraDB RDS for MySQL instance.

Can I cancel an ongoing backup download task?

No, you cannot cancel a backup download task after it is started.