All Products
Search
Document Center

PolarDB:Download backup files

Last Updated:Feb 01, 2024

This topic describes how to download the backup files of a PolarDB cluster.

Release date

November 24, 2023

Note

This feature is in canary release and will be formally launched later.

Scenarios

In addition to the full data restoration and individual database or table restoration features, PolarDB provides the backup download feature. You can download the backup files of a cluster to your computer for the following purposes:

  • Query the data of a backup file.

  • Query the data of a specific point in time.

  • Save backup files to your computer.

  • Download backup files and use backup files for auditing.

  • Upload backup files to an Object Storage Service (OSS) bucket.

Note

The downloaded backup data cannot be directly restored to a PolarDB for MySQL cluster. However, you can restore the downloaded backup data to a self-managed MySQL database. For more information, see Restore the data of a PolarDB cluster to a self-managed MySQL database by using snapshot backup files.

Billing

You are charged fees for file conversion and Internet traffic generated for the download.

Note

If you fail to download backup files, no fees are incurred.

Internet traffic fees

  • Downloads over an internal network: You are not charged for the traffic that you consume.

  • Downloads over the Internet: After the free quota of 500 GB per cluster-month is exhausted, you are charged for the excess traffic that you consume based on the pay-as-you-go billing method. Fees are deducted from your account on a daily basis. For more information, see Network traffic fees.

Note

We recommend that you purchase a subscription network plan to offset the fee for the Internet traffic that is used to download the backup files. A large capacity of a network plan indicates a high discount. For more information, see Use network plans.

File conversion fees

In PolarDB, backup files can be converted into CSV, SQL, or Parquet files. You are charged for file conversions. No free quota is provided for backup file conversion. The following table describes the unit prices for backup file conversion in different regions.

Region

Unit price (USD per GB)

Public cloud-related regions

0.03125

Note

Log on to the PolarDB console. On the Download Backup tab of the Backup and Restoration page, you can view Monthly Backup Data Size (for file conversion data size) and Monthly Internet Traffic.

Prerequisites

Supported versions

The PolarDB for MySQL cluster must meet the following requirements:

  • This feature is supported only on a cluster of PolarDB for MySQL Enterprise Edition.

  • The cluster runs PolarDB for MySQL 5.6, 5.7, 8.0.1, or 8.0.2.

Supported regions

China (Chengdu), China (Guangzhou), China (Qingdao), China (Beijing), China (Shanghai), China (Zhangjiakou), China (Hangzhou), China (Shenzhen), China (Hong Kong), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo), Singapore, US (Silicon Valley), and US (Virginia).

Note

The feature will be available in the other regions soon.

Permission requirements for RAM accounts

The authentication capability of the backup download feature is provided based on the policy management feature of RAM.

Grant the permissions to download backup files

If you cannot use the backup download feature, such as creating and querying download tasks, check whether the AliyunDBSFullAccess role (for permissions to use DBS) is attached to the RAM user that you use. For more information about how to grant permissions to a RAM user, see Grant permissions to a RAM user.

Revoke the permissions to obtain download URLs

If you want the RAM user to use the backup and restoration feature but not to obtain download URLs, you can create a custom policy to revoke the permissions to obtain download URLs. For more information, see Create a custom policy. A sample script:

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Deny",
            "Action": "dbs:DescribeDownloadBackupSetStorageInfo",
            "Resource": "*"
        }
    ]
}

After the custom policy is created, you must attach the custom policy to the RAM user. For more information, see Grant permissions to a RAM user. Then, the RAM user cannot obtain download URLs.

Grant read-only RAM users the permissions to download backup files

For security reasons, if the AliyunPolardbReadOnlyAccess) role (read-only permissions on PolarDB resources) is attached to a RAM user, the read-only RAM user cannot download backup files.

If the RAM user needs to download backup files, you can attach the AliyunDBSReadOnlyAccess role (read-only permissions on DBS resources) to the RAM user. Then, the RAM user can view the download URL for a created download task.

Note

For more information about how to grant permissions to a RAM user, see Grant permissions to a RAM user.

image.png

Other requirements

The disk encryption feature is disabled for your PolarDB cluster. This feature is not supported on a PolarDB cluster with disk encryption feature enabled.

Limits

  • The backup download feature allows you to export most schemas. Some schemas cannot be exported.

    • You can export column information, primary key indexes, non-primary key indexes, unique indexes, partitioned table information, table engines, and table-level or database-level character sets and character collations.

    • You cannot export expression indexes, foreign keys, generated columns, hidden columns, views, functions, stored procedures, system variables, or triggers.

  • The backup download feature does not support fields of spatial data types. If your PolarDB cluster contains fields of the following spatial data types, backup file conversions fail:

    GEOMETRY, POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, and GEOMETRYCOLLECTION

  • The CSV files that you export from your PolarDB cluster do not contain the following system databases:

    information_schema, mysql, performance_schema, sys, and __recycle_bin__

  • If you set the Download Destination parameter to OSS, only Standard storage class is supported. For more information about how to change storage classes, see Convert storage classes.

  • You can download only the data of level-1 backup files. You cannot download the data of level-2 backup files.

Procedure

  1. Log on to the PolarDB console. Select a region in the upper-left corner and click the ID of the cluster to go to the Basic Information page.

  2. In the left-side navigation pane, click Backup and Restoration.

  3. On the Backup and Restoration page, click Data Backups.

  4. Find the backup set and click Download Backup in the Actions column.

    image.png

    Note
    • By default, the PolarDB console displays the backup sets that are generated over the most recent eight days. If you want to view the backup sets that are generated eight days ago, you must change the default time range.

    • If Download Backup is not displayed, check whether the major engine version or region of the cluster meets the requirements that are described in Prerequisites.

  5. In the Select Download Mode step of the wizard, select Download by Time or Download by Backup Set and click Next.

    image.png

  6. In the Select Download Content step of the wizard, click Next.

    Note

    Download Instances is selected by default.

  7. In the Select Download Destination and Format step of the wizard, set the Download Destination parameter.

    Important
    • We recommend that you set the Download Destination parameter to OSS. This method is efficient.

    • The backup download task cannot be canceled after it is started.

    • OSS (recommended): PolarDB writes backup data directly to OSS. You must configure the Bucket Name and Directory Prefix parameters.

      Note

      If you do not have the permissions to access OSS resources, follow the instructions in the console and click Authorize > Confirm Authorization Policy. After the permissions are granted, configure the parameters for the backup download task.

    • URL: PolarDB provides an internal or external URL. You can use this URL to download backup data.

      Note
      • If you set the Download Destination parameter to URL, the downloaded data is temporarily stored in the built-in storage of DBS. For more information, see Built-in storage and OSS.

      • URL downloads involve data packaging. Additional waiting time is required and dependent on the logical size of the backup set. If the logical size of the backup set is more than 1 TB, we recommend that you select OSS.

      image.png

  8. Configure the Download Format parameter.

    Note

    You can set the Download Format parameter to CSV, SQL, or Parquet.

  9. Read and select the instructions, and then click Complete.

  10. The Backup Download tab appears.

    Important
    • When you use download backup files, file conversion fees and Internet traffic fees are generated. For more information, see Billing rules.

    • If you use a third-party download tool to download backup files, additional traffic may be required. As a result, you may be charged additional fees. Proceed with caution. For more information, see Download commands.

    • Computing resources are competed to convert the downloaded backup sets. As a result, temporary resource allocation may fail, and the download task fails. The download task may also fail due to special data formats. If your download task fails, try again or contact DBS technical support.

    • No fees are generated for failed tasks.

    • If you set the Download Destination parameter to OSS and the status of the task changes from Running to Finished, the download is successful. You can view and use the downloaded file in the specified OSS bucket.

      Note

      If you use this method, data is directly downloaded to your OSS bucket, and you can use and delete the data based on your business requirements.

    • If you set the Download Destination parameter to URL and when the status of the task changes to Finished, click Generate Link in the Download Destination column. In the Generate Link dialog box, configure the Validity Period parameter and click Generate Link. Then, you can download backup data by using the generated URL.

      Important
      • The validity period of the URL ranges from 5 minutes to 1 day. The default validity period is 2 hours.

      • We recommend that you save the URL at the earliest opportunity and keep the URL confidential.

      • You can obtain the download URL within three days after the download task is complete. After the URL expires, the downloaded data is deleted. If you want to use the data, you must initiate a new download task and obtain the new download URL.

Note

You can use an SQL file or CSV file that is downloaded and converted by using the backup download feature to restore the data of your PolarDB cluster to a self-managed MySQL database. For more information, see Restore the data of a PolarDB cluster to a self-managed MySQL database by using snapshot backup files.

FAQ

  • How do I use the data backup files and log backup files that I downloaded?

    You can restore the data of the backup files that you downloaded to self-managed MySQL databases. For more information, see Restore the data of a PolarDB cluster to a self-managed MySQL database by using snapshot backup files.

  • What do I do if the ERROR 1148 (42000): The used command is not allowed with this MySQL version error message appears when I restore the data of a backup file that I downloaded by using the backup download feature to a self-managed MySQL database?

    Execute the SHOW VARIABLES LIKE 'local_infile'; statement on the self-managed MySQL database. If OFF is returned in the output, execute the SET statement to set the global local_infile parameter to 1 to enable file import. After the preceding operations are complete, run the import script again.

  • After I download a backup file, why is the size of the backup file that is displayed in the PolarDB console inconsistent with the size of the downloaded backup file?

    The downloaded backup file is compressed. As a result, the size of the downloaded backup file is smaller than the file size that is displayed in the PolarDB console. You can check the data integrity after the data is restored from the backup file.

  • Can I directly restore the backup files that are downloaded from the PolarDB console to another PolarDB cluster?

    You cannot directly restore a backup file to another PolarDB cluster. If you want to restore the backup files to another PolarDB cluster, perform the following operations: Restore the backup file to a self-managed MySQL database and then migrate data from the self-managed database to the required PolarDB cluster. For more information, see Restore the data of a PolarDB cluster to a self-managed MySQL database by using snapshot backup files and Migrate data from a self-managed MySQL database to a PolarDB for MySQL cluster.

  • Can I cancel an ongoing backup download task?

    No, you cannot cancel a backup download task after it is started.

Download commands

This section describes the commands that are commonly used to download backup files.

Note
  • If the speed of a download is lower than 64 KB per second, the download may be interrupted. When you download a backup file, we recommend that you ensure optimal network status.

  • If you want to download a backup file to a disk that is attached by using ossfs, you must adjust the value of the multipart_size parameter for ossfs. The maximum value of this parameter is 100 GB. If the size of the backup file that you want to download exceeds 100 GB, the download fails. For more information about the ossfs plug-in and its parameter settings, see ossfs and Common options.

  • We recommend that you use the commands, such as wget and curl, that are provided in this topic to download a backup file. If you use a third-party tool to download a backup file, the backup file may be downloaded multiple times. As a result, the amount of data that is downloaded is greater than the size of the backup file, and you may be charged for the traffic that is consumed to download the excess amount of data over the Internet.

wget

Command: nohup wget -c -t 0 "The URL that is used to download the backup file" -O The save path and name of the backup file that you want to download > The file to which the downloaded data is saved &
Example: nohup wget -c -t 0 "https://example.aliyundoc.com/examplebackup.qp.xb" -O /backup/examplebackup.qp.xb > /tmp/download.log &
The following list describes the options in the preceding command:
-t 0: enables an unlimited number of retries. 
-c: enables resumable uploads. 
-O: specifies the save path and file name of the backup file after the backup file is downloaded. 
nohup: prevents interruptions to the download and specifies that the process automatically exits after the download is complete. If you accidentally replicate data or disconnect your database client during the download, the download is interrupted.

curl

Command: nohup curl -C - --retry 10 "The URL that is used to download the backup file" -o The user-defined name of the downloaded backup file> The file to which the downloaded data is saved&
Example: nohup curl -C --retry 10 "https://example.aliyundoc.com/examplebackup.qp.xb" -o backup.qp.xb > /tmp/download.log &
The following list describes the options in the preceding command:
--retry 10: enables up to 10 retries if the download fails. 
-C -: enables resumable uploads. 
-o: specifies the save path and file name of the backup file after the backup file is downloaded. 
nohup: prevents interruptions to the download and specifies that the process automatically exits after the download is complete. If you accidentally replicate data or disconnect your database client during the download, the download is interrupted.

Related API operations

APITitleDescription
DescribeDownloadBackupSetStorageInfoDescribeDownloadBackupSetStorageInfoQueries the storage information of a downloaded backup set.
DescribeDownloadSupportDescribeDownloadSupportQueries whether an instance supports the advanced download feature.
CreateDownloadCreateDownloadCreates an advanced download task.
DescribeDownloadTaskDescribeDownloadTaskQueries download tasks.