Download cluster backup files - PolarDB - Alibaba Cloud Documentation Center

PolarDB allows you to download cluster backup files to your local system for long-term backup storage, historical data queries, backup transfers (such as to OSS), and audit requirements.

Note

The downloaded backup data cannot be directly restored to a PolarDB for MySQL cluster. You can first restore them to a self-managed MySQL database.

Release date

November 24, 2023

Note

The backup file download feature is in canary release and will be officially launched soon.

Billing

After a backup file is created, the system temporarily stores it in Database Backup Service (DBS)'s built-in storage. A temporary download URL is generated, which points to this file. The backup file download feature allows you to download the backup file by using the URL. You can also transfer the backup file to your Object Storage Service (OSS). The billing method varies based on the download method.

Note

If a download task fails, no fees are charged.
To check the calculated backup data volume and outbound internet traffic for the month, go to the PolarDB console and navigate to the cluster's Settings and Management > Backup and Restoration > Backup Download List page.

Storage fees

URL: No storage fees are charged.
OSS: The backup files are transferred to your OSS. For information about the OSS storage fees, see Storage fees.

File conversion fees

Whether you download backup files from a URL or transfer backup files to OSS, conversion fees are charged when the backup files are converted to the SQL, CSV, Parquet, or CSV-with-header format. There is no free quota for the format conversion. The pricing rate is $0.03125/GB.

Internet traffic fees

URL:
- Download over an internal network: You are not charged for traffic that you consume.
- Download over the Internet: There is a free quota of 500 GB per month per cluster. Any data transfer exceeding the free quota is billed based on the volume of data downloaded. The charges are calculated daily. For information about the specific pricing, see Network traffic fees.
Note
We recommend that you purchase a subscription network plan to offset the fees generated for downloading backups over the internet. The larger the network plan capacity, the higher the discount.
OSS: For information about the traffic fees for transferring the backup files to OSS, see Traffic fees.

Prerequisites

Cluster: Only Enterprise Edition clusters of the Cluster edition support the backup file download feature.
Region: China (Chengdu), China (Guangzhou), China (Qingdao), China (Beijing), China (Shanghai), China (Zhangjiakou), China (Hangzhou), China (Shenzhen), China (Hong Kong), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo), Singapore, US (Silicon Valley), US (Virginia), and Germany (Frankfurt).
Note
The feature will be available in other regions soon.
Others:
- The RAM user that you use has permissions to download backup files. To grant permissions to a RAM user, see RAM user permissions.
- Backup data is not encrypted. Encrypted PolarDB clusters do not support downloading backup files.

Limits

The backup download feature allows you to export most schema elements. However, some schema elements cannot be exported:
- Supported for export: column information, primary key index, non-primary key indexes, unique index, partitioned table information, table engine, and table-level or database-level character sets and collations.
- Not supported for export: expression indexes, foreign keys, generated columns, hidden columns, views, functions, stored procedures, system variables, and triggers.
The backup download feature does not support fields of spatial data types. If your PolarDB cluster contains fields of the following spatial data types, backup file conversions fail:
GEOMETRY, POINT, LINESTRING, POLYGON, MULTIPOINT, MULTILINESTRING, MULTIPOLYGON, GEOMETRYCOLLECTION
The files to export do not include the following system databases:
information_schema, mysql, performance_schema, sys, __recycle_bin__
If you select the OSS download method, make sure that the OSS uses the Standard storage class. For information about how to convert the storage class, see Convert storage classes.
Note
You must manually create a destination OSS bucket. If you have already created a bucket for storing backup files, ignore this step.
The backup file download feature supports downloading only level-1 backup files.
If cold data archiving is enabled for the cluster, you cannot download archived cold data.
If the names of the databases or tables in the downloaded backup files contain slashes (/), the task fails.

Procedure

(Recommended) PolarDB console

Log on to the PolarDB console, click Clusters in the navigation pane on the left, select the Region where the cluster is deployed, and then click the cluster ID to open the cluster details page.
In the left-side navigation pane, click Settings and Management > Backup and Restoration.
Click the Data Backups tab. Find the backup set that you want to download and then click Download Cluster Backup in the Actions column.
Note
- By default, the PolarDB console displays the backup sets that are generated over the most recent eight days. If you want to view the backup sets that are generated eight days ago, you must change the default time range.
- If Download Cluster Backup is not available, check whether the database edition, edition, and region of your cluster meets the prerequisites.
In the Point in Time and Backup Set step of the wizard, select Download By Point in Time or Download By Backup Set, and click Next in the lower-left corner of the page.
In the Instance, Database, and Table step of the wizard, click Next in the lower-left corner of the page. By default, Instance Download is selected.
In the Destination and Format step of the wizard, select the download destination and format.
Important
- We recommend that you set the download destination to OSS, which is more time-efficient.
- The backup download task cannot be canceled after it is started.
- Fees are charged for backup file download tasks. For information about the fees, see Billing.
(Recommended) OSS
When the download destination is OSS, data is written directly to your OSS. You can delete it as needed after use.
1. Enter your OSS Bucket Name and directory prefix (such as xx/xx).
  Note
  You need to manually create an OSS bucket. If you have already created a bucket for storing backup files, you can enter it directly.
  The OSS storage class must be Standard. To convert the storage class, see Storage class conversion.
2. Select the download format. Currently, you can set the download format to CSV, SQL, Parquet, or CSV-with-header. You can choose based on your business requirements.
3. Read and check the relevant information, and click Complete. The page automatically jumps to the Backup Download List tab, where you wait for the task to complete.
  Note
  The backup file download feature requires authorization for the DBS service to access your cloud resources. If you have not previously authorized access permissions, follow the console prompts to click Authorize > Confirm Authorization. After the permissions are granted, configure the parameters for the advanced download task.
  Because the computation conversion requires competing for computing resources on the service side, temporary resource allocation failures may occur, leading to download failures. Additionally, some unsupported special data formats may also cause download failures. If your download task fails, try again or contact DBS technical support.
  No fees are generated for failed tasks.
4. View backup files: If the status of the download task is Successful, the download is complete, and you can view the files in the corresponding OSS bucket.
URL
When the download destination is URL, the system temporarily stores the converted data in DBS built-in storage. No storage fees are charged for this process.
Note
URL downloads involve data packaging, which requires additional waiting time. The duration of this packaging process is largely dependent on the logical size of the backup set. If the logical backup size is large (such as over 1 TB), we recommend downloading to OSS.
1. Select the download format. Currently, you can set the download format to CSV, SQL, Parquet, or CSV-with-header. You can choose based on your business requirements.
2. Read and check the relevant information, and click Complete. The page automatically jumps to the Backup Download List tab, where you wait for the task to complete.
  Note
  The backup file download feature requires authorization for the DBS service to access your cloud resources. If you have not previously authorized access permissions, follow the console prompts to click Authorize > Confirm Authorization. After the permissions are granted, configure the parameters of the backup download task.
  Because the computation conversion requires competing for computing resources on the service side, temporary resource allocation failures may occur, leading to download failures. Additionally, some unsupported special data formats may also cause download failures. If your download task fails, try again or contact DBS technical support.
  No fees are generated for failed tasks.
3. View backup files: If the status of the download task is Successful, the download is complete, and you can click Generate Link in the Download Destination column, set the link Validity Period, and click Generate Link to download the backup data through the generated internal or external network link.
  Note
  You can obtain the download URL within three days after the download task is completed. The validity period of the URL can be set from 5 minutes to 1 day, with a default validity period of 2 hours.
  Completed tasks expire after three days (the download link will expire), and the corresponding data will be cleaned up after the task expires (cleanup is automatically executed some time after the task expires). If you do not retain the data within the specified period of time but want to use the data, you must initiate a new download task and obtain a new download URL.
  We recommend that you save the URL at the earliest opportunity and keep the URL confidential.
  If you use a third-party download tool to download backup files, additional traffic may be generated and you may be charged additional fees. Proceed with caution. For specific download methods, see Download commands.

API

DescribeDownloadSupport: checks whether your cluster supports the backup file download feature.
Note
If the API returns that the feature is not supported, check whether your cluster's database edition, edition, and region meet the prerequisites.
CreateDownload: creates a backup file download task.
Note
- To download backup files, you must have the AliyunDBSDefaultRole permission, which permits the DBS service to access your cloud resources.
- If you use the OSS download method, first manually create the corresponding OSS bucket and make sure that the OSS use the Standard storage class.
- If you use the URL download method, save the returned parameter TaskId (task ID) for obtaining the download link.
- The backup download task cannot be canceled after it is started.
- Fees are charged for backup file download tasks. For more information about the fees, see Billing.
(Optional) DescribeDownloadTask: views the backup file download task information of the cluster, including the download task status.
Obtain backup files:
- If you use the OSS download method, the system writes the backup data directly to your OSS, and you can view the data in the corresponding bucket.
- If you use the URL download method, you can use DescribeDownloadBackupSetStorageInfo to obtain the download link.
  Note
  - You can obtain the download URL within three days after the download task is completed. The validity period of the URL can be set from 5 minutes to 1 day, with a default validity period of 2 hours.
  - Completed tasks expire after three days (the download link will expire), and the corresponding data will be cleaned up after the task expires (cleanup is automatically executed some time after the task expires). If you do not retain the data within the specified period of time but want to use the data, you must initiate a new download task and obtain a new download URL.
  - We recommend that you save the URL at the earliest opportunity and keep the URL confidential.
  - If you use a third-party download tool to download backup files, additional traffic may be generated and you may be charged additional fees. Proceed with caution. For specific download methods, see Download commands.

FAQ

Can I cancel an ongoing backup download task?

No, you cannot cancel a backup download task after it is started.

How do I use the downloaded data backup and log backup files?

You can restore the downloaded backup files to a self-managed MySQL database.

What do I do if I encounter the error ERROR 1148 (42000): The used command is not allowed with this MySQL version when restoring the downloaded backup files to a local MySQL database?

Run the show variables like 'local_infile'; query script on the MySQL database. If the result is OFF, run the set global local_infile = 1;statement to enable file import. After the preceding operations are complete, run the import script again.

Why is the backup size displayed in the PolarDB console different from the actual downloaded backup file size?

The downloaded backup files are compressed and generally smaller than the backup size displayed in the PolarDB console. You can restore the files to check if the data is complete.

Can I directly restore a downloaded backup file to another PolarDB cluster?

You cannot directly restore the downloaded backup files to another PolarDB clusters. You can first restore the backup file to a self-managed MySQL database, and then use DTS to migrate the data from the self-managed MySQL to the PolarDB for MySQL cluster.

Appendix

RAM user permissions

The backup file download feature relies on RAM for permission control. You can control backup file downloads by granting or restricting backup file download permissions for RAM users.

Grant permissions to download backup files

If you cannot use the backup file download feature (such as creating or querying download tasks), check whether the data disaster recovery management permissions (AliyunDBSFullAccess) are granted to your RAM user. For information about how to grant permissions, see Grant permissions to a RAM user.

Prevent a RAM user from obtaining backup file download URLs

If you want the RAM user to use the backup and restoration feature but do not want the RAM user to obtain download URLs, you can create a create a custom policy to prevent the RAM user from calling the API to obtain download URLs. Sample script:

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Deny",
            "Action": "dbs:DescribeDownloadBackupSetStorageInfo",
            "Resource": "*"
        }
    ]
}

After you create the custom policy, grant it to your RAM user. Then, the RAM user cannot obtain download URLs.

Allow read-only RAM users to download backup files

To ensure security, a RAM user that has read-only permissions to PolarDB databases (AliyunPolardbReadOnlyAccess) is not allowed to download backup files.

If the RAM user needs to download backup files, you can grant the read-only access to data disaster recovery permissions (AliyunDBSReadOnlyAccess) to the user. Then, the RAM user can view the download links for existing backup download tasks. For information about how to grant permissions, see Grant permissions to a RAM user.

Download commands

Note

If the download speed is slower than 64 KB per second, the download may be interrupted. We recommend that you ensure optimal network status when you download a backup file.
If you download backup files to an OSSFS-mounted disk, adjust the multipart_size parameter of OSSFS. By default, this parameter supports files up to 100 GB in size. If the backup file exceeds 100 GB, the download fails. For more information, see OSSFS and Options supported by ossfs.
We recommend that you use the wget and curl commands provided in this topic to download backup files. If you use a third-party tool to download a backup file, the backup file may be downloaded multiple times. As a result, the amount of data that is downloaded is greater than the size of the backup file, and you may be charged for the traffic consumed to download the excess amount of data over the Internet.

wget

nohup wget -c -t 0 "backup file download URL" -O destination file path and name > redirect log information to a specific log file &

Parameters

Parameter	Description
nohup	Ensures that the download continues even if the terminal is accidentally closed or the session disconnects. After the download is completed, the process automatically exits.
-c	Enables resumable downloads.
-t	The number of retry attempts. Setting it to 0 means unlimited retries.
-O	Specifies the destination file path and name.

Example

nohup wget -c -t 0 "http://dbs-xxx.aliyuncs.com/xxx.tar.gz?xxx" -O /backup/examplebackup.tar.gz > /tmp/download.log &

curl

nohup curl -C - --retry 10 "backup file download URL" -o custom file name > redirect log information to a specific log file &

Parameters

Parameter	Description
nohup	Ensures that the download continues even if the terminal is accidentally closed or the session disconnects. After the download is completed, the process automatically exits.
-C -	Enables resumable downloads.
--retry	The number of retry attempts.
-o	Specifies the destination file path and name.

Example

nohup curl -C - --retry 10 "http://dbs-xxx.aliyuncs.com/xxx.tar.gz?xxx" -o /backup/examplebackup.tar.gz > /tmp/download.log &

Release date

Billing

Storage fees

File conversion fees

Internet traffic fees

Prerequisites

Limits

Procedure

(Recommended) PolarDB console

(Recommended) OSS

URL

API

FAQ

Appendix

RAM user permissions

Grant permissions to download backup files

Prevent a RAM user from obtaining backup file download URLs

Allow read-only RAM users to download backup files

Download commands

wget

curl