All Products
Search
Document Center

Cloud Backup:Back up a large amount of OSS objects by using the OSS inventory feature

Last Updated:Mar 07, 2024

You can use the inventory feature of Object Storage Service (OSS) to periodically generate inventory lists. Cloud Backup allows you to back up a large amount of OSS objects by using the OSS inventory feature, which significantly improves the backup efficiency and performance. Cloud Backup provides flexible backup policies to help you back up data to the cloud. You can view and restore data at any time. This topic describes how to use Cloud Backup to back up a large amount of objects in an OSS bucket.

Prerequisites

  • Cloud Backup is activated. You are not charged for activating Cloud Backup. If you use the OSS backup feature of Cloud Backup, you are charged for the storage usage of backup vaults and OSS requests. OSS request fees are included in your OSS bills. For more information, see Billing of the OSS backup feature.

  • An OSS bucket whose storage class is Standard or Infrequent Access (IA) is created. For more information, see Create buckets.

  • If you want to use an existing OSS inventory list to create an OSS backup plan, the OSS inventory list must meet the following requirements:

    • The inventory list of the latest version is enabled. The bucket prefix that is used in the inventory list is the same as the value of the Source Path parameter that is specified in the Create Backup Plan panel.

    • The inventory list must include the Object Size, Storage Class, Last Modified Date, and ETag fields.

    OSS清单

    For more information, see Configure bucket inventory.

Background information

You can use the inventory feature to export the information about specific objects in a bucket, such as the number, sizes, storage classes, and encryption status of objects. Cloud Backup allows you to back up a large amount of OSS objects by using the OSS inventory feature, which significantly improves the backup efficiency and performance. If you need to back up more than 100 million objects, we recommend that you use the OSS inventory feature. OSS determines the time at which OSS inventory lists are generated. If you perform a backup before an inventory list is generated, the backup fails. For more information, see Configure bucket inventory.

Usage notes

When you use the OSS inventory feature of Cloud Backup to back up OSS data, take note of the following information:

  • Cloud Backup allows you to back up and restore buckets whose storage class is Standard or IA. Cloud Backup does not support the backup and restoration of buckets whose storage class is Archive, Cold Archive, or Deep Cold Archive.

  • Cloud Backup allows you to back up and restore objects whose storage class is Standard or IA. Cloud Backup does not support the backup and restoration of objects whose storage class is Archive, Cold Archive, or Deep Cold Archive.

  • Cloud Backup does not support the backup and restoration of access control lists (ACLs) that control the read and write permissions on objects.

  • You are charged for OSS API operations that are called by Cloud Backup during the backup of OSS objects. OSS request fees are included in your OSS bills. For more information, see Billing overview.

  • When Cloud Backup reads OSS data from buckets or objects that are infrequently accessed, OSS data retrieval fees are incurred. You are charged based on the size of retrieved IA data. OSS data retrieval fees are included in your OSS bills. For more information, see Data processing fees.

  • If you delete an existing backup, the data integrity of subsequent backups is not affected.

  • You can manage OSS inventory lists in the OSS console. For more information about how to create an inventory for an OSS bucket, see Configure bucket inventory.

  • The scheduled backup plan that is enabled when you create an OSS bucket is free for 30 days. When the free trial expires, the scheduled backup plan is automatically disabled. We recommend that you switch to paid usage in a timely manner.

    Important

    A free-trial backup plan has limits. You cannot modify the source path and backup cycle. Each free-trial backup plan uses a separate backup vault. You cannot change the backup vault after you switch to paid usage. If you plan to back up all your data to the same backup vault in the future, do not use a free-trial backup plan.

  • To improve the performance of incremental backup, Cloud Backup uses the latest inventory lists that are generated within the last seven days.

    The system may require a short period of time to generate an OSS inventory list. When you use the OSS inventory list, take note of the following limits:

    • If no inventory list is detected by a running backup job, the backup job fails.

    • If a backup job detects that the OSS inventory list remains the same as the OSS inventory list for the previous backup, the backup job fails.

    • If a backup job is triggered, Cloud Backup uses only the latest OSS inventory list to back up data. After a backup job is completed, the objects that you add to the OSS inventory list are backed up in the next backup cycle.

    • The interval at which backup jobs are performed must be greater than or equal to the interval at which OSS inventory lists are generated. This way, you can use an OSS inventory list each time you run a backup job.

    Therefore, we recommend that you set the time to start the first backup to a point after an OSS inventory list is generated or click Run Now to manually perform a backup after an OSS inventory list is generated. This way, you can enable the inventory list in time to accelerate your backup jobs.

Create a backup plan to periodically back up a large amount of OSS objects

Note

After you create a backup plan, the first backup job performs a full backup and subsequent backup jobs perform incremental backups.

To create a backup plan for an OSS bucket, perform the following steps:

  1. Log on to the Cloud Backup console.

  2. In the left-side navigation pane, choose Backup > OSS Backup.

  3. In the top navigation bar, select the region where the OSS bucket resides.

  4. On the OSS Backup page, click Create Backup Plan.

  5. In the Create Backup Plan panel, configure the parameters and then click OK.

    Note
    • You can use each backup plan free of charge for one month, which starts from the day when the backup plan is created. For example, if you create a backup plan on February 14, 2020, you can use the backup plan free of charge until March 14, 2020. You are charged for OSS API operations that are called by Cloud Backup during the backup of OSS objects. You are also charged for the storage of OSS inventory lists. The charges are included in your OSS bills. For more information, see Billing overview.

    • You cannot modify the source path and backup cycle of a free-trial backup plan. Each free-trial backup plan uses a separate backup vault. You cannot change the backup vault after you change a free-trial backup plan to a paid backup plan. If you plan to back up all your data to the same backup vault in the future, do not use a free-trial backup plan.

    This topic describes how to configure a backup plan in paid mode. You can follow a similar procedure to configure a backup plan in free mode.

    1. Click Switch to Paid Plan.

    2. Configure the basic parameters described in the following table.

      Important
      • If the region that you select supports backup policies, you can set a backup plan only by associating it with a backup policy. Cloud Backup periodically backs up OSS objects based on the backup policy.

      • To view the regions that support backup policies, click Policy Center in the left-side navigation pane of the Cloud Backup console. For more information about how to create a backup policy, see Create a backup policy.

      Parameter

      Description

      Source OSS Bucket

      Select the name of the OSS bucket that you want to back up from the drop-down list. Cloud Backup automatically obtains all OSS buckets in the region within your account.

      Source Path

      Select a directory of the OSS bucket from the drop-down list. By default, all objects in the root directory of the OSS bucket are backed up.

      Backup Policy

      Select a proper backup policy from the drop-down list.

      Cloud Backup automatically backs up data sources based on the backup policy that you configure. Backup policies help you flexibly manage data sources. A backup policy includes the following settings: backup vault encryption method, backup interval, retention period, cross-region replication policy, and automatic archiving of backups.

      If the default backup policy does not meet your requirements, you can click Create Policy or Edit Policy to create or modify a backup policy.

      For more information about the parameters in a backup policy, see Manage backup policies.

      Performance Optimization

      Use OSS Inventory

      Specify whether to enable the OSS inventory feature to back up OSS objects. Set this parameter to Select Inventory.

      • No: disables the OSS inventory feature. You must back up objects in traversal mode.

      • Select Inventory: You can use an existing OSS inventory list to accelerate incremental backup of OSS objects.

      OSS Inventory Name

      This parameter is required only if you set the Use OSS Inventory parameter to Select Inventory. You can select a created OSS inventory list from the OSS Inventory Name drop-down list.

      The OSS inventory list that you create must meet the following requirements:

      • The inventory list of the latest version is enabled. The bucket prefix that is used in the inventory list is the same as the value of the Source Path parameter that is specified in the Create Backup Plan panel.

      • The inventory list must include the Object Size, Storage Class, Last Modified Date, and ETag fields.

      • The interval at which OSS inventory lists are generated must be less than or equal to the interval at which backup jobs are performed. This way, you can use an OSS inventory list each time you run a backup job.

      Delete inventory file after backup

      Specify whether to delete the inventory list when the backup plan is completed. If you set this parameter, you may delete the inventory lists of the bucket. We recommend that you store the inventory lists in a dedicated bucket or set the parameter to Do Not Delete to manually manage the lists.

      • Do Not Delete: retains the inventory lists. If other applications access the OSS inventory lists, we recommend that you set the parameter to this value.

      • Delete Current: deletes the inventory list that is used in the backup job.

      • Delete All: deletes all the inventory lists that are generated before the backup after the backup is completed. If no other applications access the OSS inventory lists, we recommend that you set the parameter to this value.

    After the backup plan is created, the backup plan is in the Running state. Cloud Backup backs up OSS objects from the specified start time at the specified backup interval.

Backup is successful

When the backup execution time is reached, Cloud Backup runs the backup job. If the Status of the backup job is Successful, the backup job is completed on the current day.image.png

You can view each backup record in the Backup History section. Later, you can restore files based on these historical backups. For more information, see Create a restore job for an OSS bucket.image.png

Troubleshooting

If the Status of the backup plan or backup job is abnormal, you can resolve the issue based on the following instructions.

  • Status of the backup job: Failed (The OSS bucket does not exist.)

    The OSS bucket is deleted.

    • If the OSS bucket is deleted as expected, ignore the backup job status. To prevent additional backup storage fees, we recommend that you perform the following operation to delete the related storage vault: Click Storage Vaults in the left-side navigation pane, and find and delete the storage vault.

    • If the OSS bucket is deleted unexpectedly, perform the following operation to restore the OSS bucket: Create an OSS bucket in the corresponding region and use the OSS bucket as the restoration destination to restore OSS objects. For more information, see Create a restore job for an OSS bucket.image.png

  • Status of the backup job: Failed (No OSS inventory list is available. Please wait for the next backup when the latest inventory list is automatically detected and used.)

    The OSS inventory list is not found in the specified directory. Go to the OSS console to check whether the bucket inventory has been created.

  • Status of the backup plan: Suspended

    The backup plan has been suspended. After the backup plan is suspended, Cloud Backup no longer backs up objects from the OSS bucket. We recommend that you resume the backup plan at your earliest opportunity.

Related operations

The following table describes the operations that you can perform in the Actions column on the Backup Plans tab.

Operation

Description

Backup History

Views all the backup records of the OSS bucket or the backup records in the last three months.

Execute Immediately

Immediately runs a backup plan.

Edit

Modifies the parameters of an OSS backup plan that does not meet your business requirements.

Suspend Plan

Suspends a running backup plan.

Resume Plan

Resumes a suspended backup plan.

Delete Plan

Deletes a running backup plan. After you delete the backup plan, Cloud Backup no longer runs backup jobs for the backup plan but retains the backup data.

Alert Settings

If a backup job fails or the client is disconnected from the server, Cloud Backup sends alert notifications based on the alert policy that you configure. For more information, see Configure alert notifications.

What to do next

Create a restore job for an OSS bucket

References