This topic describes how to restore archived data from a data source and migrate the data to Object Storage Service (OSS).

Alibaba Cloud OSS

OSS provides a tool named ossutil to restore archived files. This tool allows you to specify a prefix to restore the objects whose names start with the prefix. The following example shows you how to use the ossutil tool to restore archived files. The operating system used in the example is Linux.

  1. Download and install the ossutil tool based on your operating system. Configure execution permissions. For more information, see Install ossutil.
  2. In Linux, run the ./ossutil restore oss://bucket/path -r ./config.xml command to restore the archived objects whose names start with the specified prefix.
    • -r specifies the prefix. All archived objects whose names start with the prefix in the bucket are restored.
    • config.xml is a local XML file and contains parameters that are used to restore archived objects. The following sample code provides an example:
      <RestoreRequest>
          <Days>3</Days>
          <JobParameters>
              <Tier>Bulk</Tier>
          </JobParameters>
      </RestoreRequest>

      The following table describes the parameters.

      ParameterDescription
      Days Specifies the number of days that the restored objects remain in the restored state. The valid values range from 1 to 7.
      Tier Specifies the restore mode for the archived objects. Valid values: Expedited, Standard, and Bulk.
      • Expedited: The object is restored within 1 hour.
      • Standard: The object is restored in 2 to 5 hours.
      • Bulk: The object is restored in 5 to 12 hours.
      Important You are charged for data restoration. For more information, see Data processing fees.

    For more information about the restored objects, see restore.

  3. After the data is restored, migrate the data to OSS. For more information, see Migrate data between Alibaba Cloud OSS buckets.

AWS S3

Amazon Simple Storage Service (S3) allows you to use the S3cmd tool or the Batch Operations feature to restore archived data.
  • The S3cmd tool can restore approximately 100 files per second. It is suitable for scenarios in which fewer than 100,000 files need to be restored.
  • If you want to restore more than 100,000 files, we recommend that you use the Batch Operations feature. This feature allows you to restore hundreds of, tens of millions of, or even billions of files in a simplified manner. To use this feature, you must configure it in the AWS console.

To use the S3cmd tool to restore files, perform the following steps:

  1. Run the pip install s3cmd command to install the S3cmd tool.
  2. Configure the AccessKey ID and AccessKey secret by using the CLI. Run the s3cmd -configure command to configure the access key pair.
  3. Run the s3cmd restore -D 10 -priority='standard' s3://bucket/object command to restore the files in the specified bucket.
    • -D specifies the number of days that the restored files remain in the restored state. If you leave this parameter empty, the default value 1 is used. In this example, -D is set to 10.
    • -priority specifies the restore priority. Valid values: bulk, standard, and expedited. In this example, -priority is set to standard.

    The fees that you are charged to restore the archived files vary based on the restore priority. For more information, see the billing topics in the AWS S3 documentation.

  4. After the data is restored, migrate the data to OSS. For more information, see Migrate data from Amazon S3 to OSS.
To use the Batch Operations feature to restore files, perform the following steps:
Important To use the Batch Operations feature to restore objects, create at least one AWS inventory list for the bucket in which the objects to be restored reside and make sure that at least one report is generated.
  1. Create a role and grant the role full permissions on the bucket.
    1. Log on to AWS console and go to the Identity and Access Management (IAM) console. In the left-side navigation pane, click Roles. Then, click Create role.
    2. In the Trust step, select AWS service > S3 in the Select role type section, select S3 Batch Operations in the Select your use case section, and then click Next: Permissions.
    3. In the Permissions step, search for the permission policies related to S3 by using the keyword S3. In the search result, select AmazonS3FullAccess. Then, click Next: Tags.
    4. You can add tags as needed and then click Next: Review.
    5. In the Review step, specify the role name and role description and click Create role.
  2. Use an inventory item to create a job to restore the archived files.
    1. On the bucket details page, click Management. On the Management tab, click Inventory.
    2. On the Inventory subtab, select the inventory item that you want to use to create a job and click Create job from manifest.
      Important If no report is generated after you configure the inventory item, the Batch Operations feature is unavailable.
    3. Set parameters in the Choose Region and the Choose manifest sections and click Next.
    4. Select Restore as Operation. Enter the number of days that the restored objects remain in the restored state. Select Bulk retrieval or Standard retrieval. We recommend that you select Standard retrieval because the only difference is that a Standard retrieval consumes less time to restore the same amount of data than a Bulk retrieval. Then, click Next.
      Specify the number of days that the restored objects remain in the restored state based on the amount of data that you want to migrate. This prevents restored data from becoming unavailable during the migration.
    5. Complete the remaining settings and click Next.
      Retain the default settings for the description and priority, specify a path to save the report generated after the migration is complete, select the IAM role that you created, and then add tags or leave tags empty.
    6. After you confirm the information, click Create job.
    7. You are directed to the page that shows the list of batch operation jobs. If the job is in the Awaiting your confirmation state, select the job and click Confirm and run.
      The Complete state does not indicate that all the files are restored. A Standard retrieval takes 5 to 12 hours. You can estimate the time when the files are restored. You can also view the details of each file to check whether the file is restored. For more information, see the topics relevant to batch restore in the AWS S3 documentation.
  3. After the data is restored, migrate the data to OSS. For more information, see Migrate data from Amazon S3 to OSS.

Azure Blob

Azure Blob allows you to modify blob properties to change access tiers in the console to restore archived files. Batch restore is supported. To restore an archive blob, perform the following steps:

  1. Find the blob that you want to restore and click the More icon that corresponds to the blob. Click Change tire. Then, set the Access tire parameter to Cool. The restoration may take several hours. For more information, see the topics relevant to restoring data in the Azure Blob documentation.
    The restore priority can be set to High or Standard. You can use the x-ms-rehydrate-priority property to specify the restore priority when you configure the blob access tier.
    • Standard: The restore request is processed in the order the request is received and can take up to 15 hours.
    • High: High requests have a higher priority than Standard requests. Requests for objects that are smaller than 10 GB are completed within 1 hour.
    Note If you do not want to restore the archive blob, you can copy the blob instead. Your original blob remains in the archived state when a blob is created in the Hot access tier. You can migrate your data from the created blob to OSS.

    After a restore request is initiated, you cannot cancel the request. During the restoration process, the x-ms-access-tier blob property continues to show as archive until the restoration is complete. To confirm the status and progress of the restoration process, call the Get Blob Properties operation to check the x-ms-archive-status and the x-ms-rehydrate-priority blob properties.

  2. After the data is restored, migrate the data to OSS. For more information, see Migrate data from Azure Blob to OSS.

GCP

Google Cloud Platform (GCP) provides a tool named gsutil to restore archived data. This tool allows you to modify the storage class of a specified file to restore the file. This tool does not support batch restore. To use the gsutil tool to restore an archived file, perform the following steps:

  1. Download and install the gsutil tool. Then, verify your identity.
  2. Run the gsutil rewrite -s STORAGE_CLASS gs://PATH_TO_OBJECT command to restore the specified object. For more information, see the topics relevant to restoring data in the GCP documentation.
    • STORAGE_CLASS specifies the storage class to which you want to restore the object to, such as nearline.
    • PATH_TO_OBJECT specifies the name of the object that you want to restore, such as pets/dog.png.
  3. After the data is restored, migrate the data to OSS. For more information, see Migrate data from GCP to OSS.
90

Tencent Cloud COS

Tencent Cloud Object Storage (COS) provides a tool named coscmd to restore archived files. This tool supports batch restore. To use the coscmd tool to restore files, perform the following steps:

  1. Run the pip install coscmd command to install the coscmd tool.
  2. Run the coscmd config -a ak -s sk -b bucket -r region command in the CLI to configure the AccessKey pair, bucket, and region.
  3. Run the coscmd restore -r -d 10 -t Expedited examplefolder/ command to restore the files in the specified bucket.
    • -d specifies the number of days that the restored files remain in the restored state. If you leave this parameter empty, the default value 7 is used. In this example, -d is set to 10.
    • -t specifies the restore priority. Valid values: Expedited, Standard, and Bulk. The default value is Standard. In this example, -t is set to Expedited.

    The fees that you are charged to restore the archived files vary based on the restore priority. For more information, see the topics relevant to restoring data in the COS documentation.

  4. After the data is restored, migrate the data to OSS. For more information, see Migrate data from COS to OSS.

Huawei Cloud OBS

HUAWEI CLOUD Object Storage Service (OBS) provides a tool named obsutil to restore archived files. This tool supports batch restore. To use the obsutil tool to restore files, perform the following steps:

  1. Download the obsutil tool.
  2. Decompress the installation file of the obsutil tool and run this tool.
    • For Windows, decompress the installation file of the obsutil tool and double-click obsutil.exe.
    • For Linux or macOS, run the tar -xzvf command to decompress the installation file of the obsutil tool, go to the directory to which the file is decompressed, and then run the chmod 755 obsutil command to grant the tool the execution permissions.
  3. Run the corresponding command to restore the files in the specified bucket. If you specify a bucket name in the command, all the archived files in the bucket are restored. For more information, see the topics relevant to restoring data in the HUAWEI CLOUD OBS documentation.

    For Windows, run the obsutil restore obs://bucket-test/key -r -f -i ak -k sk -e endpoint command. For Linux or macOS, run the ./obsutil restore obs://bucket-test/key -r -f -i ak -k sk -e endpoint command.

    • -r: specifies the prefix in the names of the archived objects that you want to restore at the same time.
    • -f: specifies that objects are force restored with no confirmation message prompted. This parameter is optional.
    • -v: specifies the prefix in the names of the archived objects that you want to restore at the same time. The versioning feature is implemented on these archived objects. This parameter is optional.
    • -j: specifies the maximum number of archived objects that are concurrently restored. This parameter is optional.
    • -d: specifies the number of days that the restored files remain in the restored state. This parameter is required. If you leave this parameter empty, the default value 1 is used. The valid values range from 0 to 30. In this example, -d is set to 10.
    • -t: specifies the restore priority. Valid values: bulk and standard. If the specified value is expedited, objects are restored in 1 to 5 minutes. If the specified value is standard, objects are restored in 3 to 5 hours. The default value is expedited. In this example, the default value is used.
  4. After the data is restored, migrate the data to OSS. For more information, see Migrate data from HUAWEI CLOUD OBS to OSS.

Baidu Cloud BOS

Baidu Object Storage (BOS) provides the RestoreObject API operation that you can call to restore a specified archived file. This API operation cannot be used to restore multiple files at the same time. To restore a specified file, perform the following steps:

  1. Call the RestoreObject API operation to restore the specified file. For more information, see the topic that describes the RestoreObject API operation in the BOS documentation.

    Sample requests

    The x-bce-restore-days parameter specifies the number of days that the restored file remains in the restored state. The default value is 7. The valid values range from 0 to 30. The x-bce-restore-tier parameter specifies the restore priority. The valid values are Expedited and Standard.
    POST /ObjectName?restore HTTP/1.1
    Host: <BucketName>.bj.bcebos.com
    Date: <Date>
    Content-Length: 0
    Authorization: <Authorization_String>
    x-bce-restore-tier:Expedited/Standard
    x-bce-restore-days:3
    Sample responses
    HTTP/1.1 200 OK
    x-bce-request-id: 4db4b34d-653d-4d9a-b49b-3049ca786409
    Date: Wed, 31 May 2018 08:34:40 GMT
    Server: BceBos
  2. After the data is restored, migrate the data to OSS. For more information, see Migrate data from BOS to OSS.

Kingsoft Cloud KS3

Kingsoft Standard Storage Service (KS3) provides the RestoreObject API operation that you can call to restore a specified file. The API operation cannot be used to restore multiple files at the same time. To restore a specified file, perform the following steps:

  1. Call the RestoreObject API operation to restore the specified file.
    Sample requests
    POST /{ObjectKey}?restore HTTP/1.1
    Host: {BucketName}.{endpoint}
    Date: {date}
    Authorization: {SignatureValue}
    Note
    • By default, a restored object remains in the restored state for 24 hours. Before the object is re-archived, you can call RestoreObject again to prolong the time during which the object remains in the restore state by 24 hours. For the same object, you can call RestoreObject up to seven times. This way, a restored object can remain in the restored state for up to seven days.
    • The first time you call RestoreObject to restore an archived object, the status code 202 is returned. If you call RestoreObject to restore the object again when the object is being restored, the status code 409 and the error message RestoreAlreadyInProgress are returned. The error message indicates that a restore request is in progress. You need to wait for up to 10 minutes until the object is restored. If you call the API operation to restore the object again when the object is in the restored state, status code 200 is returned and the object can remain in the restored state for another 24 hours.
    • You are charged when you restore an archived file. Each time you call RestoreObject, you are charged for requests that are made to complete the restore. For more billing information, see the topics relevant to restoring data in the KS3 documentation.
    Sample responses
    HTTP/1.1 200 OK
    Server: Tengine
    Date: Fri, 26 Dec 2018 06:40:02 GMT
    Content-Length: 0
    Connection: keep-alive
    X-Application-Context: application
    x-kss-request-id: 1d147c34a9d542aa8dc7f0f70d3db893
    x-kss-storage-class: ARCHIVE
  2. After the data is restored, migrate the data to OSS. For more information, see Migrate data from KS3 to OSS.

Qiniu Cloud KODO

Qiniu Cloud provides an API operation that you can call to restore archived files at the same time. You can perform the following operations:

  1. Call the API operation to restore archived files at the same time. For more information, see the topics relevant to restoring data in the Qiniu Cloud-Object Storage documentation.

    Sample requests

    EncodedEntryURI specifies the Base64-encoded address in the <Bucket>:<Key> format. FreezeAfterDays specifies the number of days that the restored files remain in the restored state. The valid values range from 1 to 7.
    POST /batch HTTP/1.1
    Host:           rs.qbox.me
    Content-Type:   application/x-www-form-urlencoded
    Authorization:  Qiniu <AccessToken>
    
    op=/restoreAr/<EncodedEntryURI>/freezeAfterDays/<FreezeAfterDays>&op=/restoreAr/<EncodedEntryURI>/freezeAfterDays/<FreezeAfterDays>&...
    Sample responses
    HTTP/1.1 200 OK
    Content-Type: application/json
    Cache-Control: no-store
  2. After the data is restored, migrate the data to OSS. For more information, see Migrate data from Qiniu Cloud-Object Storage to OSS.

Ucloud US3

Ucloud US3 provides a tool named us3cli to restore files. This tool supports batch restore. To use the us3cli tool to restore files, perform the following steps:

  1. Download and install the us3cli tool and grant the tool execution permissions.
  2. Run the ./us3cli config config1 --accesskey ak --secretkey --endpoint endpoint command in the CLI to configure the AccessKey pair and the endpoint.
  3. Run the ./us3cli restore us3://bucket/dir -r --parallel 30 command to restore files in the specified bucket and specify the number of files to be concurrently restored. For more information, see the restore command description in the "Frequently used commands" topic in the US3 documentation.
    • -r specifies the recursive folder. All the files in this folder, including the files in the subfolders, are to be restored.
    • --parallel specifies the number of objects that are concurrently restored. The default value is 10. In this example, --parallel is set to 30.
  4. After the data is restored, migrate the data to OSS. For more information, see Migrate data from Ucloud US3 to OSS.