:Solutions to migrate archived data

Alibaba Cloud OSS

OSS provides a tool named ossutil to restore archived files. This tool allows you to specify a prefix to restore the objects whose names start with the prefix. The following example shows you how to use the ossutil tool to restore archived files. The operating system used in the example is Linux.

1. Download and install the ossutil tool based on your OS. Configure execution permissions. For more information, see Install ossutil.
2. In Linux, run the ./ossutil restore oss://bucket/path -r ./config.xml command to restore the archived objects whose names start with the specified prefix.
   • -r specifies the prefix. All archived objects whose names start with the prefix in the bucket are restored.
   • config.xml is a local XML file and contains parameters that are used to restore archived objects. The following code provides an example:

     <RestoreRequest>
         <Days>3</Days>
         <JobParameters>
             <Tier>Bulk</Tier>
         </JobParameters>
     </RestoreRequest>

     The following table describes the parameters.

     Parameter	Description
     Days	Specifies the number of days that the restored objects remain in the restored state. The valid values range from 1 to 7.
     Tier	Specifies the restore mode for the archived objects. Valid values: Expedited, Standard, and Bulk. Expedited: The object is restored within 1 hour. Standard: The object is restored in 2 to 5 hours. Bulk: The object is restored in 5 to 12 hours. Important You are charged for data restoration. For more information, see Data processing fees.

   For more information about the restored objects, see restore.

3. After the data is restored, migrate the data to OSS. For more information, see Preparations.

AWS S3

Restore files by using the S3cmd tool

1. Run the pip install s3cmd command to install the S3cmd tool.
2. Configure the AccessKey ID and AccessKey secret by using the CLI. Run the s3cmd -configure command to configure the access key pair.
3. Run the s3cmd restore -D 10 -priority='standard' s3://bucket/object command to restore the files in the specified bucket.
   • -D specifies the number of days that the restored files remain in the restored state. If you leave this parameter empty, the default value 1 is used. In this example, -D is set to 10.
   • -priority specifies the restore priority. Valid values: bulk, standard, and expedited. In this example, -priority is set to standard.

   The fees that you are charged to restore the archived files vary based on the restore priority. For more information, see the billing topics in the AWS S3 documentation.

4. After the data is restored, migrate the data to OSS. For more information, see Prepare for a migration task.

Use the Batch Operations feature

1. Create a role and grant the role full permissions on the bucket.
   1. Log on to AWS console and go to the Identity and Access Management (IAM) console. In the left-side navigation pane, click Roles. Then, click Create role.
   2. In the Trust step, select AWS service > S3 in the Select role type section, select S3 Batch Operations in the Select your use case section, and then click Next: Permissions.
   3. In the Permissions step, search for the permission policies related to S3 by using the keyword S3. In the search result, select AmazonS3FullAccess. Then, click Next: Tags.
   4. You can add tags as needed and then click Next: Review.
   5. In the Review step, specify the role name and role description and click Create role.
2. Use an inventory item to create a job to restore the archived files.
   1. On the bucket details page, click Management. On the Management tab, click Inventory.
   2. On the Inventory subtab, select the inventory item that you want to use to create a job and click Create job from manifest.
      Important If no report is generated after you configure the inventory item, the Batch Operations feature is unavailable.
   3. Set parameters in the Choose Region and the Choose manifest sections and click Next.
   4. Select Restore as Operation. Enter the number of days that the restored objects remain in the restored state. Select Bulk retrieval or Standard retrieval. We recommend that you select Standard retrieval because the only difference is that a Standard retrieval consumes less time to restore the same amount of data than a Bulk retrieval. Then, click Next.
      Specify the number of days that the restored objects remain in the restored state based on the amount of data that you want to migrate. This prevents restored data from becoming unavailable during the migration.
   5. Complete the remaining settings and click Next.
      Retain the default settings for the description and priority, specify a path to save the report generated after the migration is complete, select the IAM role that you created, and then add tags or leave tags empty.
   6. After you confirm the information, click Create job.
   7. You are directed to the page that shows the list of batch operation jobs. If the job is in the Awaiting your confirmation state, select the job and click Confirm and run.
      The Complete state does not indicate that all the files are restored. A Standard retrieval takes 5 to 12 hours. You can estimate the time when the files are restored. You can also view the details of each file to check whether the file is restored. For more information, see the topics relevant to batch restore in the AWS S3 documentation.
3. After the data is restored, migrate the data to OSS. For more information, see Prepare for a migration task.

Tencent Cloud COS

Tencent Cloud Object Storage (COS) provides a tool named coscmd to restore archived files. This tool supports batch restore. To use the coscmd tool to restore files, perform the following steps:

1. Run the pip install coscmd command to install the coscmd tool.
2. Run the coscmd config -a ak -s sk -b bucket -r region command in the CLI to configure the AccessKey pair, bucket, and region.
3. Run the coscmd restore -r -d 10 -t Expedited examplefolder/ command to restore the files in the specified bucket.
   • -d specifies the number of days that the restored files remain in the restored state. If you leave this parameter empty, the default value 7 is used. In this example, -d is set to 10.
   • -t specifies the restore priority. Valid values: Expedited, Standard, and Bulk. The default value is Standard. In this example, -t is set to Expedited.

   The fees that you are charged to restore the archived files vary based on the restore priority. For more information, see the topics relevant to restoring data in the COS documentation.

4. After the data is restored, migrate the data to OSS. For more information, see Prepare for a migration task.

Qiniu Cloud KODO

Qiniu Cloud provides an API operation that you can call to restore archived files at the same time. You can perform the following operations:

1. Call the API operation to restore archived files at the same time. For more information, see the topics relevant to restoring data in the Qiniu Cloud-Object Storage documentation.

   Sample requests

   EncodedEntryURI specifies the Base64-encoded address in the <Bucket>:<Key> format. FreezeAfterDays specifies the number of days that the restored files remain in the restored state. The valid values range from 1 to 7.

   POST /batch HTTP/1.1
   Host:           rs.qbox.me
   Content-Type:   application/x-www-form-urlencoded
   Authorization:  Qiniu <AccessToken>
   
   op=/restoreAr/<EncodedEntryURI>/freezeAfterDays/<FreezeAfterDays>&op=/restoreAr/<EncodedEntryURI>/freezeAfterDays/<FreezeAfterDays>&...

   Sample responses

   HTTP/1.1 200 OK
   Content-Type: application/json
   Cache-Control: no-store

2. After the data is restored, migrate the data to OSS. For more information, see Preparations.