All Products
Search
Document Center

:Migrate data to the cloud and verify the migration result

Last Updated:Sep 30, 2024

This topic describes how to return a Data Transport II device, migrate your data to the cloud, and verify the migration result after your data is migrated to the Data Transport II device and the device is powered off.

Prerequisites

Your data is migrated to a Data Transport II device, the data migration task is complete, and the device is powered off.

Return the device

1. Check the migration task status

Run the following commands in sequence to check whether the migration task status is succeed, and check whether the migrated data is consistent with that in the source file by performing CRC-64 verification.:

  • Run the cdmgwclient command to access the corresponding directory.

  • Run the bash console.sh status <job_name> command to view the migration task status. Replace job_name with the actual migration task name.

2. Stop the migration task

Important

Make sure that you perform the following operation after the migration is complete. Otherwise, some data may fail to be migrated. You are not allowed to power off the device before you stop the task.

Run the following command in sequence to stop the migration task:

  • beforepoweroff

3. Uninstall and lock a storage pool

Run the following command to uninstall and lock a storage pool:

  • crypt close

4. Power off the device

Run the poweroff command to power off the device.

5. Check accessories

  1. Check whether two optical transceiver modules are installed on a 10 Gbit/s optical port.

  2. Check whether two optical transceiver modules are installed on a 25 Gbit/s optical port.

  3. Check whether the power cable is properly stowed under the top panel of the device.

  4. Return the Data Transport device through the specified logistics company that cooperates with Alibaba Cloud.

Migrate data to the cloud and verify the migration result

Grant permissions to a RAM role

Alibaba Cloud uses a specific Resource Access Management (RAM) role to upload data in the Data Transport device to the Object Storage Service (OSS) bucket that you specified. You must grant the required permissions to the specified RAM role. Perform the following steps to grant the required permissions to the specified RAM role:

Important

If you use bucket policies to grant permissions to the RAM role by syntax, the newly added policy overwrites the existing policy. Make sure that the newly added policy contains the content of the existing policy. Otherwise, operations related to the existing policy may fail.

  • Log on to the OSS console.

  • In the left-side navigation pane, click Buckets and click the name of the desired bucket.

  • In the left-side navigation pane, choose Permission Control > Bucket Policy.

  • On the Bucket Policy tab, select Add by Syntax. In the editor, specify a custom bucket policy, click Edit, and then click Save.

    • Grant the RAM role the permissions to list, read, delete, and write all resources in the bucket.

Note

The following policy configurations are for your reference only. Replace the following parameters with the actual values.

  • <mybucket>: In the brackets, enter the name of the destination bucket.

  • <myuid>: In the brackets, enter the UID of the Alibaba Cloud account to which the destination bucket belongs.

  • <uid>: In the brackets, enter the UID provided by Alibaba Cloud, such as 1986505809429276.

  • <rolename>: In the brackets, enter the role name provided by Alibaba Cloud, such as mgw-data-transport-role.

Replace only the preceding parameters. For more information about RAM policies for OSS, see Common examples of RAM policies.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Principal": [
         "arn:sts::1986505809429276:assumed-role/mgw-data-transport-role/*"
      ],
      "Resource": [
        "acs:oss:*:<myuid>:<mybucket>",
        "acs:oss:*:<myuid>:<mybucket>/*"
      ]
    }
  ]
}

Migrate data to the cloud

Alibaba Cloud uses a specific RAM role that you grant the required permissions to migrate the data in the Data Transport device to the cloud. After the data is migrated to the cloud, Alibaba Cloud will provide you with a data migration report. You can verify the migration results based on the data migration report. This is the end of the migration task.

Important

After the data migration is complete, the data migration report is stored in the specified OSS bucket. For more information, see the Data migration report section.

Data migration report

After a data migration report is generated, the migration report files are stored in the specified OSS bucket. The directories in which migration report files are stored are shown in the following code block:

OSS://<bucket>/<prefix>/aliyun_import_report/<uid>/<jobid>/<runtimeid>/total_list/
OSS://<bucket>/<prefix>/aliyun_import_report/<uid>/<jobid>/<runtimeid>/failed_list/
OSS://<bucket>/<prefix>/aliyun_import_report/<uid>/<jobid>/<runtimeid>/skipped_list/

Field

Description

bucket

The OSS bucket to which data is migrated.

prefix

The prefix of the directory in the destination bucket to which you migrate the data.

uid

The UID provided by Alibaba Cloud. Example: 1986505809429276.

jobid

The migration task ID provided by Alibaba Cloud.

runtimeid

The execution record ID of the migration task. This ID is provided by Alibaba Cloud.

A migration report contains three types of files: the file that lists all files to be migrated, the file that lists the files that fail to be migrated, and the file that lists the files that are skipped during migration. You can download these files and view them for more details. We recommend that you use the ossbrowser or ossutil tool to view these files.

Note

Rules for naming migration report files

  • A file named uid@jobid@runtimeid_total_list_n lists all files to be migrated. One or more files of this type may exist. The value of n is an integer greater than or equal to 0.

  • A file named uid@jobid@runtimeid_failed_list_n lists the files that fail to be migrated. One or more files of this type may exist. The value of n is an integer greater than or equal to 0.

  • A file named uid@jobid@runtimeid_skipped_list_n lists the files that are skipped during migration. One or more files of this type may exist. The value of n is an integer greater than or equal to 0.

The fields contained in the migration report files describe various attributes of the files during the migration from a Data Transport device to the destination bucket. The following table describes the fields.

Field

Description

srcObjectName

The name of the source file.

destObjectName

The name of the destination file.

srcObjectSize

The size of the source file.

destObjectSize

The size of the destination file.

srcContentMd5

The MD5 hash value of the source file, which is used for consistency verification.

destContentMd5

The MD5 hash value of the destination file, which is used for consistency verification.

srcCrc64

The CRC-64 value of the source file, which is used for consistency verification.

destCrc64

The CRC-64 value of the destination file, which is used for consistency verification.

srcLastModified

The time when the source file was last modified.

destLastModified

The time when the destination file was last modified.

srcVersionId

This field is left empty.

destVersionId

This field is left empty.

startTime

The time when the migration started.

endTime

The time when the migration was complete.

isAbnormal

Indicates whether an exception occurred during migration. A value of false indicates that no exception occurred. A value of true indicates that an exception occurred.

detailMsg

The cause of the exception.

Verify the migration result

Data Transport only processes the migration of data and does not ensure data consistency or integrity. After the migration task is complete, you must review all the migrated data and verify the data consistency between the source and destination data addresses. You must also revoke the permissions on the Data Transport device granted to the specified RAM role.

Warning

Make sure that you verify the migrated data at the destination after the migration task is complete. If you delete the source data without verifying data consistency, you are responsible for any losses and consequences that arise.