Preparations for migrating data - Data Online Migration - Alibaba Cloud Documentation Center

This topic describes the operations that you must perform before you migrate data.

Step 1: Create a destination bucket

Create an Object Storage Service (OSS) bucket as the destination to store the migrated data. For more information, see Create buckets.

Step 2: Create a RAM user and grant permissions to the RAM user

Important

The Resource Access Management (RAM) user is used to perform the data migration task. You must create RAM roles and perform the data migration task as the RAM user. We recommend that you create the RAM user within the Alibaba Cloud account that owns the source or destination OSS bucket.
If you have not created a RAM user, you can create a RAM user and grant permissions to the RAM user.

Log on to the RAM console of the Alibaba Cloud account. On the Users page, click Add Permissions in the Actions column of the RAM user that you created.

System policy: AliyunOSSImportFullAccess (Manage permissions for OSS Data Online Migration).

Custom policy: The policy must include the ram:CreateRole, ram:CreatePolicy, ram:AttachPolicyToRole, and ram:ListRoles permissions.

You can refer to Create a custom policy for permission management. The following is the policy script code:

{
    "Version":"1",
    "Statement":[
        {
            "Effect":"Allow",
            "Action":[
                "ram:CreateRole",
                "ram:CreatePolicy",
                "ram:AttachPolicyToRole",
                "ram:ListRoles"
            ],
            "Resource":"*"
        }
    ]
}

Step 3: Grant permissions on the source bucket

Perform the corresponding operations based on whether the source bucket belongs to the current Alibaba Cloud account.

Note

This current Alibaba Cloud account refers to the account that is used to log on to the Data Online Migration console. The operations vary based on whether the source bucket belongs to the current Alibaba Cloud account.

The source bucket belongs to the current Alibaba Cloud account

Automatic authorization:
We strongly recommend that you use automatic role authorization in the migration console to perform one-click authorization. You can perform this operation in Migration Implementation > Step 2 > Authorize Role.
Manual authorization:

Note

You can perform manual authorization in the following scenarios:

You want to grant permissions on multiple source buckets to a RAM role. This allows you to effectively manage multiple source buckets.
You do not want to create more RAM roles because the number of RAM roles within the current Alibaba Cloud account is close to the upper limit.
Automatic authorization is not applicable or cannot be used.

1. Create a RAM role for data migration

Log on to the RAM console of the RAM user that you created, and click Create Role on the role details page.

For Trusted entity type, select Alibaba Cloud Service.
For Trusted service, select Data Transport.
Enter a Role Name (The role name must be in lowercase letters).

2. Grant permissions on the source bucket

On the Roles page, click Add Permissions in the Actions column of the RAM role that you created.

Custom policy: The policy must include the oss:List* and oss:Get* permissions.

You can refer to Create a custom policy for permission management. The following is the policy script code:

Note

The following policy is only for reference. Replace <mySrcBucket> with the name of the source bucket.

For more information about OSS permission policies, see Common examples of RAM policies.

Important

If the source bucket contains objects that are encrypted by using Key Management Service (KMS), you must also grant the AliyunKMSFullAccess system policy to the RAM role.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*"
      ],
      "Resource": [
        "acs:oss:*:*:<mySrcBucket>",
        "acs:oss:*:*:<mySrcBucket>/*"
      ]
    }
  ]
}

The source bucket does not belong to the current Alibaba Cloud account

1. Create a RAM role for data migration

Log on to the RAM console of the RAM user that you created, and click Create Role on the role details page.

For Trusted entity type, select Alibaba Cloud Service.
For Trusted service, select Data Transport.
Enter a Role Name (The role name must be in lowercase letters).

2. Grant permissions on the source bucket

Important

If you configure a bucket policy by specifying policy statements to grant the RAM role the required permissions, the new bucket policy overwrites the existing bucket policy. Make sure that the new bucket policy contains the content of the existing bucket policy. Otherwise, the authorization based on the existing bucket policy may fail.

Log on to the OSS Management Console by using the account that owns the source bucket.
In the left-side navigation pane, click Buckets, and then select the bucket.
In the left-side navigation pane, choose Access Control > Bucket Policy.
In the Authorize By Policy Syntax section, add a custom bucket policy, and then click Edit And Save.

Grant the RAM role permissions to list and read all resources in the bucket
Note
The following access policy is for your reference only. Replace <otherSrcBucket> with the Source Bucket Name, <otherUid> with the source bucket owner's Primary Account UID, <myUid> with the Migration Console Primary Account UID, and <roleName> with the role name That You Just Created. Replace these placeholders with actual values. For more information about OSS access policies, see Common examples of RAM policies.
```
{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Principal": [
         "arn:sts::<myUid>:assumed-role/<roleName>/*"
      ],
      "Resource": [
        "acs:oss:*:<otherUid>:<otherSrcBucket>",
        "acs:oss:*:<otherUid>:<otherSrcBucket>/*"
      ]
    }
  ]
}
```

3. Grant KMS permissions

If the source bucket contains objects that are encrypted by using KMS, you must also grant the AliyunKMSFullAccess system policy to the RAM role.
If the source bucket contains objects that are encrypted by using a custom KMS key, you must also grant the RAM role permissions to access the KMS key. To do this, perform the following steps:
1. Log on to the KMS console and find the key.
2. Configure Key Policy, select Other Alibaba Cloud Account, and enter the ARN of the authorized principal. For more information, see Configure key policies.

Step 4: Grant permissions on the destination bucket

Perform the corresponding operations based on whether the destination bucket belongs to the current Alibaba Cloud account.

The destination bucket belongs to the current Alibaba Cloud account

Automatic authorization:
We strongly recommend that you use automatic role authorization in the migration console to perform one-click authorization. You can perform this operation in Migration Implementation > Step 3 > Authorize Role.
Manual authorization:
Grant permissions on the destination bucket
On the Roles page, click Add Permissions in the Actions column of the RAM role that you created.
- Custom policy: The policy must include the oss:List*, oss:Get*, oss:Put*, and oss:AbortMultipartUpload* permissions.
You can refer to Create a custom policy for permission management. The following is the policy script code:
Note
The following policy is only for reference. Replace <myDestBucket> with the name of the destination bucket that belongs to the current Alibaba Cloud account.
For more information about OSS permission policies, see Common examples of RAM policies.
Important
If server-side encryption is enabled for the destination bucket by using KMS, you must also grant the AliyunKMSFullAccess system policy to the RAM role.
```
{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Resource": [
        "acs:oss:*:*:<myDestBucket>",
        "acs:oss:*:*:<myDestBucket>/*"
      ]
    }
  ]
}
```

The destination bucket does not belong to the current Alibaba Cloud account

1. Grant permissions on the destination bucket

Important

Log on to the OSS Management Console by using the account that owns the destination bucket,
In the left-side navigation pane, click Buckets, and then select the bucket.
In the left-side navigation pane, choose Access Control > Bucket Policy.
In the Authorize By Policy Syntax section, add a custom bucket policy, and then click Edit And Save.
- Grant the RAM role permissions to list, read, delete, and write all resources in the bucket.

Note

The following access policy is for your reference only. Replace <otherDestBucket> with the Destination Bucket Name, <otherUid> with the destination bucket owner's Primary Account UID, <myUid> with the Migration Console Primary Account UID, and <roleName> with the Role Name created above. Replace these placeholders with actual values. For more information about OSS access policies, see Common examples of RAM policies.

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "oss:List*",
        "oss:Get*",
        "oss:Put*",
        "oss:AbortMultipartUpload"
      ],
      "Principal": [
         "arn:sts::<myUid>:assumed-role/<roleName>/*"
      ],
      "Resource": [
        "acs:oss:*:<otherUid>:<otherDestBucket>",
        "acs:oss:*:<otherUid>:<otherDestBucket>/*"
      ]
    }
  ]
}

2. Grant KMS permissions

If server-side encryption is enabled for the destination bucket by using KMS, you must also grant the AliyunKMSFullAccess system policy to the RAM role.
If server-side encryption is enabled for the destination bucket by using a custom KMS key, you must also grant the RAM role permissions to access the KMS key. To do this, perform the following steps:
1. Log on to the KMS console and find the key.
2. Configure Key Policy, select Other Alibaba Cloud Account, and enter the ARN of the authorized principal. For more information, see Configure key policies.

Step 5: Restore data in the source bucket

Before you create a migration task to migrate data of the Archive storage class, you must manually restore the data. Take note of the following items when you restore the data:

Before you create a source data address and a migration task, make sure that data of the Archive storage class is restored.
Specify the number of days during which the restored data remains in the restored state based on the amount of data that you want to migrate. This can prevent data from entering the archived state during data migration.
You may be charged for the restoration operation. The fee may be relatively high. For more information about the billing methods, contact the service provider that offers the source bucket.

Note

Data Online Migration does not restore data in archived objects at the source data address during data migration. The objects that are not restored or are being restored cannot be migrated.

For more information about how to restore objects in Alibaba Cloud OSS, see Restore objects.

Step 1: Create a destination bucket

Step 2: Create a RAM user and grant permissions to the RAM user

Step 3: Grant permissions on the source bucket

The source bucket belongs to the current Alibaba Cloud account

Automatic authorization:

Manual authorization:

1. Create a RAM role for data migration

2. Grant permissions on the source bucket

The source bucket does not belong to the current Alibaba Cloud account

1. Create a RAM role for data migration

2. Grant permissions on the source bucket

3. Grant KMS permissions

Step 4: Grant permissions on the destination bucket

The destination bucket belongs to the current Alibaba Cloud account

Automatic authorization:

Manual authorization:

Grant permissions on the destination bucket

The destination bucket does not belong to the current Alibaba Cloud account

1. Grant permissions on the destination bucket

2. Grant KMS permissions

Step 5: Restore data in the source bucket