:Migration implementation

Prerequisites

Prepare the following items before you start the migration:

• A Resource Access Management (RAM) user that you use to log on to the Data Online Migration console.

• The access key pair of the source BOS account, consisting of an AccessKeyId and a SecretAccessKey. You enter this key pair when you create the source address.

• An OSS bucket that stores the migrated data. You specify this bucket when you create the destination address.

• Role authorization for the destination bucket. The authorization method depends on whether the destination bucket belongs to the account of the Data Online Migration console. For the available options, see the Authorization role parameter in Step 3: Create a destination address.

Usage notes

• The online migration service accesses source data using the standard public APIs of the source storage service provider. The migration behavior depends on the provider's service implementation.

• Online migration consumes resources at both the source and the destination, potentially impacting your business operations. If your services are critical, evaluate the potential impact and configure a speed limit for the migration task, or run the task during off-peak hours.

• The online migration service checks files at both the source and destination before a migration begins. However, if a file at the source has the same name as a file at the destination and the migration task is configured to overwrite existing files, the service overwrites the destination file. If the files contain different data, rename one of the files or back up the destination file before starting the migration.

• The online migration service preserves the last modified time of source files. If a lifecycle rule is configured for the destination bucket, a migrated file whose last modified time matches the rule's conditions may be deleted or transitioned to the specified archive storage class.

Migration limitations

Review the following limitations and risks before you create any migration task:

• You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

• A quota also applies to the number of migration tasks that can run concurrently. For the quota that applies to your region, see the important notice in Step 4: Create a migration task.

• BOS attributes are handled as follows during migration:

  • Attributes that are migrated: x-bce-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, and Content-Language.

    Warning

    When the source address uses a standard Baidu BOS domain name, a GetObject call forces Content-Disposition to attachment. This attribute is migrated to the destination, where it no longer matches the actual attribute at the source. When a browser accesses such a file, for example an HTML file, the file is downloaded instead of displayed.

  • Attributes that are not migrated (including but not limited to): Expires, StorageClass, ACL, server-side encryption, and Tagging.

    Note

    The behavior of attributes that are not listed cannot be determined in advance. Use the result of the actual migration as the reference.

Step 1: Log on to the console and select a region

1. Log on to the Data Online Migration console as the RAM user that you created.

2. The region you select in the upper-left corner of the top navigation bar is the migration service deployment region. Therefore, select the region where your data source is located or the one closest to it.

   The region you select at the top of the console is the migration service deployment region. Available regions in the Chinese mainland include China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), and China (Ulanqab). Other available regions include China (Hong Kong), Singapore, Germany (Frankfurt), and US (Virginia).

   Important

   • Data endpoints and migration tasks are region-specific. Choose your region carefully.

   • We recommend selecting your source data's region. If this region is not available, select the closest available region.

   • For cross-border migrations, we recommend enabling transfer acceleration to improve migration speed. Buckets with transfer acceleration enabled incur transfer acceleration fees. For more information about transfer acceleration, see Accelerate access to OSS.

Step 2: Create a source address

The source address points the migration task to the BOS bucket that holds your data.

1. In the navigation pane on the left, choose Data Online Migration > Address Management, and then click Create Address.

2. On the Create Data Address page, configure the following parameters and then click OK.

Step 3: Create a destination address

Now that the source address is ready, create the destination address that identifies the OSS bucket that receives the data.

1. In the navigation pane on the left, choose Data Online Migration > Address Management, and then click Create Address.

2. On the Create Data Address page, configure the following parameters and then click OK.

Step 4: Create a migration task

1. In the left-side navigation pane, choose Data Online Migration > Migration Tasks, and then click Create Task.

2. On the Select Address page, configure the following parameters, and then click Next.

   Parameter	Required	Description
   Name	Yes	Enter a name for the migration task. The name must meet the following requirements: The name must be 3 to 63 characters in length. The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive. The name cannot start with a hyphen (-) or an underscore (_).
   Source Address	Yes	Select an existing source address.
   Destination Address	Yes	Select an existing destination address.

3. On the Task Configurations page, configure the following parameters.

   Parameter	Required	Description
   Basic settings
   Migration Bandwidth	No	Select the migration bandwidth. Default: The maximum available bandwidth. The actual speed depends on the file size and the number of files. Specify an upper limit: Specify a maximum bandwidth limit. Important The actual migration bandwidth depends on factors such as the data source, network conditions, destination throttling, and file size, and may not reach the specified upper limit. Select a value based on your data source, destination, business requirements, and network bandwidth. Improper throttling can disrupt your business operations.
   Files Migrated Per Second	No	Select the number of files migrated per second. Default: Uses the default rate for files migrated per second. Specify an upper limit: Specify a maximum number of files migrated per second. Important The actual number of files migrated per second is affected by factors such as the data source, network conditions, destination throttling, and file size, and may not reach the specified upper limit. Select a value based on your data source, destination, business requirements, and network bandwidth. Improper throttling can disrupt your business operations.
   Overwrite Mode	Yes	Specifies how to handle files with the same name. Do not overwrite: Skips the file. Overwrite All: The source file overwrites the destination file. Overwrite based on the last modification time: If the last modified time of the source file is later than that of the destination file, the destination file is overwritten. If the last modified time of the source and destination files are the same, the destination file is overwritten if their Size or Content-Type differs. Warning With the Overwrite based on the last modification time policy, there is a risk that an older file may overwrite a newer file because it does not strictly guarantee that newer files will be preserved. If you select the Overwrite based on the last modification time policy, ensure that the source files can return metadata such as last modified time, Size, and Content-Type. Otherwise, the overwrite policy may not work as expected. When you select Do not overwrite or Overwrite if last modified time is later, the service sends one request to the source and one to the destination to retrieve metadata for the overwrite check. This incurs request fees at both the source and the destination.
   Audit settings
   Migration Report	Yes	Specifies how to push the migration report. Do not push (Default): Does not push the migration report to the destination bucket. Push: Pushes the migration report to the destination bucket. For path details, see Next steps. Important Pushing the migration report consumes storage space at the destination. Pushing the migration report may be delayed. Each task execution record has a unique ID. The migration report is pushed only once. Do not delete it unless necessary.
   Migration Logs	Yes	Specifies how to push the migration log. Do not push (Default): Does not push the migration log. Push: Pushes the migration log to Log Service (SLS). You can view the migration log in SLS. Push only file error logs.: Pushes only file error logs to Log Service (SLS). You can view these logs in SLS. When you select Push or Push only file error logs., Online Migration Service creates a project in Log Service (SLS) named aliyun-oss-import-log-<Alibaba Cloud account ID>-<current deployment region>, for example, aliyun-oss-import-log-137918634953****-cn-hangzhou. Important Before you select Push or Push only file error logs., ensure you have completed the following operations. Otherwise, the migration task may fail. You have activated Log Service (SLS). You have granted the required permissions on the Authorize page.
   Authorize	No	This option appears only when you set Migration Logs to Push or Push only file error logs.. Click Authorize to go to the Cloud Resource Access Authorization page. The system creates the AliyunOSSImportSlsAuditRole role and grants it the necessary permissions. Click Agree to Authorization to complete the authorization.
   Filters
   File Name	No	A filter based on filenames. Supports Include and Exclude rules using regular expressions based on the RE2 library (only a subset of the syntax is supported). For example: .*\.jpg$ matches all files that end with .jpg. ^file.* matches all files at the root level that start with file by default. If you set a prefix for the source address, such as data/to/oss/, you must use ^data/to/oss/file.* to match all files that start with file under that prefix. .*/picture/.* matches any subdirectory named picture. Important When you use an Include rule, all matching files are migrated. If there are multiple Include rules, any file that matches at least one rule is migrated. For example, if you have two files, picture.jpg and picture.png, and you set an Include rule for .*\.jpg$, only picture.jpg is migrated. If you also add an Include rule for .*\.png$, both files are migrated. When you use an Exclude rule, no matching files are migrated. If there are multiple Exclude rules, any file that matches at least one rule is not migrated. For example, if you have two files, picture.jpg and picture.png, and you set an Exclude rule for .*\.jpg$, only picture.png is migrated. If you also add an Exclude rule for .*\.png$, neither file is migrated. Exclude rules take precedence. If a file matches both an Exclude rule and an Include rule, the file is not migrated. For example, for a file named file.txt, if you set an Exclude rule for .*\.txt$ and an Include rule for file.*, the file file.txt is not migrated.
   File Modification Time	No	Filters files based on their last modified time. You can specify a time range to migrate only files whose last modified time falls within that range. The rules are as follows: If you specify only a start time of January 1, 2019, the task migrates only files last modified on or after that date. If you specify only an end time of January 1, 2022, the task migrates only files last modified on or before that date. If you specify a start time of January 1, 2019 and an end time of January 1, 2022, the task migrates only files last modified on or between these two dates.
   Migration configuration
   Retain last modified time	Yes	Specifies whether to retain the last modified time of the source file. Retain (Default): The destination file retains the last modified time of the source file. Do not retain: The last modified time is not set.
   Convert Appendable Object to Normal Object/Multipart Object	No	Specifies whether to convert Appendable Objects to Normal Objects or Multipart Objects. Yes: Converts source Appendable Objects to Normal Objects or Multipart Objects at the destination. No: Source Appendable Objects are migrated to the destination as Appendable Objects. Important This option is visible and configurable only if your account is on the allowlist. This option is currently supported only for OSS-to-OSS migration tasks.
   Specify destination storage class	No	Specifies whether to set the storage class for destination files. Specify: Migrated files are assigned the specified storage class at the destination. The following storage classes are available: Standard Infrequent Access Archive Cold Archive Deep Cold Archive Do not specify (Default): Migrated files at the destination use the default storage class of the destination bucket. Important This option is visible and configurable only if your account is on the allowlist. This option is currently supported only for tasks with an OSS destination.
   Task scheduling
   Execution time	No	Important If a migration task is still running at its next scheduled execution time, that execution is postponed until the next scheduled time after the current run completes. This continues until the specified number of executions is reached. Concurrency limit for migration tasks: Deployment regions in the Chinese mainland and China (Hong Kong) support a maximum of 10 concurrent tasks. Overseas regions support a maximum of 5 concurrent tasks. Exceeding this limit may cause scheduled tasks to fail. Defines when to run the migration task. Immediately: Runs the task as soon as it is created. At the Specified Time: Specifies a daily time window for the task to run. By default, the task starts at the specified start time and pauses at the specified stop time. Periodic Scheduling: Runs the task based on a specified frequency and number of executions. Execution frequency: Supports five types of frequencies: Hourly, Daily, Weekly, Specific days of the week, and Custom. For more information, see Execution frequency reference. Number of executions: Specifies how many times the task runs. If not set, the task runs once by default. For the maximum number of executions, refer to the console prompt. Important You can manually start and stop tasks at any time, regardless of the custom execution time settings.

4. Read the Online Migration Service Agreement, select I understand and confirm the compliance commitment statement and acknowledge my responsibility to verify data consistency after the migration task is complete, and then click Next.

5. Review your configuration settings. If everything is correct, click OK and wait for the migration task to run.

Execution frequency

Step 5: Verify data