Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Making a Difference with Alibaba Cloud OSS – Best Practices for Data Migration

Making a Difference with Alibaba Cloud OSS – Best Practices for Data Migration

Learn about the best practices of Alibaba Cloud OSS and how easy it is to migrate data from one bucket to another, from an on-premises setup to Alibaba Cloud OSS.

By Shantanu Kaushik

Alibaba Cloud Object Storage Service (OSS) offers a best-in-class user experience with its network-based data access services. When it comes to storage, any developer or administrator will have to make an informed decision based on what the service or database will need in terms of resources to properly execute and maintain a viable link for high-availability and reliability.

Cloud computing has introduced ways to cut down time-consuming tasks for system administrators. The infrastructure that is being offered is highly-scalable and allows for real-time, on-demand provisioning without scaling the resources up or down.

Storage is one area that has made a significant improvement in the cloud universe. Alibaba Cloud has upgraded the storage service to offer maximum performance and redundancies to ensure a stable service. With that, disaster recovery plays an important role in service selection for any IT strategist.

The Object Storage Service enables you to store and retrieve a variety of unstructured data files over a network. These may include text, images, audio, and video files.

With OSS, one can:

  • Create single or multiple buckets and upload objects to each bucket
  • Deploy at a small, medium, or large scale
  • Share the bucket using its assigned web address
  • Deploy it with any product that requires an unaffected data stream
  • Manage the Object Storage Service using the console or a variety of APIs, tools, and SDKs.
  • Backup and restore data

Best Practices | Object Storage Service | Data Migration

Alibaba Cloud Object Storage Service (OSS) allows for easy data migration. Migration is a task that requires the utmost clarity in service and delivery. OSS supports the functionality to migrate data between buckets that are stored across multiple regions. It also supports data migration between the same region and migration between different Alibaba Cloud accounts.

The Alibaba Cloud Data Transport helps migrate data from on-premises to the Object Storage Service. It also offers migration between cloud resources and from one Alibaba Cloud account to another. 'Let's take a look at a scenario where an external data center is being migrated to Alibaba Cloud Object Storage Service (OSS).

1

The Data Transport service enables you to easily transport your data from various data sources into Alibaba Cloud OSS. To do this, all you need to do is to log into your Data Transport console and create a migration job. You will need to punch in the source data stream address and the OSS end-point where you plan to migrate data.

Before migrating, make sure you:

  1. Analyze and access the data requirement

    • Size of data to be transported, the number of files, and the type of data. You will need this information when creating a bucket at the end-point.
  2. Create a bucket at the destination end-point with Alibaba Cloud Object Storage Service

  3. Authorize and grant permissions

    • Here you will need to grant permissions to one or many users, depending on your requirement. You can set the access level of any user, give access, and block privileges for selected users.
    • You will need to use the Resource and Access Management]( https://www.alibabacloud.com/product/ram) to achieve this. RAM provides Identity and Access Management features for a wide variety of Alibaba Cloud products.
    • You can grant full access, read-only access, administrator access, ECS full access, etc. Go through the list and select which user will be granted permission for which task and operation.

Things to keep in mind while creating a migration job:

  • Before you migrate, the data transport service will check the origin and destination files for duplicates. The migration job will automatically replace the duplicate files with the ones being migrated. Be sure to back up any data at the end-point before you start the migration job.
  • The migration job is network resource hungry. To ensure that your network doesn't get bogged down during the migration process, you can easily set a speed limit for data transfers. Do this to ensure business application availability that may be using the shared resources.
  • The migration Job will create symbolic links for data and files that are being migrated. It will automatically link any file specified with an address to the URL already pointing to this file on the website/user resource.
  • You can only transport data with a limitation of one bucket at a time. Be sure to create independent tasks for different data buckets.

Things to keep in mind while creating an end-point in Alibaba Cloud OSS:

You can select HTTP or HTTPS-based end-points and access those from the Internet. Follow these links:

If you are using Internal HTTP or HTTPS end-points, you can access those with these links:

Choose the Correct Migration Type

When you have checked all of the boxes for bucket migration, you will need to choose a migration type. You can choose to migrate using full migration or incremental migration.

Full Migration

With full migration, you can choose the Start Point Time file, this will enable you to migrate files that were modified after the specified date. When you choose this setting and re-run the migration task after some time, only the data that has changed after the last migration job will be considered for migration.

Incremental Migration

Alibaba Cloud Object Storage Service (OSS) allows you to migrate data in short bursts. You can specify the migration interval, Start Point Time, and migration times as parameters. After the initial migration with modification time as parameter is completed, the incremental migration scenario will follow the specified migration interval and migration times parameters to run the migration job.

Manage Your Migration

After you have created the migration job, you can check its status from the console. Alternatively, you can select the flow control settings for your migration job:

  1. Log in to your Data Transport Console

  2. Navigate to "Data Online Migration"

    • Select "Migration Jobs"
    • Manage
  3. Here, you can stop or reset a migration job

  4. If you wish to view a migration report:

    • Click on "Generate Migration Report"
    • Then, export it

  5. If your migration fails, you can check the report for detailed reasons:

    • Go to "Manage"
    • Click on "Retry"

The Data Transport service makes almost every migration task possible with a few clicks. You can choose from the options that best suit you and finish a migration job without losing any business.

Wrapping Up

Business Continuity is important. Alibaba Cloud OSS has made it easy to migrate data from one bucket to another or from an on-premises setup to Alibaba Cloud OSS. You can also choose to migrate from bucket to bucket from different Alibaba Cloud OSS accounts.

The service utilizes Data Transport for transmission and also incorporates products like ECS, RAM, CDN, and MTS for better management and data operation. The overall system has an operating system that optimizes data delivery with a secure and reliable channel at hand.

Upcoming Articles

  1. Migrate Data from Third Party Services to OSS
Migration Best practices Object Storage Service Disaster Recovery Resource and Access Management data transports
0 0 0
Share on

Read previous post:

Making CDN Programmable: Use Cases, Script Overview, and EdgeScript Demos

Read next post:

Cloud Storage Re-Defined – Alibaba Cloud OSS (Part 1)

Alibaba Clouder

2,605 posts | 747 followers

You may also like

Comments

Alibaba Clouder

2,605 posts | 747 followers

Related Products

More Posts by Alibaba Clouder

See All