By Yashi Su
When enterprises migrate container service platforms across clouds or from on-premises data centers to the cloud, they are confronted with multi-faceted challenges. These include, but are not limited to, data security, service interruption, platform compatibility, and migration complexity. For application systems with complex microservice architectures and large volumes of data, migrating a container service platform requires intricate operations, and any misstep can lead to business interruptions or data loss.
This article will briefly introduce the backup center of Alibaba Cloud Container Service for Kubernetes (ACK), and take the specific challenges encountered by a technology company in its actual migration process as an example to explain how to effectively use the backup center to help enterprises migrate container service platforms.
ACK provides the backup center as an all-in-one containerized disaster recovery and migration solution for businesses that run in Kubernetes clusters.
Backup center overview: https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/backup-center-overview
Disaster recovery: The O&M personnel of a cluster can create periodic backup plans or one-time application backups with a single click from the console. Compared with ETCD backup, the backup center supports selecting applications for backup based on dimensions such as namespaces, labels, and resource types. For stateful applications, it also supports backing up the storage volume data mounted by the business simultaneously. Enterprises with established GitOps workflows can also use the backup center's data protection feature to perform disaster recovery that targets only the storage volume data.
Container service platform migration: In hybrid cloud environments, the backup center makes it easy to back up selected resources and storage volume data from the cluster to Alibaba Cloud, and then restore this backup in the target ACK cluster, achieving seamless application migration. The backup center offers an intuitive user interface and enhances observability, making it convenient to verify specific resource backup lists and monitor the progress of real-time data backups.
Specifically, the O&M personnel of a cluster can follow the following steps to back up services from the original platform and restore them to ACK:
• Create an ACK One registered cluster to connect Kubernetes clusters from other cloud providers or on-premises that need to be migrated.
(If the business involves a large volume of data, it's recommended to establish connectivity via Cloud Enterprise Network.)
• Once connected, the Kubernetes clusters gain access to most ACK capabilities. Deploy the backup center component within the registered cluster.
• In the registered cluster, back up the services to be migrated to a specified backup vault. If the services are stateful applications, you can back up the data within the storage volumes simultaneously.
• Create or use an existing target ACK cluster and make sure that the cluster has sufficient computing resources to deploy the backup center component.
• Initialize the backup vault in the target ACK cluster. After waiting for the backup records to synchronize, proceed to restore the services.
Overview of ACK One registered clusters: https://www.alibabacloud.com/help/en/ack/overview-9
A technology company needs to migrate all applications and their associated storage volumes in a business cluster that provides external services and an internal cluster to Alibaba Cloud. The following section describes the challenges encountered in migrating both cluster resources and storage volume data, as well as the solutions provided by the backup center.
Challenges: On one hand, the original Kubernetes clusters have been running for several years, with some resources having unclear purposes or being unable to be redeployed with current CI tools. During migration, it's critical to ensure these resources are fully restored to maintain business continuity. On the other hand, since certain self-built databases need to be migrated to Alibaba Cloud's middleware, the backup strategy requires fine-tuning. This includes excluding unnecessary applications and related data to minimize data transfer and improve migration efficiency.
Solutions: The requirement can be abstracted into needing a full backup while excluding specific applications and corresponding storage volumes. Given that cross-cluster migration involves changes to the cluster's system components and runtime environment, resources like those under the kube-system namespace should also be excluded. You can select the backup content by excluding namespaces or annotating the volumes that do not need to be backed up.
Similarly, when you need to migrate only some applications and resources, you can filter the backup content through namespaces or labels. The backup center can adaptively adjust the backup content, such as automatically backing up IngressClass when backing up Ingress, and automatically backing up CRD when backing up CR.
After the backup is complete, you can query the console to check the backed-up cluster resources and storage claim lists to confirm that nothing is missed.
Challenges: The system components and corresponding configuration methods provided by different cloud vendors are more or less different. As a result, the company cannot simply copy the original configuration files or directly use the existing CI tools for migration; instead, it needs to adapt related resources. In this case, the items that need to be adapted are:
• Network-related: The load balancing services provided by cloud vendors are different, and users are required to use specified annotations to configure services and Ingress resources in the cluster. These annotations need to be reconfigured during cross-cloud migration.
• Storage-related: The storage media provided by cloud vendors are similar, but the CSI storage plug-in drivers of clusters are different. This necessitates changes to PVC, PV, and other storage-related resource configurations.
• Image Registry URLs: Before migrating services to ACK, most companies choose to migrate images to Alibaba Cloud Container Registry (ACR). As a result, image-related fields in Deployment, StatefulSet, Job, and other application resources need their registry URLs updated.
Solutions: The backup center supports resource adjustments before restoration. Simple adjustments can be achieved directly through console options, while complex adjustments can be flexibly configured through ConfigMap before restoration. To address the aforementioned adaptation requirements, the following features are available:
• Network-related: You can use ConfigMap to overwrite annotations and automatically reserve ports by default.
• Storage-related: Automatically abstracts away differences between CSI storage plug-ins from major cloud providers, converting them into the default ACK CSI storage class and generating new PVC and PV resource description files, such as YAML files. If there are specific storage class conversion needs, the new storage class for each PVC can also be selected on the console during recovery.
• Image Registry URLs: By filling out mappings, it automatically converts matching image registry URLs.
Challenges: The company has a need to upgrade earlier-version clusters to later versions. Different versions of Kubernetes clusters may have incompatible changes in API resource definitions and behaviors, requiring adjustments to existing YAML files to ensure they function correctly in the new version.
Solutions: The backup center supports seamless migration of resources from Kubernetes clusters of version 1.16 or later to the latest version without any additional configuration or operations. The specific implementation is:
• By default, when you back up resources, all API versions supported by the resources in the cluster are backed up. For example, when you back up a Deployment in a 1.16 cluster, the four versions of extensions/v1beta1, apps/v1beta1, apps/v1beta2, and apps/v1 are backed up.
• When you restore resources, the API version recommended by the restore cluster is used to restore the resources. For example, a Deployment backed up from a 1.16 cluster will be restored as apps/v1 in a 1.31 cluster, automatically performing version conversion.
• For resources without compatible versions, such as the extensions.ingress resource in a 1.16 cluster, the backup center supports upgrading them to the latest version according to the default Kubernetes configuration and also allows skipping restoration for manual creation by O&M personnel.
Challenges: The company needs to transfer data from its original cloud provider's disks to Alibaba Cloud disks. This process is technically challenging and involves multiple stages of data transfer, significantly increasing the complexity and operational intricacy of the migration. It can also lead to longer service downtimes, impacting customer experience and service quality.
Solutions: In a hybrid cloud scenario, the backup center leverages Alibaba Cloud backup services to store volume data in the cloud. It supports block storage, object storage, and file system storage from other cloud providers, as well as mainstream self-built storage like Ceph and NFS.
During restoration, it automatically selects the default ACK CSI storage class based on the original storage type to restore data to corresponding Alibaba Cloud disks, OSS (Object Storage Service), or NAS (Network Attached Storage). Custom storage classes can also be specified for recovery, facilitating the conversion of storage media.
Challenges: Due to the real-time dynamism of storage data, services typically need to be paused during migration to prevent new data writes, avoiding data loss or mismatches after migration. Such downtime poses a challenge to business continuity.
Solutions: Alibaba Cloud backup service supports incremental backups, allowing for initial backups of storage volume data during off-peak business hours, significantly reducing the actual amount of data backed up during downtime. Additionally, the backup service compresses and deduplicates data during the backup process, effectively enhancing backup efficiency.
Challenges: During cross-cloud migration, it's essential to consider encryption solutions for data in transit to ensure the security of sensitive information.
Solutions: The cloud backup service employs local data encryption to secure data during transmission and storage in the cloud. Additionally, server-side encryption can be enabled for OSS buckets managed by the backup vault.
To ensure the integrity of application migration, the backup process covers all CRD resources in both CI and production clusters while excluding applications and storage volume data that need to switch to middleware as required. The backup center supports automatic API compatibility and flexible resource adjustments, assisting the company in seamlessly merging and upgrading clusters while avoiding numerous resource conflicts and adjustments.
The storage class conversion feature of the backup center enables data migration across cloud disks, abstracting away the differences between the underlying data type conversion and the underlying storage interface (CSI) drivers of different cloud service providers. Furthermore, to minimize the impact of migration on business operations, incremental backup capabilities are utilized to perform data backups during off-peak business hours without affecting normal application operations, thereby reducing downtime during the actual switchover.
184 posts | 32 followers
FollowAlibaba Container Service - April 17, 2024
Alibaba Clouder - July 15, 2020
Hironobu Ohara - February 3, 2023
Aliware - June 23, 2021
Hironobu Ohara - February 3, 2023
Alibaba Container Service - October 30, 2024
184 posts | 32 followers
Follow
Cloud Migration Solution
Secure and easy solutions for moving you workloads to the cloud
Learn More
Database Migration Solution
Migrating to fully managed cloud databases brings a host of benefits including scalability, reliability, and cost efficiency.
Learn More
ISV Solutions for Cloud Migration
Alibaba Cloud offers Independent Software Vendors (ISVs) the optimal cloud migration solutions to ready your cloud business with the shortest path.
Learn More
Oracle Database Migration Solution
Migrate your legacy Oracle databases to Alibaba Cloud to save on long-term costs and take advantage of improved scalability, reliability, robust security, high performance, and cloud-native features.
Learn MoreMore Posts by Alibaba Container Service
