This topic describes how to use Velero with restic integrated to migrate applications and persistent volumes (PVs) from a self-built Kubernetes cluster to a Container Service for Kubernetes cluster. You can also use the same method to migrate applications and PVs from a Kubernetes cluster managed by another cloud vendor to a Container Service for Kubernetes cluster.

Background information

In this example, you migrate a WordPress application from a self-built Kubernetes cluster to a Container Service for Kubernetes cluster. The WordPress application consists of the wordpress and mysql components. Each component uses a separate Network File System (NFS) PV to store application data. The WordPress application is exposed through a NodePort service.

Procedure

  1. Step 1: Migrate container images to Container Registry
  2. Step 2: Deploy MinIO in the Container Service for Kubernetes cluster
  3. Step 3: Deploy a WordPress application and Velero in the self-built Kubernetes cluster
  4. Step 4: Deploy Velero in the Container Service for Kubernetes cluster and restore the WordPress application
  5. Step 5: Check the running status of the WordPress application

Step 1: Migrate container images to Container Registry

Generally, a self-built Kubernetes cluster resides in a local IDC and container images are stored in a self-built image repository. Before migrating applications, you must migrate the container images to Container Registry. For more information, see Migrate container images.

The WordPress application uses the following container images:
registry.api.paas.com:5000/admin/wordpress:latest
registry.api.paas.com:5000/admin/mysql:8
After the migration, the corresponding images in Container Registry are as follows:
registry.cn-hangzhou.aliyuncs.com/ack-migration/wordpress:latest
registry.cn-hangzhou.aliyuncs.com/ack-migration/mysql:8

Step 2: Deploy MinIO in the Container Service for Kubernetes cluster

MinIO is an object storage service. When you migrate the WordPress application, you can use MinIO to temporarily store data of the PVs used by the WordPress application.

  1. Create a namespace for MinIO.
    kubectl create ns minio
  2. Set a username and password for logging on to MinIO. Replace MINIO_ACCESS_KEY_VALUE and MINIO_SECRET_KEY_VALUE in the minio.yaml file with the user name and password respectively. Then, deploy MinIO in the Container Service for Kubernetes cluster.
    export MINIO_ACCESS_KEY=admin
    export MINIO_SECRET_KEY=admin12345
    export BUCKET_NAME=velero
    sed -i "s/MINIO_ACCESS_KEY_VALUE/$MINIO_ACCESS_KEY/g" minio.yaml
    sed -i "s/MINIO_SECRET_KEY_VALUE/$MINIO_SECRET_KEY/g" minio.yaml
    sed -i "s/BUCKET/$BUCKET_NAME/g" minio.yaml
    kubectl apply -f minio.yaml
    A 20 GB standard solid-state drive (SSD) disk is attached to the MinIO application for storing the data of PVs. The MinIO application is exposed through a LoadBalancer service. A bucket named velero is created in MinIO.
    The sample code in the minio.yaml file is as follows:
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: storage-pvc
      namespace: minio-test
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: alicloud-disk-ssd
      resources:
        requests:
          storage: 20Gi
    
    ---
    apiVersion: apps/v1beta1
    kind: Deployment
    metadata:
      namespace: minio-test
      name: minio
      labels:
        component: minio
    spec:
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            component: minio
        spec:
          volumes:
          - name: storage
            persistentVolumeClaim:
              claimName: storage-pvc
          - name: config
            emptyDir: {}
          containers:
          - name: minio
            image: minio/minio:latest
            imagePullPolicy: IfNotPresent
            args:
            - server
            - /storage
            - --config-dir=/config
            env:
            - name: MINIO_ACCESS_KEY
              value: MINIO_ACCESS_KEY_VALUE
            - name: MINIO_SECRET_KEY
              value: MINIO_SECRET_KEY_VALUE
            ports:
            - containerPort: 9000
            volumeMounts:
            - name: storage
              mountPath: "/storage"
            - name: config
              mountPath: "/config"
    
    ---
    apiVersion: v1
    kind: Service
    metadata:
      namespace: minio-test
      name: minio
      labels:
        component: minio
    spec:
      # ClusterIP is recommended for production environments.
      # Change to NodePort if needed per documentation,
      # but only if you run MinIO in a test/trial environment, for example with Minikube.
      type: LoadBalancer
      ports:
        - port: 9000
          targetPort: 9000
          protocol: TCP
      selector:
        component: minio
    
     ---
     apiVersion: batch/v1
    kind: Job
    metadata:
      namespace: minio-test
      name: minio-setup
      labels:
        component: minio
    spec:
      template:
        metadata:
          name: minio-setup
        spec:
          restartPolicy: OnFailure
          volumes:
          - name: config
            emptyDir: {}
          containers:
          - name: mc
            image: minio/mc:latest
            imagePullPolicy: IfNotPresent
            command:
            - /bin/sh
            - -c
            - "mc --config-dir=/config config host add BUCKET http://minio:9000 MINIO_ACCESS_KEY_VALUE MINIO_SECRET_KEY_VALUE  && mc --config-dir=/config mb -p BUCKET/BUCKET"
            volumeMounts:
            - name: config
              mountPath: "/config"
  3. Go to the Container Service console. Choose Ingresses and Load Balancing > Services in the left-side navigation pane. On the Services page, select the target cluster and namespace.
    In the service list that appears, you can find the deployed MinIO service.
  4. Click the public endpoint of the MinIO service to access MinIO Browser by using the IP address and port number.

Step 3: Deploy a WordPress application and Velero in the self-built Kubernetes cluster

  1. Deploy a WordPress application in the self-built Kubernetes cluster.
    Note In this example, you create a WordPress application in the self-built Kubernetes cluster and migrate the application to a Container Service for Kubernetes cluster. If you want to migrate an existing application, skip this step.
    The WordPress application consists of the wordpress and mysql components. Each component uses a separate NFS PV to store application data. The WordPress application is exposed through a NodePort service. The sample code in the YAML file for deploying the WordPress application is as follows:
    # 1. Create an NFS StorageClass.
    $ cat nfs-sc.yaml
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: nfs
    provisioner: helm.default/nfs
    reclaimPolicy: Delete
    $ kubectl apply -f  nfs-sc.yaml
    
    # 2. Create a secret to store the password of the mysql component. Assume that the password is mysql. You can run the echo -n "mysql" |base64 command to obtain the Base64-encoded string of the password.
    $ cat secret.yaml
    apiVersion: v1
    kind: Secret
    metadata:
      name: mysql
    type: Opaque
    data:
      password: bXlzcWw=
    $ kubectl apply -f secret.yaml
    
    # 3. Create a service, a persistent volume claim (PVC), and a deployment for the mysql component.
    $ cat mysql.yaml
    apiVersion: v1
    kind: Service
    metadata:
      name: mysql
      labels:
        app: mysql
    spec:
      type: ClusterIP
      ports:
        - port: 3306
      selector:
        app: mysql
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: mysql-volumeclaim
      annotations:
        volume.beta.kubernetes.io/storage-class: "nfs"
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
    
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: mysql
      labels:
        app: mysql
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: mysql
      template:
        metadata:
          labels:
            app: mysql
        spec:
          securityContext:
            runAsUser: 999
            runAsGroup: 999
            fsGroup: 999
          containers:
            - image: registry.api.paas.com:5000/admin/mysql:8
              name: mysql
              args:
                - "--default-authentication-plugin=mysql_native_password"
              env:
                - name: MYSQL_ROOT_PASSWORD
                  valueFrom:
                    secretKeyRef:
                      name: mysql
                      key: password
              ports:
                - containerPort: 3306
                  name: mysql
              volumeMounts:
                - name: mysql-persistent-storage
                  mountPath: /var/lib/mysql
          volumes:
            - name: mysql-persistent-storage
              persistentVolumeClaim:
                claimName: mysql-volumeclaim
     $ kubectl apply -f mysql.yaml
    
     # 4. Create a PVC, a deployment, and a service for the wordpress component.
     $ cat wordpress.yaml
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: wordpress
      name: wordpress
    spec:
      ports:
        - port: 80
          targetPort: 80
          protocol: TCP
          nodePort: 31570
      selector:
        app: wordpress
      type: NodePort
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: wordpress-volumeclaim
      annotations:
        volume.beta.kubernetes.io/storage-class: "nfs"
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: wordpress
      labels:
        app: wordpress
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: wordpress
      template:
        metadata:
          labels:
            app: wordpress
        spec:
          containers:
            - image: registry.api.paas.com:5000/admin/wordpress
              name: wordpress
              env:
              - name: WORDPRESS_DB_HOST
                value: mysql:3306
              - name: WORDPRESS_DB_PASSWORD
                valueFrom:
                  secretKeyRef:
                    name: mysql
                    key: password
              ports:
                - containerPort: 80
                  name: wordpress
              volumeMounts:
                - name: wordpress-persistent-storage
                  mountPath: /var/www/html
          volumes:
            - name: wordpress-persistent-storage
              persistentVolumeClaim:
                claimName: wordpress-volumeclaim
     $ kubectl apply -f wordpress.yaml
    In the test environment, bind the domain name of the WordPress application to the IP address of the node where the application is deployed in the hosts file of your operating system. Then, access the WordPress application through http://wordpress.myk8s.paas.com:31570/. Make sure that the WordPress application is accessible.
  2. Deploy Velero in the self-built Kubernetes cluster.
    1. Download Velero, install it, and verify the installation by checking the Velero version.
      The installation package used in this example is velero-v1.2.0-linux-amd64.tar.gz.
      $ tar xvf velero-v1.2.0-linux-amd64.tar.gz
      $ mv velero-v1.2.0-linux-amd64/velero /usr/bin/velero
      $ velero version
      Client:
        Version: v1.2.0
        Git commit: 5d008491bbf681658d3e372da1a9d3a21ca4c03c
      Server:
        Version:
    2. Add the access information of MinIO to the credentials-velero file.
      [default]
      aws_access_key_id=<YourAccessKeyId>
      aws_secret_access_key=<YourAccessKeySecret>
      Note In the preceding code, replace YourAccessKeyId and YourAccessKeySecret with the username and password for logging on to MinIO.
    3. Replace MINIO_SERVER_ADDRESS with the actual address of the MinIO server in the following command and run the command to deploy Velero.
      velero install --provider aws --image registry.cn-hangzhou.aliyuncs.com/acs/velero:v1.2.0 --bucket velero --secret-file ./credentials-velero --use-volume-snapshots=false --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://MINIO_SERVER_ADDRESS:9000 --use-restic --plugins registry.cn-hangzhou.aliyuncs.com/acs/velero-plugin-for-aws:v1.0.0 --wait
    4. Check the running status of pods.
      kubectl -n velero get po
      NAME                      READY   STATUS    RESTARTS   AGE
      restic-fqwsc              1/1     Running   0          41s
      restic-kfzqt              1/1     Running   0          41s
      restic-klxhc              1/1     Running   0          41s
      restic-ql2kr              1/1     Running   0          41s
      restic-qrsrn              1/1     Running   0          41s
      restic-srjmm              1/1     Running   0          41s
      velero-67b975f5cb-68nj4   1/1     Running   0          41s
  3. Back up the WordPress application.
    • If you want to back up only the WordPress application without PVs, run the following commands:
      $ velero backup create wordpress-backup-without-pv --include-namespaces wordpress
      Backup request "wordpress-backup-without-pv" submitted successfully.
      Run `velero backup describe wordpress-backup-without-pv` or `velero backup logs wordpress-backup-without-pv` for more details.
      
      $ velero backup get
      NAME                          STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
      wordpress-backup-without-pv   Completed   2019-12-12 14:08:24 +0800 CST   29d       default            <none>
    • If you want to back up the WordPress application with PVs, run the following commands:
      # Annotate the pods to which PVs are attached. Assume that the WordPress application runs in two pods, namely, wordpress-7cf5849f47-mbvx4 and mysql-74dddbdcc8-h2tls.
      # The PV attached to the wordpress-7cf5849f47-mbvx4 pod is mysql-persistent-storage. The PV attached to the mysql-74dddbdcc8-h2tls pod is wordpress-persistent-storage. Then, the commands for annotating the pods are as follows:
      $ kubectl -n wordpress annotate pod/wordpress-7cf5849f47-mbvx4 backup.velero.io/backup-volumes=wordpress-persistent-storage
      pod/wordpress-7cf5849f47-mbvx4 annotated
      $ kubectl -n wordpress annotate pod/mysql-74dddbdcc8-h2tls backup.velero.io/backup-volumes=mysql-persistent-storage
      pod/mysql-74dddbdcc8-h2tls annotated
      
      # Back up the WordPress application.
      $ velero backup create wordpress-backup-with-pv --include-namespaces wordpress
      Backup request "wordpress-backup-with-pv" submitted successfully.
      Run `velero backup describe wordpress-backup-with-pv` or `velero backup logs wordpress-backup-with-pv` for more details.
      $ velero backup get
      NAME                          STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
      wordpress-backup-with-pv      Completed   2019-12-12 14:23:40 +0800 CST   29d       default            <none>
      wordpress-backup-without-pv   Completed   2019-12-12 14:08:24 +0800 CST   29d       default            <none>
    You can find the backup files on the MinIO server.

Step 4: Deploy Velero in the Container Service for Kubernetes cluster and restore the WordPress application

  1. Deploy Velero in the Container Service for Kubernetes cluster.
    The method for deploying Velero in the Container Service for Kubernetes cluster is similar to that in the self-built Kubernetes cluster. For more information, see Deploy Velero in the self-built Kubernetes cluster.
    After you deploy Velero in the Container Service for Kubernetes cluster, run the following command to check the status of backups:
    $ velero backup get
    NAME                          STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
    wordpress-backup-with-pv      Completed   2019-12-12 14:23:40 +0800 CST   29d       default            <none>
    wordpress-backup-without-pv   Completed   2019-12-12 14:08:24 +0800 CST   29d       default            <none>
  2. Restore the WordPress application to the Container Service for Kubernetes cluster.
    In the self-built Kubernetes cluster, the WordPress application uses NFS PVs. Accordingly, you can configure Apsara File Storage NAS volumes for the WordPress application in the Container Service for Kubernetes cluster. In this example, you create an NFS StorageClass for the WordPress application but use Alibaba Cloud SSD disks as back-end storage.

    In this example, the Container Service for Kubernetes cluster uses the Container Storage Interface (CSI) plug-in. For more information, see Use dynamic disk volumes.

    1. Create a StorageClass.
      If you restore the WordPress application without PVs, skip this step.
      $ cat nfs.yaml
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
         name: nfs
      provisioner: diskplugin.csi.alibabacloud.com
      parameters:
          type: cloud_ssd
      reclaimPolicy: Retain
      
      $ kubectl apply -f nfs.yaml
      storageclass.storage.k8s.io/nfs created
    2. Restore the WordPress application.
      $ velero restore create --from-backup wordpress-backup-with-pv
      $ velero restore get
      NAME                                      BACKUP                     STATUS       WARNINGS   ERRORS   CREATED                         SELECTOR
      wordpress-backup-with-pv-20191212152745   wordpress-backup-with-pv   InProgress   0          0        2019-12-12 15:27:45 +0800 CST   <none>
      $ velero restore get
      If you check the running status of the WordPress application now, errors are returned, indicating that the container images cannot be pulled.
      $ kubectl -n wordpress get po
      NAME                         READY   STATUS         RESTARTS   AGE
      mysql-669b4666cd-trsnz       0/1     ErrImagePull   0          19m
      mysql-74dddbdcc8-h2tls       0/1     Init:0/1       0          19m
      wordpress-7cf5849f47-mbvx4   0/1     Init:0/1       0          19m
      wordpress-bb5d74d95-xcjxw    0/1     ErrImagePull   0          19m
    3. Set the image fields in the wordpress and mysql deployments to the URLs of the container images migrated to Container Registry. For more information, see Step 1: Migrate container images to Container Registry.
      # Modify the image URLs in the deployments.
      $ kubectl -n wordpress edit deployment mysql
      $ kubectl -n wordpress edit deployment wordpress

Step 5: Check the running status of the WordPress application

Run the following command to check the running status of the WordPress application:
$ kubectl -n wordpress get po
NAME                         READY   STATUS    RESTARTS   AGE
mysql-678b5d8499-vckfd       1/1     Running   0          100s
wordpress-8566f5f7d8-7shk6   1/1     Running   0          3m18s

In the test environment, bind the domain name of the WordPress application to the IP address of the node where the application is deployed in the hosts file of your operating system. Then, access the WordPress application through http://wordpress.myk8s.paas.com:31570/. Make sure that the WordPress application is accessible.