Container Service for Kubernetes:Modify resource configurations during application restoration

How to modify resource configurations during application restoration

Default modifications

When the backup center component restores an application, it automatically modifies resource configurations. Typically, the component modifies the following resource configurations:

• The component modifies temporary information, such as the UIDs of resources.

• The component changes the volume plug-in from FlexVolume to Container Storage Interface (CSI).

• The component updates APIs.

• If the restoration is performed across clouds, the component modifies the relevant settings to ensure compatibility.

Common modifications

You can manually modify the parameters of a restore task to modify specific resource configurations. Typically, you can modify the following resource configurations:

• You can change the namespace to which the resources belong.

• You can change the StorageClass and image address used by the application.

• You can overwrite the annotations of Services and Ingresses to enable compatibility with the network plug-in.

Generic modifiers

You can use resource modifiers provided by Velero to modify resource configurations on demand. To do this, you need to create a ConfigMap to specify the parameters that you want to modify. This allows you to perform the add, delete, and replace operations by using JSON patches.

Prerequisites

• migrate-controller 1.8.1 or later is installed in both the backup cluster and restore cluster. For more information, see Install migrate-controller and grant permissions.

• A backup vault is created in the backup cluster. For more information, see Create a backup vault. The following table describes the basic information of the backup vault.

  Note

  If the backup cluster, restore cluster, and backup vault reside in different regions, public network access must be enabled for both backup and restore clusters.

  Backup vault name	OSS bucket name	OSS bucket subdirectory	OSS bucket region	Visible scope
  vault	cnfs-oss-test	subpath	cn-beijing	<No impact on queries>

• If you want to use Velero resource modifiers, make sure that the cluster runs Kubernetes 1.20 or later and uses the CSI plug-in.

Example

This section describes how to back up and restore an NGINX application. After the application is backed up, a restore task is created to restore the application and change the namespace to which the new application belongs and the image repository address. This section also describes how to use resource modifiers to modify the mount path of the volume used by the application and delete node affinity rules.

The NGINX application runs as a StatefulSet deployed in the default namespace of the backup cluster and uses an image from the OpenAnolis community. The image address is anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis. The volumeClaimTemplates parameter is added to the application configuration to mount a persistent volume claim (PVC) that uses the alicloud-disk-topology-alltype StorageClass. This way, a disk volume is automatically provisioned for the application.

The following YAML template is an example.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
  namespace: default
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  serviceName: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - preference:
              matchExpressions:
              - key: node-role.kubernetes.io/control-plane
                operator: Exists
            weight: 1
      containers:
      - image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/nginx/html/
          name: www
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      labels:
        app: nginx
      name: www
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
      storageClassName: alicloud-disk-topology-alltype
      volumeMode: Filesystem

Run the following command to check whether the application runs as expected and whether a volume is mounted to the application:

kubectl get pod && kubectl get pvc

Expected output:

NAME    READY   STATUS    RESTARTS   AGE
web-0   1/1     Running   0          39s
web-1   1/1     Running   0          30s
NAME        STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS                     VOLUMEATTRIBUTESCLASS   AGE
www-web-0   Bound    d-2zefofsg0uzoiaxxxxxx   20Gi       RWO            alicloud-disk-topology-alltype   <unset>                 3m18s
www-web-1   Bound    d-2zedgdlghw2oudxxxxxx   20Gi       RWO            alicloud-disk-topology-alltype   <unset>                 30s

Step 1: Back up the application in the backup cluster

You can create a backup task in the ACK console or by using kubectl.

(Optional) Step 2: View the JSON file of the resource backups in the OSS console

The JSON file contains information about all the resources that are backed up, including the names, types, sizes, and creation dates of the resources. You can view the JSON file to check whether the resource that you want to restore is contained in the file. The following operations show how to view the JSON file generated by the backup task in the backup vault created in Prerequisites.

1. Log on to the OSS console.

2. In the left-side navigation pane, click Buckets. On the Buckets page, click the cnfs-oss-test bucket.

3. In the left-side navigation pane, choose Object Management > Objects.

   In the /subpath/backups/Instant backup task name>/ path, find and download the <Instant backup task name>.tar.gz package. Then, decompress the package to view the information in the file.

Step 3: Configure custom resource modification policies in the restore cluster

A custom resource modification policy has two parameters: conditions and patches. The conditions parameter specifies the rules used to select the resources to be modified. The patches parameter specifies the fields to be modified and how the fields are modified. The following ConfigMap is an example. The ConfigMap deletes node affinity rules and changes the volume mount path to /data for the StatefulSets and pods that meet the conditions.

apiVersion: v1
data:
  modifier: |
    version: v1
    resourceModifierRules:
    - conditions:
        groupResource: statefulsets.apps
        namespaces:
        - default
        labelSelector:
          matchLabels:
            app: nginx
      patches:
      - operation: remove
        path: "/spec/template/spec/affinity/nodeAffinity"
      - operation: replace
        path: "/spec/template/spec/containers/0/volumeMounts/0/mountPath"
        value: "/data"
    - conditions:
        groupResource: pods
        resourceNameRegex: "^web.*$"
        namespaces:
        - default
        labelSelector:
          matchLabels:
            app: nginx
      patches:
      - operation: remove
        path: "/spec/affinity"
      - operation: replace
        path: "/spec/containers/0/volumeMounts/0/mountPath"
        value: "/data"
kind: ConfigMap
metadata:
  name: <backupName>-resource-modifier
  namespace: csdr

The following table describes the fields in the conditions parameter.

matchLabels:
  app: nginx 

The following table describes the fields in the patches parameter:

(Optional) Step 4: Initialize the backup vault

Use the following backuplocation.yaml example to initialize the backup vault in the restore cluster. Skip this step if already initialized.

apiVersion: csdr.alibabacloud.com/v1beta1
kind: BackupLocation
metadata:
  name: <backup-vault-name>
  namespace: csdr
spec:
  backupSyncPeriod: 0s
  # The following configuration must match the backup vault initial setup. You can also initialize via the console.
  config:
    network: internal
    region: <oss-bucket-region>
  objectStorage:
    bucket: <oss-bucket-name>
    prefix: <oss-bucket-subpath>
  provider: alibabacloud

Step 5: Restore the modified application in the restore cluster

Create a restore task in the restore cluster to deploy the modified application. The restore task references the custom resource modification policy you configured in the previous step. The restore task uses the namespaceMapping and imageRegistryMapping parameters to enable namespace mapping and image repository mapping in the Map format.

1. Create a file named applicationrestore.yaml and copy the following code block to the file.

   The file is used to create a restore task that restores the resources in the default1 namespace (including volumes) to the default1 namespace. The restore task changes the image address used by the application, which is provided by the OpenAnolis community, to an image address provided by Container Registry. Make sure that the image is synchronized before the image address is changed.

   apiVersion: csdr.alibabacloud.com/v1beta1
   kind: ApplicationRestore
   metadata:
     name: <restoreName>
     namespace: csdr
   spec:
     backupName: <backupName>
     namespaceMapping:
       default: default1   # The default1 namespace to which the backup file is restored. 
     imageRegistryMapping:
       anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis: registry.cn-beijing.aliyuncs.com/<acrRegistry>  # Make sure that the image is synchronized from the original image repository to the new image repository before the image address is changed. 
     resourceModifier:
       kind: ConfigMap
       name: <backupName>-resource-modifier

2. Run the following command in the restore cluster to deploy the applicationrestore object to create a restore task:

   kubectl apply -f applicationrestore.yaml

3. Run the following command to query the restore task:

   Note

   You can also view the restore task in the ACK console. For more information, see Step 3: Restore applications and volumes.

   kubectl -n csdr get applicationrestore <restoreName> --watch

   Expected output:

   NAME                                    PHASE       AGE
   <restoreName>                           Completed   18h

Step 6: Check whether the application is restored

Run the following command to query the restored StatefulSet:

kubectl -n default1 get sts web -oyaml

Expected output:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"web","namespace":"default"},"spec":{"replicas":2,"selector":{"matchLabels":{"app":"nginx"}},"serviceName":"nginx","template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"affinity":{"nodeAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"preference":{"matchExpressions":[{"key":"node-role.kubernetes.io/master","operator":"Exists"}]},"weight":1}]}},"containers":[{"image":"anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.2","imagePullPolicy":"IfNotPresent","name":"nginx","ports":[{"containerPort":80,"name":"web","protocol":"TCP"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/usr/share/nginx/html/","name":"www"}]}]}},"volumeClaimTemplates":[{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"labels":{"app":"nginx"},"name":"www"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"20Gi"}},"storageClassName":"alicloud-disk-topology-alltype","volumeMode":"Filesystem"}}]}}
  creationTimestamp: "2024-10-09T08:32:50Z"
  generation: 1
  labels:
    app: nginx
    velero.io/backup-name: <Instant backup task name>
    velero.io/restore-name: <Restore task name>
  name: web
  namespace: default1
  resourceVersion: "119622"
  uid: d23878ea-0b9f-40ba-b61b-1ff6bb77eb43
spec:
  persistentVolumeClaimRetentionPolicy:
    whenDeleted: Retain
    whenScaled: Retain
  podManagementPolicy: OrderedReady
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nginx
  serviceName: nginx
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      affinity: {}
      containers:
      - image: registry.cn-beijing.aliyuncs.com/<acrRegistry>/nginx:1.14.1-8.6
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
          name: web
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /data
          name: www
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
      name: www
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
      storageClassName: alicloud-disk-topology-alltype
      volumeMode: Filesystem
    status:
      phase: Pending
status:
  availableReplicas: 0
  collisionCount: 0
  currentRevision: web-7b454646b4
  observedGeneration: 1
  replicas: 2
  updateRevision: web-7b454646b4

The output indicates that the image address, volume mount path, and namespace are modified. In addition, the node affinity rules are deleted.