Apsara File Storage NAS:Mount a NAS file system on a self-managed Kubernetes cluster

Prerequisites

• A registered cluster is created and an external cluster is connected to the registered cluster. For more information, see Create a registered cluster in the ACK console and Use onectl to create a registered cluster.

• Elastic Compute Service (ECS) instances are added to the external cluster. For more information, see Create and scale out a node pool .

  Important

  The CSI plug-in provided by Alibaba Cloud can be deployed only on ECS instances. Therefore, you must add the alibabacloud.com/external=true label to the ECS instances in the external cluster.

• A kubectl client is connected to the external cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Precautions

• If the external cluster is deployed on Alibaba Cloud and ECS instances are added to the cluster, you must add labels to the ECS instances. For more information about how to add labels to ECS instances, see Add labels to ECS instances in an external cluster that is registered with ACK.

• If you use the node pool feature to add ECS instances to the registered external cluster, the ECS instances have the alibabacloud.com/external=true label by default.

Step 1: Grant a Resource Access Management (RAM) user the permissions to manage the CSI plug-in

Step 2: Install the CSI plug-in

onectl addon install csi-plugin
onectl addon install csi-provisioner

Addon csi-plugin, version **** installed.
Addon csi-provisioner, version **** installed.

Mount a NAS file system by using a built-in Kubernetes volume

You can use the following methods to mount a NAS file system to persist application data:

• Mount a statically provisioned NAS volume by using NFS

• Mount a dynamically provisioned NAS volume by using NFS

Prerequisites

• A Network File System (NFS) file system is created. For more information, see Create a file system.

  If you want to encrypt data in a NAS file system, configure the encryption settings when you create the NAS file system.

• A mount target is created. For more information, see Create a mount target.

• A self-managed Kubernetes cluster is created and the cluster version is 1.16 or later.

• Port 111 and port 2049 are specified in the security group rule of the pod scheduling node. For more information, see Add a security group rule.

• If the self-managed Kubernetes cluster is deployed in a data center, the data center must be connected to Alibaba Cloud. You can connect the data center to Alibaba Cloud by using Express Connect, Smart Access Gateway (SAG), or VPN Gateway. For more information, see the following topics:

  • What is Express Connect?

  • What is SAG?

  • What is VPN Gateway?

Limits

The limits on mounting scenarios, the number of file systems, and file sharing protocols vary with the type of the NAS file system. For more information, see Limits.

Method 1: Mount a NAS file system by using a built-in Kubernetes volume

1. Log on to the self-managed Kubernetes cluster.

2. Configure the nodes in the Kubernetes cluster.

   You must configure all nodes in the Kubernetes cluster. Before you add a node to the Kubernetes cluster, you must perform the following steps:

   1. Install an NFS client. For more information, see Step 1: Install an NFS client.

   2. Run the following command to configure the /etc/nfsmount.conf file:

      cat <<EOF >> /etc/nfsmount.conf
      [ Server "file-system-id.region.nas.aliyuncs.com" ]
      vers=3
      Proto=tcp
      Lock=False
      resvport=False
      rsize=1048576
      wsize=1048576
      hard=True
      timeo=600
      retrans=2
      EOF

      In the preceding command, file-system-id.region.nas.aliyuncs.com is the domain name of the mount target for the NAS file system. Replace the domain name with the actual value. To obtain the domain name of the mount target, perform the following steps: Log on to the NAS console. On the File System List page, click the file system ID. Click the Mount Targets tab. Then, move the pointer over the icon in the Mount Target column.

      Note

      • We recommend that you use only the NFSv3 protocol to mount a NAS file system.

      • All the parameters in the preceding command are required. You do not need to modify other parameters except the file-system-id.region.nas.aliyuncs.com and vers parameters. The vers parameter specifies the protocol version.

   3. Verify that the NAS file system is connected to the Kubernetes cluster.

      1. Run the following command to mount the NFS file system:

         mount -t nfs file-system-id.region.nas.aliyuncs.com:/ /mnt 

         In the preceding command, file-system-id.region.nas.aliyuncs.com is the domain name of the mount target for the NAS file system. Replace the domain name with the actual value.

         If the mount fails, ping the domain name of the mount target and check whether port 2049 is connected. If the mount failure is not caused by network issues, use the script tool to troubleshoot the issue. For more information, see Fix mount issues.

      2. Run the following command to view the mount information:

         mount | grep nfs | grep mnt 

         If the parameters include vers=3, hard, noresvport, and nolock, the network is connected.

         After you confirm that the network is connected, run the umount /mnt command to delete the mounted NFS file system.

3. Start a pod to mount the NFS file system by using a built-in Kubernetes volume.

   In this example, a pod named nginx is started to mount the NFS file system.

   1. Configure the nginx.yaml file.

      cat  << EOF > ./nginx.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: nginx-deployment
      spec:
        selector:
          matchLabels:
            app: nginx
        replicas: 1
        template:
          metadata:
            labels:
              app: nginx
          spec:
            containers:
            - name: nginx
              image: nginx:1.14.2
              ports:
              - containerPort: 80
              volumeMounts:
              - mountPath: /data
                name: test-nfs
            volumes:
            - name: test-nfs
              nfs:
                server: file-system-id.region.nas.aliyuncs.com    # The domain name of the mount target for the NAS file system. Replace the domain name with the actual value. Example: 7bexxxxxx-xxxx.ap-southeast-1.nas.aliyuncs.com. 
                path: /    # The directory path of the NAS file system. The directory must be an existing directory or the root directory. The root directory of a General-purpose NAS file system is "/". The root directory of an Extreme NAS file system is "/share". 
      EOF

   2. Enable the nginx pod to use the NFS file system.

      kubectl apply -f ./nginx.yaml

4. Check whether the pod is started as expected and whether the NFS file system is mounted.

   1. Run the following command to check the status of the pod:

      • Command

        kubectl get pods | grep nginx 

      • Output

        If the pod is in the Running state, the pod is started as expected.

   2. Run the following command to view the mount result:

      • Command

        kubectl exec nginx-deployment-85c45c8b76-h9vcr -ti stat /data 

      • Output

        If an output similar to the following example appears, the mount is successful.

Method 2: Mount a NAS file system by using a CSI plug-in

1. Deploy the CSI plug-in.

   1. Create a YAML file that is used to deploy CSI-ALL.

      Create a file named csi-all.yaml and copy the following template into the file. Replace the values of the ACCESS_KEY_ID and ACCESS_KEY_SECRET parameters with the AccessKey ID and AccessKey secret of your Alibaba Cloud account. For more information about how to obtain the AccessKey pair of your Alibaba Cloud account, see Create an AccessKey pair.

      ---
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: csi-admin
        namespace: kube-system
      ---
      kind: ClusterRole
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: alicloud-csi-plugin
      rules:
        - apiGroups: [""]
          resources: ["secrets"]
          verbs: ["get", "create", "list"]
        - apiGroups: [""]
          resources: ["persistentvolumes"]
          verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
        - apiGroups: [""]
          resources: ["persistentvolumeclaims"]
          verbs: ["get", "list", "watch", "update"]
        - apiGroups: [""]
          resources: ["persistentvolumeclaims/status"]
          verbs: ["get", "list", "watch", "update", "patch"]
        - apiGroups: ["storage.k8s.io"]
          resources: ["storageclasses"]
          verbs: ["get", "list", "watch"]
        - apiGroups: ["storage.k8s.io"]
          resources: ["csinodes"]
          verbs: ["get", "list", "watch"]
        - apiGroups: [""]
          resources: ["events"]
          verbs: ["get", "list", "watch", "create", "update", "patch"]
        - apiGroups: [""]
          resources: ["endpoints"]
          verbs: ["get", "watch", "list", "delete", "update", "create"]
        - apiGroups: [""]
          resources: ["configmaps"]
          verbs: ["get", "watch", "list", "delete", "update", "create"]
        - apiGroups: [""]
          resources: ["nodes"]
          verbs: ["get", "list", "watch", "update"]
        - apiGroups: ["csi.storage.k8s.io"]
          resources: ["csinodeinfos"]
          verbs: ["get", "list", "watch"]
        - apiGroups: ["storage.k8s.io"]
          resources: ["volumeattachments"]
          verbs: ["get", "list", "watch", "update", "patch"]
        - apiGroups: ["snapshot.storage.k8s.io"]
          resources: ["volumesnapshotclasses"]
          verbs: ["get", "list", "watch", "create"]
        - apiGroups: ["snapshot.storage.k8s.io"]
          resources: ["volumesnapshotcontents"]
          verbs: ["create", "get", "list", "watch", "update", "delete"]
        - apiGroups: ["snapshot.storage.k8s.io"]
          resources: ["volumesnapshots"]
          verbs: ["get", "list", "watch", "update", "create"]
        - apiGroups: ["apiextensions.k8s.io"]
          resources: ["customresourcedefinitions"]
          verbs: ["create", "list", "watch", "delete", "get", "update", "patch"]
        - apiGroups: ["coordination.k8s.io"]
          resources: ["leases"]
          verbs: ["get", "create", "list", "watch", "delete", "update"]
        - apiGroups: ["snapshot.storage.k8s.io"]
          resources: ["volumesnapshotcontents/status"]
          verbs: ["update"]
        - apiGroups: ["storage.k8s.io"]
          resources: ["volumeattachments/status"]
          verbs: ["patch"]
        - apiGroups: ["snapshot.storage.k8s.io"]
          resources: ["volumesnapshots/status"]
          verbs: ["update"]
        - apiGroups: ["storage.k8s.io"]
          resources: ["storageclasses"]
          verbs: ["get", "list", "watch"]
        - apiGroups: [""]
          resources: ["namespaces"]
          verbs: ["get", "list"]
        - apiGroups: [""]
          resources: ["pods","pods/exec"]
          verbs: ["create", "delete", "get", "post", "list", "watch", "patch", "udpate"]
        - apiGroups: ["storage.alibabacloud.com"]
          resources: ["rules"]
          verbs: ["get"]
        - apiGroups: ["storage.alibabacloud.com"]
          resources: ["containernetworkfilesystems"]
          verbs: ["get","list", "watch"]
      ---
      kind: ClusterRoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: alicloud-csi-plugin
      subjects:
        - kind: ServiceAccount
          name: csi-admin
          namespace: kube-system
      roleRef:
        kind: ClusterRole
        name: alicloud-csi-plugin
        apiGroup: rbac.authorization.k8s.io
      ---
      apiVersion: storage.k8s.io/v1
      kind: CSIDriver
      metadata:
        name: diskplugin.csi.alibabacloud.com
      spec:
        attachRequired: true
        podInfoOnMount: true
      ---
      apiVersion: storage.k8s.io/v1
      kind: CSIDriver
      metadata:
        name: nasplugin.csi.alibabacloud.com
      spec:
        attachRequired: false
        podInfoOnMount: true
      ---
      apiVersion: storage.k8s.io/v1
      kind: CSIDriver
      metadata:
        name: ossplugin.csi.alibabacloud.com
      spec:
        attachRequired: false
        podInfoOnMount: true
      ---
      kind: DaemonSet
      apiVersion: apps/v1
      metadata:
        name: csi-plugin
        namespace: kube-system
      spec:
        selector:
          matchLabels:
            app: csi-plugin
        template:
          metadata:
            labels:
              app: csi-plugin
          spec:
            tolerations:
              - operator: Exists
            affinity:
              nodeAffinity:
                requiredDuringSchedulingIgnoredDuringExecution:
                  nodeSelectorTerms:
                  - matchExpressions:
                    - key: type
                      operator: NotIn
                      values:
                      - virtual-kubelet
            nodeSelector:
              kubernetes.io/os: linux
            serviceAccount: csi-admin
            priorityClassName: system-node-critical
            hostNetwork: true
            hostPID: true
            dnsPolicy: ClusterFirst
            containers:
              - name: disk-driver-registrar
                image: registry.cn-beijing.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
                  - "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
                volumeMounts:
                  - name: kubelet-dir
                    mountPath: /var/lib/kubelet
                  - name: registration-dir
                    mountPath: /registration
              - name: nas-driver-registrar
                image: registry.cn-beijing.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
                  - "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
                volumeMounts:
                  - name: kubelet-dir
                    mountPath: /var/lib/kubelet/
                  - name: registration-dir
                    mountPath: /registration
              - name: oss-driver-registrar
                image: registry.cn-beijing.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
                  - "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
                volumeMounts:
                  - name: kubelet-dir
                    mountPath: /var/lib/kubelet/
                  - name: registration-dir
                    mountPath: /registration
              - name: csi-plugin
                securityContext:
                  privileged: true
                  allowPrivilegeEscalation: true
                image: registry.cn-beijing.aliyuncs.com/acs/csi-plugin:v1.24.5-39a3970-aliyun
                args:
                  - "--endpoint=$(CSI_ENDPOINT)"
                  - "--v=2"
                  - "--driver=oss,nas,disk"
                env:
                  - name: KUBE_NODE_NAME
                    valueFrom:
                      fieldRef:
                        apiVersion: v1
                        fieldPath: spec.nodeName
                  - name: CSI_ENDPOINT
                    value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
                  - name: MAX_VOLUMES_PERNODE
                    value: "15"
                  - name: SERVICE_TYPE
                    value: "plugin"
                  - name: ACCESS_KEY_ID
                    value: "xxx"
                  - name: ACCESS_KEY_SECRET
                    value: "xxx"
                  - name: KUBELET_ROOT_DIR
                    value: "/var/lib/kubelet"
                resources:
                  requests:
                    cpu: 100m
                    memory: 128Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                livenessProbe:
                  httpGet:
                    path: /healthz
                    port: healthz
                    scheme: HTTP
                  initialDelaySeconds: 10
                  periodSeconds: 30
                  timeoutSeconds: 5
                  failureThreshold: 5
                readinessProbe:
                  httpGet:
                    path: /healthz
                    port: healthz
                  initialDelaySeconds: 10
                  periodSeconds: 30
                  timeoutSeconds: 5
                  failureThreshold: 5
                ports:
                  - name: healthz
                    containerPort: 11260
                volumeMounts:
                  - name: kubelet-dir
                    mountPath: /var/lib/kubelet/
                    mountPropagation: "Bidirectional"
                  - name: etc
                    mountPath: /host/etc
                  - name: host-log
                    mountPath: /var/log/
                  - name: ossconnectordir
                    mountPath: /host/usr/
                  - name: container-dir
                    mountPath: /var/lib/container
                    mountPropagation: "Bidirectional"
                  - name: host-dev
                    mountPath: /dev
                    mountPropagation: "HostToContainer"
                    readOnly: true
                  - mountPath: /host/var/run/
                    name: fuse-metrics-dir
            volumes:
              - name: fuse-metrics-dir
                hostPath:
                  path: /var/run/
                  type: DirectoryOrCreate
              - name: registration-dir
                hostPath:
                  path: /var/lib/kubelet/plugins_registry
                  type: DirectoryOrCreate
              - name: container-dir
                hostPath:
                  path: /var/lib/container
                  type: DirectoryOrCreate
              - name: kubelet-dir
                hostPath:
                  path: /var/lib/kubelet
                  type: Directory
              - name: host-dev
                hostPath:
                  path: /dev
              - name: host-log
                hostPath:
                  path: /var/log/
              - name: etc
                hostPath:
                  path: /etc
              - name: ossconnectordir
                hostPath:
                  path: /usr/
        updateStrategy:
          rollingUpdate:
            maxUnavailable: 30%
          type: RollingUpdate
      
      ---
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
        name: alicloud-disk-topology-alltype
      parameters:
        type: cloud_essd,cloud_ssd,cloud_efficiency
      provisioner: diskplugin.csi.alibabacloud.com
      reclaimPolicy: Delete
      allowVolumeExpansion: true
      volumeBindingMode: WaitForFirstConsumer
      ---
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
         name: alicloud-disk-available
      provisioner: diskplugin.csi.alibabacloud.com
      parameters:
          type: available
      reclaimPolicy: Delete
      allowVolumeExpansion: true
      ---
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
         name: alicloud-disk-essd
      provisioner: diskplugin.csi.alibabacloud.com
      parameters:
          type: cloud_essd
      reclaimPolicy: Delete
      allowVolumeExpansion: true
      ---
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
         name: alicloud-disk-ssd
      provisioner: diskplugin.csi.alibabacloud.com
      parameters:
          type: cloud_ssd
      reclaimPolicy: Delete
      allowVolumeExpansion: true
      ---
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
         name: alicloud-disk-efficiency
      provisioner: diskplugin.csi.alibabacloud.com
      parameters:
          type: cloud_efficiency
      reclaimPolicy: Delete
      allowVolumeExpansion: true
      ---
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
         name: alicloud-disk-topology
      provisioner: diskplugin.csi.alibabacloud.com
      parameters:
          type: available
      reclaimPolicy: Delete
      volumeBindingMode: WaitForFirstConsumer
      allowVolumeExpansion: true
      ---
      kind: Deployment
      apiVersion: apps/v1
      metadata:
        name: csi-provisioner
        namespace: kube-system
      spec:
        selector:
          matchLabels:
            app: csi-provisioner
        strategy:
          rollingUpdate:
            maxSurge: 0
            maxUnavailable: 1
          type: RollingUpdate
        replicas: 2
        template:
          metadata:
            labels:
              app: csi-provisioner
          spec:
            affinity:
              nodeAffinity:
                preferredDuringSchedulingIgnoredDuringExecution:
                - weight: 1
                  preference:
                    matchExpressions:
                    - key: node-role.kubernetes.io/master
                      operator: Exists
                requiredDuringSchedulingIgnoredDuringExecution:
                  nodeSelectorTerms:
                  - matchExpressions:
                    - key: type
                      operator: NotIn
                      values:
                      - virtual-kubelet
              podAntiAffinity:
                preferredDuringSchedulingIgnoredDuringExecution:
                - weight: 100
                  podAffinityTerm:
                    labelSelector:
                      matchExpressions:
                      - key: app
                        operator: In
                        values:
                        - csi-provisioner
                    topologyKey: kubernetes.io/hostname
            tolerations:
            - effect: NoSchedule
              operator: Exists
              key: node-role.kubernetes.io/master
            - effect: NoSchedule
              operator: Exists
              key: node.cloudprovider.kubernetes.io/uninitialized
            serviceAccount: csi-admin
            hostPID: true
            priorityClassName: system-node-critical
            containers:
              - name: external-disk-provisioner
                image: registry.cn-beijing.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--csi-address=$(ADDRESS)"
                  - "--feature-gates=Topology=True"
                  - "--volume-name-prefix=disk"
                  - "--strict-topology=true"
                  - "--timeout=150s"
                  - "--leader-election=true"
                  - "--retry-interval-start=500ms"
                  - "--extra-create-metadata=true"
                  - "--default-fstype=ext4"
                  - "--v=5"
                env:
                  - name: ADDRESS
                    value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
                volumeMounts:
                  - name: disk-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
              - name: external-disk-attacher
                image: registry.cn-beijing.aliyuncs.com/acs/csi-attacher:v3.3-72dd428b-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=$(ADDRESS)"
                  - "--leader-election=true"
                env:
                  - name: ADDRESS
                    value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
                volumeMounts:
                  - name: disk-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
              - name: external-disk-resizer
                image: registry.cn-beijing.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=$(ADDRESS)"
                  - "--leader-election"
                env:
                  - name: ADDRESS
                    value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
                volumeMounts:
                  - name: disk-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
              - name: external-nas-provisioner
                image: registry.cn-beijing.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--csi-address=$(ADDRESS)"
                  - "--volume-name-prefix=nas"
                  - "--timeout=150s"
                  - "--leader-election=true"
                  - "--retry-interval-start=500ms"
                  - "--default-fstype=nfs"
                  - "--v=5"
                env:
                  - name: ADDRESS
                    value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
                volumeMounts:
                  - name: nas-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
              - name: external-nas-resizer
                image: registry.cn-beijing.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=$(ADDRESS)"
                  - "--leader-election"
                env:
                  - name: ADDRESS
                    value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
                volumeMounts:
                  - name: nas-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
              - name: external-oss-provisioner
                args:
                  - --csi-address=$(ADDRESS)
                  - --volume-name-prefix=oss
                  - --timeout=150s
                  - --leader-election=true
                  - --retry-interval-start=500ms
                  - --default-fstype=ossfs
                  - --v=5
                env:
                - name: ADDRESS
                  value: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com/csi.sock
                image: registry.cn-beijing.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun
                resources:
                  limits:
                    cpu: 500m
                    memory: 1Gi
                  requests:
                    cpu: 10m
                    memory: 16Mi
                volumeMounts:
                - mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
                  name: oss-provisioner-dir
              - name: external-csi-snapshotter
                image: registry.cn-beijing.aliyuncs.com/acs/csi-snapshotter:v4.0.0-a230d5b3-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--csi-address=$(ADDRESS)"
                  - "--leader-election=true"
                  - "--extra-create-metadata=true"
                env:
                  - name: ADDRESS
                    value: /csi/csi.sock
                volumeMounts:
                  - name: disk-provisioner-dir
                    mountPath: /csi
              - name: external-snapshot-controller
                image: registry.cn-beijing.aliyuncs.com/acs/snapshot-controller:v4.0.0-a230d5b3-aliyun
                resources:
                  requests:
                    cpu: 10m
                    memory: 16Mi
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                args:
                  - "--v=5"
                  - "--leader-election=true"
              - name: csi-provisioner
                securityContext:
                  privileged: true
                image: registry.cn-beijing.aliyuncs.com/acs/csi-plugin:v1.24.5-39a3970-aliyun
                args:
                  - "--endpoint=$(CSI_ENDPOINT)"
                  - "--v=2"
                  - "--driver=nas,disk,oss"
                env:
                  - name: CSI_ENDPOINT
                    value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
                  - name: MAX_VOLUMES_PERNODE
                    value: "15"
                  - name: SERVICE_TYPE
                    value: "provisioner"
                  - name: "CLUSTER_ID"
                    value: "abc"
                  - name: "ACCESS_KEY_ID"
                    value: "xxx"
                  - name: "ACCESS_KEY_SECRET"
                    value: "xxx"
      
                livenessProbe:
                  httpGet:
                    path: /healthz
                    port: healthz
                    scheme: HTTP
                  initialDelaySeconds: 10
                  periodSeconds: 30
                  timeoutSeconds: 5
                  failureThreshold: 5
                readinessProbe:
                  httpGet:
                    path: /healthz
                    port: healthz
                  initialDelaySeconds: 5
                  periodSeconds: 20
                ports:
                  - name: healthz
                    containerPort: 11270
                volumeMounts:
                  - name: host-log
                    mountPath: /var/log/
                  - name: disk-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
                  - name: nas-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
                  - name: oss-provisioner-dir
                    mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
                  - mountPath: /var/addon
                    name: addon-token
                    readOnly: true
                  - mountPath: /mnt
                    mountPropagation: Bidirectional
                    name: host-dev
                  - mountPath: /host/etc
                    name: etc
                resources:
                  limits:
                    cpu: 500m
                    memory: 1024Mi
                  requests:
                    cpu: 100m
                    memory: 128Mi
            volumes:
              - name: disk-provisioner-dir
                emptyDir: {}
              - name: nas-provisioner-dir
                emptyDir: {}
              - name: oss-provisioner-dir
                emptyDir: {}
              - name: host-log
                hostPath:
                  path: /var/log/
              - name: etc
                hostPath:
                  path: /etc
                  type: ""
              - name: host-dev
                hostPath:
                  path: /mnt
                  type: ""
              - name: addon-token
                secret:
                  defaultMode: 420
                  optional: true
                  items:
                  - key: addon.token.config
                    path: token-config
                  secretName: addon.csi.token

   2. Deploy CSI-ALL.

      kubectl apply -f csi-all.yaml

   3. Check the deployment result.

      kubectl -n kube-system get pods

      Expected command output:

      NAME                                    READY   STATUS    RESTARTS   AGE
      csi-plugin-5h6gq                       4/4     Running   0          30m
      csi-plugin-gjjp7                       4/4     Running   0          31m
      csi-provisioner-cfc45d84c-lzpvm        9/9     Running   0          31m
      csi-provisioner-cfc45d84c-zczfn        9/9     Running   0          31m
      ......

2. Mount the file system.

   • For more information about how to mount a statically provisioned NAS volume, see Mount a statically provisioned NAS volume by using NFS.

   • For more information about how to mount a dynamically provisioned NAS volume, see Mount a dynamically provisioned NAS volume by using NFS.