The csi-compatible-controller component lets you install and use Container Storage Interface (CSI) components in a cluster that already uses FlexVolume, which enables both storage types to coexist. After you install this component, you can sequentially switch the storage type for your applications to gradually migrate from FlexVolume to CSI. This topic describes how to use the csi-compatible-controller component to perform the migration.
Index
Prepare the environment
Install the csi-compatible-controller component
Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, find the one you want to manage and click its name. In the navigation pane on the left, click Add-ons.
In the Storage tab of the Component Management page, find the csi-compatible-controller component, click Install, and then click Confirm.
Install the CSI component
After you install the csi-compatible-controller, use the following YAML files to install the Container Storage Interface (CSI). Install csi-plugin and csi-provisioner in sequence. You cannot install CSI from the Add-ons page in the ACK console because the CSI configurations vary based on the Kubernetes version of the ACK cluster. Use the kubectl apply command to apply the YAML files that correspond to your cluster's Kubernetes version.
Note
Replace the image region ID in the YAML file. For example, replace
cn-hangzhouwith the region ID of your cluster.In the following YAML file, replace
{{.ClusterID}}with your cluster ID.
ACK clusters that run Kubernetes 1.20 or later
Click to view the csi-plugin YAML file
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: kube-system
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["csi-ossfs-credentials"]
verbs: ["get", "patch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch", "create", "delete" ]
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["csi-local-plugin-cert"]
verbs: ["get"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
- apiGroups: [""]
resources: ["endpoints"]
resourceNames: ["cnfs-cache-ds-service"]
verbs: ["get"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["csi-plugin", "ack-cluster-profile"]
verbs: ["get"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["storage-monitor-service"]
verbs: ["get"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "update", "patch", "list", "watch"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch", "create"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update", "create"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "list", "watch", "delete", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["rules"]
verbs: ["get"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["containernetworkfilesystems"]
verbs: ["get","list", "watch"]
- apiGroups: [""]
resources: ["nodes/stats"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: alicloud-csi-plugin
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: alicloud-csi-plugin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: diskplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: nasplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: ossplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-plugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-plugin
template:
metadata:
labels:
app: csi-plugin
spec:
tolerations:
- operator: Exists
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
nodeSelector:
kubernetes.io/os: linux
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
hostIPC: true
hostPID: true
dnsPolicy: ClusterFirst
containers:
- name: disk-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
- name: registration-dir
mountPath: /registration
- name: nas-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: oss-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: csi-plugin
securityContext:
privileged: true
allowPrivilegeEscalation: true
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-plugin:v1.26.8-e724570-aliyun
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=oss,nas,disk"
env:
- name: DEFAULT_REGISTRY
value: registry-vpc.cn-shenzhen.aliyuncs.com
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: SERVICE_TYPE
value: "plugin"
- name: MAX_VOLUMES_PERNODE
value: "15"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 1024Mi
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
ports:
- name: healthz
containerPort: 11260
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
mountPropagation: "Bidirectional"
- name: etc
mountPath: /host/etc
- name: host-log
mountPath: /var/log/
- name: ossconnectordir
mountPath: /host/usr/
- name: container-dir
mountPath: /var/lib/container
mountPropagation: "Bidirectional"
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- mountPath: /var/addon
name: addon-token
readOnly: true
- mountPath: /host/var/run/ossfs
name: ossfs-metrics-dir
- mountPath: /host/var/run/efc
name: efc-metrics-dir
- mountPath: /etc/csi-plugin/config
name: csi-plugin-cm
- name: host-mnt
mountPath: /mnt
mountPropagation: "Bidirectional"
- mountPath: /run/kata-containers/shared/direct-volumes
name: kata-direct-volumes
volumes:
- name: ossfs-metrics-dir
hostPath:
path: /var/run/ossfs
type: DirectoryOrCreate
- name: efc-metrics-dir
hostPath:
path: /var/run/efc
type: DirectoryOrCreate
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry
type: DirectoryOrCreate
- name: container-dir
hostPath:
path: /var/lib/container
type: DirectoryOrCreate
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
- name: ossconnectordir
hostPath:
path: /usr/
- name: host-mnt
hostPath:
path: /mnt
type: DirectoryOrCreate
- name: csi-plugin-cm
configMap:
name: csi-plugin
optional: true
- name: kata-direct-volumes
hostPath:
path: /run/kata-containers/shared/direct-volumes
type: DirectoryOrCreate
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
updateStrategy:
rollingUpdate:
maxUnavailable: 20%
type: RollingUpdate
Click to view the csi-provisioner YAML file
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology-alltype
parameters:
type: cloud_essd,cloud_ssd,cloud_efficiency
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-available
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_essd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-ssd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-efficiency
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_efficiency
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-provisioner
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-provisioner
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
replicas: 2
template:
metadata:
labels:
app: csi-provisioner
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-provisioner
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/master
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/control-plane
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
serviceAccount: csi-admin
hostPID: true
priorityClassName: system-node-critical
containers:
- name: external-disk-provisioner
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--csi-address=$(ADDRESS)"
- "--feature-gates=Topology=True"
- "--volume-name-prefix=disk"
- "--strict-topology=true"
- "--timeout=150s"
- "--leader-election=true"
- "--retry-interval-start=500ms"
- "--extra-create-metadata=true"
- "--default-fstype=ext4"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-attacher
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-attacher:v3.3-72dd428b-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-resizer
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 8Gi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-nas-provisioner
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--csi-address=$(ADDRESS)"
- "--volume-name-prefix=nas"
- "--timeout=150s"
- "--leader-election=true"
- "--retry-interval-start=500ms"
- "--default-fstype=nfs"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-nas-resizer
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 8Gi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-oss-provisioner
args:
- --csi-address=$(ADDRESS)
- --volume-name-prefix=oss
- --timeout=150s
- --leader-election=true
- --retry-interval-start=500ms
- --default-fstype=ossfs
- --v=5
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com/csi.sock
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 10m
memory: 16Mi
volumeMounts:
- mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
name: oss-provisioner-dir
- name: external-csi-snapshotter
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-snapshotter:v4.0.0-a230d5b3-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: /csi/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /csi
- name: external-snapshot-controller
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/snapshot-controller:v4.0.0-a230d5b3-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--leader-election=true"
- name: csi-provisioner
securityContext:
privileged: true
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-plugin:v1.26.8-e724570-aliyun
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=nas,disk,oss"
env:
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "provisioner"
- name: "CLUSTER_ID"
value: "{{.ClusterID}}"
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 5
periodSeconds: 20
ports:
- name: healthz
containerPort: 11270
volumeMounts:
- name: host-log
mountPath: /var/log/
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: oss-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
- mountPath: /var/addon
name: addon-token
readOnly: true
- mountPath: /mnt
mountPropagation: Bidirectional
name: host-mnt
- mountPath: /host/etc
name: etc
resources:
limits:
cpu: 500m
memory: 1024Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: disk-provisioner-dir
emptyDir: {}
- name: nas-provisioner-dir
emptyDir: {}
- name: oss-provisioner-dir
emptyDir: {}
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
type: ""
- name: host-mnt
hostPath:
path: /mnt
type: ""
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
ACK clusters that run Kubernetes versions earlier than 1.20
Click to view the csi-plugin YAML file
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-admin
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "create", "list"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "watch", "delete", "get", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "list", "watch", "delete", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods","pods/exec"]
verbs: ["create", "delete", "get", "post", "list", "watch", "patch", "udpate"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["rules"]
verbs: ["get"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["containernetworkfilesystems"]
verbs: ["get","list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: alicloud-csi-plugin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: diskplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: nasplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: ossplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-plugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-plugin
template:
metadata:
labels:
app: csi-plugin
spec:
tolerations:
- operator: Exists
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
nodeSelector:
beta.kubernetes.io/os: linux
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
hostPID: true
containers:
- name: disk-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
- name: registration-dir
mountPath: /registration
- name: nas-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: oss-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: csi-plugin
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.18.8.51-c504ef45-aliyun
imagePullPolicy: "Always"
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=oss,nas,disk"
env:
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "plugin"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 1024Mi
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
ports:
- name: healthz
containerPort: 11260
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
mountPropagation: "Bidirectional"
- name: etc
mountPath: /host/etc
- name: host-log
mountPath: /var/log/
- name: ossconnectordir
mountPath: /host/usr/
- name: container-dir
mountPath: /var/lib/container
mountPropagation: "Bidirectional"
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- mountPath: /var/addon
name: addon-token
readOnly: true
volumes:
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry
type: DirectoryOrCreate
- name: container-dir
hostPath:
path: /var/lib/container
type: DirectoryOrCreate
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
- name: ossconnectordir
hostPath:
path: /usr/
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
updateStrategy:
rollingUpdate:
maxUnavailable: 10%
type: RollingUpdateClick to view the csi-provisioner YAML file
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology-alltype
parameters:
type: cloud_essd,cloud_ssd,cloud_efficiency
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-available
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_essd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-ssd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-efficiency
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_efficiency
reclaimPolicy: Delete
allowVolumeExpansion: true
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-provisioner
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-provisioner
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
replicas: 2
template:
metadata:
labels:
app: csi-provisioner
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-provisioner
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/master
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
containers:
- name: external-disk-provisioner
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-cbd508573-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--provisioner=diskplugin.csi.alibabacloud.com"
- "--csi-address=$(ADDRESS)"
- "--feature-gates=Topology=True"
- "--volume-name-prefix=disk"
- "--strict-topology=true"
- "--timeout=150s"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--retry-interval-start=500ms"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-attacher
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-attacher:v2.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-resizer
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v1.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-nas-provisioner
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-cbd508573-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--provisioner=nasplugin.csi.alibabacloud.com"
- "--csi-address=$(ADDRESS)"
- "--volume-name-prefix=nas"
- "--timeout=150s"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--retry-interval-start=500ms"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-nas-resizer
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v1.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-csi-snapshotter
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-snapshotter:v4.0.0-1f9e7a7f8-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: /csi/csi.sock
imagePullPolicy: Always
volumeMounts:
- name: disk-provisioner-dir
mountPath: /csi
- name: external-snapshot-controller
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/snapshot-controller:v4.0.0-41f9e7a7f8-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--leader-election=true"
imagePullPolicy: Always
- name: csi-provisioner
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
imagePullPolicy: "Always"
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=nas,disk"
env:
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "provisioner"
- name: "CLUSTER_ID"
value: "{{.ClusterID}}"
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 5
periodSeconds: 20
ports:
- name: healthz
containerPort: 11270
volumeMounts:
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- name: host-log
mountPath: /var/log/
- name: etc
mountPath: /host/etc
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- mountPath: /var/addon
name: addon-token
readOnly: true
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: disk-provisioner-dir
emptyDir: {}
- name: nas-provisioner-dir
emptyDir: {}
- name: host-log
hostPath:
path: /var/log/
- name: host-dev
hostPath:
path: /dev
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
- name: etc
hostPath:
path: /etcVerify the CSI component
Create a StatefulSet and check that the pod status is running. This verifies that the Container Storage Interface (CSI) component is successfully deployed and the migration environment is ready.
Click to view the StatefulSet YAML file
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-csi-tr
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx"
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- name: disk-csi-tr
mountPath: /data
volumeClaimTemplates:
- metadata:
name: disk-csi-tr
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "csi-alicloud-disk-topology-alltype"
resources:
requests:
storage: 20Gi
After the components are deployed, you can use both FlexVolume and CSI components in the cluster. Then, you can gradually migrate applications from FlexVolume to CSI.
Switch the storage type used by applications
Migrate all FlexVolume storage used by applications to CSI. After all storage is migrated, you can delete the FlexVolume component.
Step 1: Convert PVCs and PVs to the CSI type
Find all persistent volume claims (PVCs) and persistent volumes (PVs) that are managed by FlexVolume and convert them to be managed by CSI. You can use the Flexvolume2CSI CLI to batch convert multiple PVCs and PVs. For more information, see Use the FlexVolume2CSI CLI to batch convert YAML files.
The following example shows how to convert a FlexVolume PVC and PV to a CSI PVC and PV. You can modify the following CSI template as needed.
Click to view the original FlexVolume PVC and PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: pvc-disk
namespace: default
flexVolume:
driver: alicloud/disk
fsType: ext4
options:
VolumeId: d-bp1bnp9homa0tyv6****
persistentVolumeReclaimPolicy: Delete
storageClassName: alicloud-disk-available
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-disk
spec:
accessModes:
- ReadWriteOnce
storageClassName: alicloud-disk-available
resources:
requests:
storage: 20GiClick to view the converted CSI PVC and PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
storageClassName: "csi-alicloud-disk-topology-alltype"
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
csi:
driver: diskplugin.csi.alibabacloud.com
volumeHandle: d-bp1bnp9homa0tyv6****
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-disk
spec:
storageClassName: "csi-alicloud-disk-topology-alltype"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
volumeName: d-bp1bnp9homa0tyv6****Step 2: Switch the reclaim policy
To avoid volume loss, you need to check the PVCs and PVs managed by FlexVolume before the conversion. If the pv.Spec.persistentVolumeReclaimPolicy parameter is set to Delete, change its value to Retain.
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: pvc-disk
namespace: default
flexVolume:
driver: alicloud/disk
fsType: ext4
options:
VolumeId: d-bp1bnp9homa0tyv6****
persistentVolumeReclaimPolicy: **Retain**
storageClassName: alicloud-disk-available
volumeMode: FilesystemStep 3: Switch the storage type for a single application
Follow these steps to change the storage type for an application from FlexVolume to CSI.
Run the following command to scale the number of StatefulSet replicas to 0:
kubectl scale --replicas=0 sts/sts-testRun the following commands to delete the FlexVolume PVC and PV from the cluster:
kubectl delete pvc pvc-disk kubectl delete pv d-bp1bnp9homa0tyv6****Run the following command to create the CSI PVC and PV:
kuectl apply -f csi-pvc-pv.yamlRun the following command to scale out the StatefulSet again:
kubectl scale --replicas=3 sts/sts-testNoteYou can use the same method to update the `pvcClaimName` in your deployments.
You can repeat these steps to change the volume type for other applications in the cluster. After you migrate all applications, you can uninstall FlexVolume. For more information, see Migrate FlexVolume without a storage cluster to CSI.
Manage components
Upgrade the component
After you install the CSI component in a FlexVolume cluster by following the preceding steps, you can obtain the updated YAML file from the text above and apply it to your cluster using the kubectl apply command.
To upgrade components on the Add-ons page in the Container Service console, you must first migrate all FlexVolume PVs and PVCs to CSI. You cannot upgrade components on the Add-ons page if both FlexVolume and CSI are used in the cluster. For more information about the migration, see Migrate FlexVolume without a storage cluster to CSI.
Delete the component
After all storage in the cluster is migrated from FlexVolume to CSI, you can delete the csi-compatible-controller component if the following conditions are met:
The FlexVolume component is deleted from the cluster.
All PVs of the FlexVolume type are deleted.
Ensure that the
--enable-controller-attach-detachparameter of the Kubelet is set totrue. For more information, see Step 3 and Step 4 in Migrate FlexVolumes without a storage cluster to CSI.
References
For more information about how to migrate a FlexVolume cluster that stores no data, see Migrate FlexVolume to CSI for clusters that store no data.