csi-compatible-controller组件支持在Flexvolume类型的集群中安装和使用CSI组件，让Flexvolume组件和CSI组件共存于一个集群内，通过安装该组件并依次切换Flexvolume集群中应用挂载的存储类型，使Flexvolume类型的存储逐渐过渡到CSI存储。本文介绍如何通过csi-compatible-controller组件完成Flexvolume至CSI的迁移。
索引
环境准备
安装csi-compatible-controller组件
登录容器服务管理控制台，在左侧导航栏选择集群。
在集群列表页面，单击目标集群名称，然后在左侧导航栏，选择运维管理 > 组件管理。
在组件管理页面，单击存储页签，找到csi-compatible-controller组件，单击卡片右下角的安装，然后单击确定。
安装CSI组件
csi-compatible-controller组件安装完成后，您需要使用如下YAML文件安装部署CSI组件。由于配置不同，不能通过集群的组件管理页面进行CSI组件的安装。安装时，请根据集群版本选择如下组件对应的YAML文件进行安装。
说明
替换YAML文件中镜像的地域ID，例如将
cn-hangzhou替换为您集群所在地域。将以下YAML文件中的
{{.ClusterID}}替换为您的集群ID。
集群版本为ACK 1.20及以上
展开查看csi-provisioner的YAML文件
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology-alltype
parameters:
type: cloud_essd,cloud_ssd,cloud_efficiency
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-available
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_essd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-ssd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-efficiency
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_efficiency
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-provisioner
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-provisioner
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
replicas: 2
template:
metadata:
labels:
app: csi-provisioner
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: Exists
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-provisioner
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/master
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/control-plane
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
serviceAccount: csi-admin
hostPID: true
priorityClassName: system-node-critical
containers:
- name: external-disk-provisioner
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--csi-address=$(ADDRESS)"
- "--feature-gates=Topology=True"
- "--volume-name-prefix=disk"
- "--strict-topology=true"
- "--timeout=150s"
- "--leader-election=true"
- "--retry-interval-start=500ms"
- "--extra-create-metadata=true"
- "--default-fstype=ext4"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-attacher
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-attacher:v3.3-72dd428b-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-resizer
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 8Gi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-nas-provisioner
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--csi-address=$(ADDRESS)"
- "--volume-name-prefix=nas"
- "--timeout=150s"
- "--leader-election=true"
- "--retry-interval-start=500ms"
- "--default-fstype=nfs"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-nas-resizer
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 8Gi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-oss-provisioner
args:
- --csi-address=$(ADDRESS)
- --volume-name-prefix=oss
- --timeout=150s
- --leader-election=true
- --retry-interval-start=500ms
- --default-fstype=ossfs
- --v=5
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com/csi.sock
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-provisioner:v3.5.0-e7da67e52-aliyun
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 10m
memory: 16Mi
volumeMounts:
- mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
name: oss-provisioner-dir
- name: external-csi-snapshotter
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-snapshotter:v4.0.0-a230d5b3-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: /csi/csi.sock
volumeMounts:
- name: disk-provisioner-dir
mountPath: /csi
- name: external-snapshot-controller
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/snapshot-controller:v4.0.0-a230d5b3-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--leader-election=true"
- name: csi-provisioner
securityContext:
privileged: true
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-plugin:v1.26.5-56d1e30-aliyun
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=nas,disk,oss"
env:
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "provisioner"
- name: "CLUSTER_ID"
value: "{{.ClusterID}}"
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 5
periodSeconds: 20
ports:
- name: healthz
containerPort: 11270
volumeMounts:
- name: host-log
mountPath: /var/log/
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: oss-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/ossplugin.csi.alibabacloud.com
- mountPath: /var/addon
name: addon-token
readOnly: true
- mountPath: /mnt
mountPropagation: Bidirectional
name: host-mnt
- mountPath: /host/etc
name: etc
resources:
limits:
cpu: 500m
memory: 1024Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: disk-provisioner-dir
emptyDir: {}
- name: nas-provisioner-dir
emptyDir: {}
- name: oss-provisioner-dir
emptyDir: {}
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
type: ""
- name: host-mnt
hostPath:
path: /mnt
type: ""
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
展开查看csi-plugin的YAML文件
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: kube-system
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["csi-ossfs-credentials"]
verbs: ["get", "patch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch", "create", "delete" ]
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["csi-local-plugin-cert"]
verbs: ["get"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
- apiGroups: [""]
resources: ["endpoints"]
resourceNames: ["cnfs-cache-ds-service"]
verbs: ["get"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["csi-plugin", "ack-cluster-profile"]
verbs: ["get"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["storage-monitor-service"]
verbs: ["get"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "update", "patch", "list", "watch"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch", "create"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update", "create"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "list", "watch", "delete", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["rules"]
verbs: ["get"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["containernetworkfilesystems"]
verbs: ["get","list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: alicloud-csi-plugin
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: alicloud-csi-plugin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: diskplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: nasplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: ossplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-plugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-plugin
template:
metadata:
labels:
app: csi-plugin
spec:
tolerations:
- operator: Exists
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
nodeSelector:
kubernetes.io/os: linux
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
hostIPC: true
hostPID: true
dnsPolicy: ClusterFirst
containers:
- name: disk-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
- name: registration-dir
mountPath: /registration
- name: nas-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: oss-driver-registrar
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
limits:
cpu: 500m
memory: 1024Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: csi-plugin
securityContext:
privileged: true
allowPrivilegeEscalation: true
image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/csi-plugin:v1.26.5-56d1e30-aliyun
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=oss,nas,disk"
env:
- name: DEFAULT_REGISTRY
value: registry-vpc.cn-shenzhen.aliyuncs.com
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: SERVICE_TYPE
value: "plugin"
- name: MAX_VOLUMES_PERNODE
value: "15"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 1024Mi
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
ports:
- name: healthz
containerPort: 11260
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
mountPropagation: "Bidirectional"
- name: etc
mountPath: /host/etc
- name: host-log
mountPath: /var/log/
- name: ossconnectordir
mountPath: /host/usr/
- name: container-dir
mountPath: /var/lib/container
mountPropagation: "Bidirectional"
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- mountPath: /var/addon
name: addon-token
readOnly: true
- mountPath: /host/var/run/ossfs
name: ossfs-metrics-dir
- mountPath: /host/var/run/efc
name: efc-metrics-dir
- mountPath: /etc/csi-plugin/config
name: csi-plugin-cm
- name: host-mnt
mountPath: /mnt
mountPropagation: "Bidirectional"
- mountPath: /run/kata-containers/shared/direct-volumes
name: kata-direct-volumes
volumes:
- name: ossfs-metrics-dir
hostPath:
path: /var/run/ossfs
type: DirectoryOrCreate
- name: efc-metrics-dir
hostPath:
path: /var/run/efc
type: DirectoryOrCreate
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry
type: DirectoryOrCreate
- name: container-dir
hostPath:
path: /var/lib/container
type: DirectoryOrCreate
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
- name: ossconnectordir
hostPath:
path: /usr/
- name: host-mnt
hostPath:
path: /mnt
type: DirectoryOrCreate
- name: csi-plugin-cm
configMap:
name: csi-plugin
optional: true
- name: kata-direct-volumes
hostPath:
path: /run/kata-containers/shared/direct-volumes
type: DirectoryOrCreate
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
updateStrategy:
rollingUpdate:
maxUnavailable: 20%
type: RollingUpdate
集群版本为ACK 1.20以下
展开查看csi-provisioner的YAML文件
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-topology-alltype
parameters:
type: cloud_essd,cloud_ssd,cloud_efficiency
provisioner: diskplugin.csi.alibabacloud.com
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-available
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: available
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-essd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_essd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-ssd
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_ssd
reclaimPolicy: Delete
allowVolumeExpansion: true
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-alicloud-disk-efficiency
provisioner: diskplugin.csi.alibabacloud.com
parameters:
type: cloud_efficiency
reclaimPolicy: Delete
allowVolumeExpansion: true
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-provisioner
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-provisioner
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
replicas: 2
template:
metadata:
labels:
app: csi-provisioner
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: node-role.kubernetes.io/master
operator: Exists
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-provisioner
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
operator: Exists
key: node-role.kubernetes.io/master
- effect: NoSchedule
operator: Exists
key: node.cloudprovider.kubernetes.io/uninitialized
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
containers:
- name: external-disk-provisioner
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-cbd508573-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--provisioner=diskplugin.csi.alibabacloud.com"
- "--csi-address=$(ADDRESS)"
- "--feature-gates=Topology=True"
- "--volume-name-prefix=disk"
- "--strict-topology=true"
- "--timeout=150s"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--retry-interval-start=500ms"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-attacher
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-attacher:v2.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-disk-resizer
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v1.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: external-nas-provisioner
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-cbd508573-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--provisioner=nasplugin.csi.alibabacloud.com"
- "--csi-address=$(ADDRESS)"
- "--volume-name-prefix=nas"
- "--timeout=150s"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--retry-interval-start=500ms"
- "--v=5"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-nas-resizer
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v1.1.0
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election"
env:
- name: ADDRESS
value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
imagePullPolicy: "Always"
volumeMounts:
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- name: external-csi-snapshotter
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-snapshotter:v4.0.0-1f9e7a7f8-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--extra-create-metadata=true"
env:
- name: ADDRESS
value: /csi/csi.sock
imagePullPolicy: Always
volumeMounts:
- name: disk-provisioner-dir
mountPath: /csi
- name: external-snapshot-controller
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/snapshot-controller:v4.0.0-41f9e7a7f8-aliyun
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--leader-election=true"
imagePullPolicy: Always
- name: csi-provisioner
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
imagePullPolicy: "Always"
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=nas,disk"
env:
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "provisioner"
- name: "CLUSTER_ID"
value: "{{.ClusterID}}"
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 5
periodSeconds: 20
ports:
- name: healthz
containerPort: 11270
volumeMounts:
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- name: host-log
mountPath: /var/log/
- name: etc
mountPath: /host/etc
- name: disk-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
- name: nas-provisioner-dir
mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
- mountPath: /var/addon
name: addon-token
readOnly: true
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: disk-provisioner-dir
emptyDir: {}
- name: nas-provisioner-dir
emptyDir: {}
- name: host-log
hostPath:
path: /var/log/
- name: host-dev
hostPath:
path: /dev
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
- name: etc
hostPath:
path: /etc展开查看csi-plugin的YAML文件
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: csi-admin
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "create", "list"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
resources: ["persistentvolumeclaims/status"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["csinodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "watch", "list", "delete", "update", "create"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update", "patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create", "list", "watch", "delete", "get", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["get", "create", "list", "watch", "delete", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments/status"]
verbs: ["patch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots/status"]
verbs: ["update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods","pods/exec"]
verbs: ["create", "delete", "get", "post", "list", "watch", "patch", "udpate"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["rules"]
verbs: ["get"]
- apiGroups: ["storage.alibabacloud.com"]
resources: ["containernetworkfilesystems"]
verbs: ["get","list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: alicloud-csi-plugin
subjects:
- kind: ServiceAccount
name: csi-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: alicloud-csi-plugin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: diskplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: nasplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:
name: ossplugin.csi.alibabacloud.com
spec:
attachRequired: false
podInfoOnMount: true
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-plugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-plugin
template:
metadata:
labels:
app: csi-plugin
spec:
tolerations:
- operator: Exists
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: type
operator: NotIn
values:
- virtual-kubelet
nodeSelector:
beta.kubernetes.io/os: linux
serviceAccount: csi-admin
priorityClassName: system-node-critical
hostNetwork: true
hostPID: true
containers:
- name: disk-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet
- name: registration-dir
mountPath: /registration
- name: nas-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/nasplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: oss-driver-registrar
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-node-driver-registrar:v1.2.0
imagePullPolicy: Always
resources:
requests:
cpu: 10m
memory: 16Mi
args:
- "--v=5"
- "--csi-address=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
- "--kubelet-registration-path=/var/lib/kubelet/csi-plugins/ossplugin.csi.alibabacloud.com/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
- name: registration-dir
mountPath: /registration
- name: csi-plugin
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.18.8.51-c504ef45-aliyun
imagePullPolicy: "Always"
args:
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=2"
- "--driver=oss,nas,disk"
env:
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
- name: MAX_VOLUMES_PERNODE
value: "15"
- name: SERVICE_TYPE
value: "plugin"
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 1024Mi
livenessProbe:
httpGet:
path: /healthz
port: healthz
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 5
ports:
- name: healthz
containerPort: 11260
volumeMounts:
- name: kubelet-dir
mountPath: /var/lib/kubelet/
mountPropagation: "Bidirectional"
- name: etc
mountPath: /host/etc
- name: host-log
mountPath: /var/log/
- name: ossconnectordir
mountPath: /host/usr/
- name: container-dir
mountPath: /var/lib/container
mountPropagation: "Bidirectional"
- name: host-dev
mountPath: /dev
mountPropagation: "HostToContainer"
- mountPath: /var/addon
name: addon-token
readOnly: true
volumes:
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry
type: DirectoryOrCreate
- name: container-dir
hostPath:
path: /var/lib/container
type: DirectoryOrCreate
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: host-dev
hostPath:
path: /dev
- name: host-log
hostPath:
path: /var/log/
- name: etc
hostPath:
path: /etc
- name: ossconnectordir
hostPath:
path: /usr/
- name: addon-token
secret:
defaultMode: 420
optional: true
items:
- key: addon.token.config
path: token-config
secretName: addon.csi.token
updateStrategy:
rollingUpdate:
maxUnavailable: 10%
type: RollingUpdate验证CSI组件
通过创建StatefulSet，查看Pod状态变为running，验证CSI组件部署成功，迁移环境准备就绪。
展开查看用于创建StatefulSet的YAML文件
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-csi-tr
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx"
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- name: disk-csi-tr
mountPath: /data
volumeClaimTemplates:
- metadata:
name: disk-csi-tr
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "csi-alicloud-disk-topology-alltype"
resources:
requests:
storage: 20Gi
上述组件部署完成后，即可在集群中同时使用Flexvolume组件和CSI组件。后续逐渐将Flexvolume类型的应用转化为CSI类型的应用。
切换应用挂载的存储类型
您需要将应用挂载使用的Flexvolume类型的存储全部转换为CSI类型，待Flexvolume集群中无Flexvolume类型的存储时，即可删除Flexvolume组件。
步骤一：转换PVC和PV为CSI类型
获取集群中所有Flexvolume类型的PVC和PV，并且将其转换为CSI类型。您可以使用命令行工具实现集群中Flexvolume类型PVC和PV的批量转换。具体操作，请参见使用Flexvolume2CSI命令行工具批量转换YAML。
本文以云盘为例，展示如何将获取到的Flexvolume类型的PVC和PV转换为CSI类型的PVC和PV。您可以根据实际情况，参考如下CSI模板进行修改。
展开查看已获取的Flexvolume类型的PVC和PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: pvc-disk
namespace: default
flexVolume:
driver: alicloud/disk
fsType: ext4
options:
VolumeId: d-bp1bnp9homa0tyv6****
persistentVolumeReclaimPolicy: Delete
storageClassName: alicloud-disk-available
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-disk
spec:
accessModes:
- ReadWriteOnce
storageClassName: alicloud-disk-available
resources:
requests:
storage: 20Gi展开查看转化后的CSI类型的PVC和PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
storageClassName: "csi-alicloud-disk-topology-alltype"
capacity:
storage: 20Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
csi:
driver: diskplugin.csi.alibabacloud.com
volumeHandle: d-bp1bnp9homa0tyv6****
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-disk
spec:
storageClassName: "csi-alicloud-disk-topology-alltype"
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
volumeName: d-bp1bnp9homa0tyv6****步骤二：切换回收策略
为确保存储资源使用的存储卷不会丢失，您需要检查待迁移的Flexvolume类型的PVC和PV。如果pv.Spec.persistentVolumeReclaimPolicy为Delete，需要将其改为Retain。
apiVersion: v1
kind: PersistentVolume
metadata:
name: d-bp1bnp9homa0tyv6****
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
claimRef:
apiVersion: v1
kind: PersistentVolumeClaim
name: pvc-disk
namespace: default
flexVolume:
driver: alicloud/disk
fsType: ext4
options:
VolumeId: d-bp1bnp9homa0tyv6****
persistentVolumeReclaimPolicy: **Retain**
storageClassName: alicloud-disk-available
volumeMode: Filesystem步骤三：切换单个应用挂载的存储类型
您可以参考以下步骤，将单个应用挂载的存储类型由Flexvolume切换为CSI类型。
执行以下命令，将StatefulSet的副本缩为0。
kubectl scale --replicas=0 sts/sts-test执行以下命令，删除集群中Flexvolume类型的PVC和PV。
kubectl delete pvc pvc-disk kubectl delete pv d-bp1bnp9homa0tyv6****执行以下命令，创建CSI类型的PVC和PV。
kuectl apply -f csi-pvc-pv.yaml执行以下命令，将StatefulSet重新扩容。
kubectl scale --replicas=3 sts/sts-test说明pvcClaimName和Deployment可按照同样方式进行切换。
您可以参考切换单个应用的方法依次切换所有应用挂载的存储类型。待所有应用切换完成后，即可删除Flexvolume组件。具体操作，请参见将无存储集群的Flexvolume迁移至CSI。
管理组件
升级组件
如果您已经按照上述步骤在Flexvolume集群中安装了CSI组件，但是需要将CSI升级到最新版本。您可以通过以下命令，手动替换集群中csi-provisioner和csi-plugin的main container的image版本。相关版本信息，请参见csi-plugin。
kubectl set image -nkube-system daemonset/csi-plugin csi-plugin=registry.cn-beijing.aliyuncs.com/acs/csi-plugin:v1.26.3-796c9dc-aliyun
kubectl set image -nkube-system deployment/csi-provisioner csi-provisioner=registry.cn-beijing.aliyuncs.com/acs/csi-plugin:v1.26.3-796c9dc-aliyun如果您需要通过容器服务管理控制台的组件管理页面升级，则需要迁移所有的Flexvolume版本的PV、PVC到CSI后才可以执行。在Flexvolume和CSI组件共存期间，将无法通过组件管理页面进行组件升级。迁移相关操作，请参见将无存储集群的Flexvolume迁移至CSI。
删除组件
当集群中所有存储由Flexvolume迁移至CSI之后，且满足以下条件时，您可以将csi-compatible-controller组件删除。
集群中的Flexvolume组件已全部删除。
所有Flexvolume类型的PV都已删除。
检查确认Kubelet的
--enable-controller-attach-detach参数都已变为true。如不满足，请修改。具体操作，请参见将无存储集群的Fllexvolume迁移至CSI文档的步骤三和步骤四。
相关文档
如果需要迁移无存储的Flexvolume集群，请参见将无存储集群的Flexvolume迁移至CSI。