Container Service for Kubernetes (ACK) provides container storage features based on the Kubernetes storage system. The storage features are integrated with Alibaba Cloud storage services and are fully compatible with native storage objects of Kubernetes, such as EmptyDir, HostPath, Secrets, and ConfigMaps. ACK provides the Container Storage Interface (CSI) plug-in based on the open source CSI. You must deploy the CSI plug-in before you can use Alibaba Cloud storage services in ACK clusters. This topic describes the overview, features, and limits of the CSI plug-in, and the permissions that are required to use the CSI plug-in.
Container storage architecture
|Alibaba Cloud storage service||Statically provisioned volume||Dynamically provisioned volume||Deployed by default||Feature||Scenario|
|Disk||Supported||Supported||Yes||Non-shared storage. A disk can be mounted on only one node.||
For more information, see Overview.
|NAS||Supported||Supported||Yes||Shared storage that provides high performance and high throughput.||
For more information, see Overview.
|OSS||Supported||Not supported||Yes||Shared storage that supports file systems in the user space.||
Note OSS buckets are mounted by using ossfs, which is implemented as a filesystem in the user space (FUSE). The write performance is limited when you use OSS buckets as volumes. We recommend that you use other storage media as volumes in scenarios that require high write performance.
For more information, see Overview.
|CPFS||Supported||Supported||No||Shared storage that features high performance and high bandwidth.||
- A PV is a piece of storage in the cluster. A PV has a lifecycle that is independent of the pod that uses the PV. Different types of PV can be created based on different StorageClasses.
- A PVC is a request for storage in the cluster. PVs are node resources consumed by pods. PVCs are claims that consume PV resources. When PVs are insufficient, PVCs can dynamically provision PVs.
Container storage features
The following table describes the storage features supported by different types of ACK cluster.
|Storage type||Feature||ACK cluster (Linux-based)||ASK cluster||Registered cluster (hybrid cloud or multicloud)||ACK@Edge||ACK cluster (Windows-based)||Dedicated ACK cluster||Sandboxed container|
|Block storage||Mount and unmount disks||✔️||✔️||❌||❌||✔️||✔️||✔️|
|Container I/O monitoring||✔️||❌||❌||❌||❌||✔️||❌|
|File systems||XFS, ext4, and dBFS are supported.||XFS and ext4 are supported.||❌||❌||NTFS is supported.||XFS and ext4 are supported.||XFS and ext4 are supported.|
|Block and bare devices||✔️||❌||❌||❌||❌||✔️||❌|
|Data restoration from snapshots||✔️||✔️||❌||❌||❌||✔️||✔️|
|Disk queue settings||✔️||❌||❌||❌||❌||✔️||❌|
|Customer Managed Key (CMK) and Bring Your Own Key (BYOK)||✔️||✔️||❌||❌||✔️||✔️||✔️|
|File storage||Create, mount, and unmount NAS file systems||✔️||✔️||✔️||✔️||❌||✔️||✔️|
|Mount and unmount Samba file systems||❌||❌||✔️||❌||✔️||❌||❌|
|NAS recycle bin||❌||❌||❌||❌||❌||❌||❌|
|Subdirectories of dynamically provisioned volumes||✔️||❌||✔️||✔️||✔️||✔️||✔️|
|CMK (Extreme NAS)||✔️||❌||❌||❌||❌||✔️||✔️|
|Object storage||Mount and unmount OSS buckets||✔️||✔️||✔️||✔️||❌||✔️||✔️|
|Local storage||Linux Volume Manager (LVM)-managed block storage||✔️||❌||✔️||✔️||❌||✔️||❌|
|Automated volume groups||✔️||❌||✔️||✔️||❌||✔️||❌|
|Node capacity scheduling||✔️||❌||✔️||✔️||❌||✔️||❌|
|PMEM Direct Mem||✔️||❌||❌||❌||❌||❌||❌|
|LVM-managed persistent memory (PMEM)||✔️||❌||❌||❌||❌||❌||❌|
CSI deployment architectures
|Managed Kubernetes cluster||Dedicated Kubernetes cluster|
In managed Kubernetes clusters, CSI-Provisioner and CSI-Plugin are deployed on worker nodes.
In dedicated Kubernetes clusters, CSI-Provisioner is deployed on master nodes. CSI-Plugin is automatically deployed as DaemonSets on master and worker nodes.
Permissions required to use CSI
Before you can use the CSI plug-in to mount, unmount, create, and delete volumes, you must grant the plug-in the permissions to access other cloud resources. You can use an AccessKey pair or a Resource Access Management (RAM) role to grant permissions to the CSI plug-in. The default method is to use a RAM role. The following table describes the two authorization methods.
|Use an AccessKey pair||Use a RAM role|
The CSI plug-in uses the RAM role AliyunCSManagedCsiRole to access resources of other cloud services. For more information, see AliyunCSManagedCsiRole. For more information about how to grant permissions to a RAM role, see Grant permissions to a RAM role.
When you use the CSI plug-in in ACK clusters, take note of the limits of the CSI plug-in and Alibaba Cloud storage services.
- Limits of Alibaba Cloud storage services
Alibaba Cloud storage service Limits Disk
- You can mount up to 15 disks as volumes on a node.
- You can provision a disk to only one pod as a volume.
- You cannot mount disks of all types on all ECS instances. For more information, see Instance families.
- You cannot mount or unmount subscription disks as volumes.
- You can mount a disk only on an ECS instance that is in the same zone as the disk.
- We recommend that you create StatefulSets instead of Deployments to use volumes that
are created from disks.
Note Deployments are used to create stateless applications. When a pod is restarted, the start time of the new pod may overlap the end time of the old pod. If multiple pods are created for a Deployment, no dedicated volume is provisioned for each pod.
- The minimum capacity of each volume is 20 GiB.
- You can mount a NAS file system only on an ECS instance that is deployed in the same virtual private cloud (VPC) as the NAS file system.
- The number of NAS file systems that you can create is subject to a quota limit. To request a quota increase, Submit a ticket to the NAS team.
OSS We recommend that you do not perform data write operations on volumes that are created from OSS buckets. Use other storage media for data write operations.Note OSS buckets are mounted by using ossfs, which is implemented as a filesystem in the user space (FUSE). The write performance is limited when you use OSS buckets as volumes. We recommend that you use other storage media as volumes in scenarios that require high write performance. CPFS
- The CPFS driver is highly dependent on the OS kernel. After you deploy the CPFS environment, do not upgrade the OS kernel.
- You can install the CPFS driver but cannot upgrade the CPFS driver.
The CSI-CPFS plug-in is a Kubernetes CSI component that mounts CPFS file systems as volumes on pods for the use of applications.
The CPFS driver is a client driver that implements the CPFS protocol at the kernel layer.Relationship between the CSI-CPFS plug-in and the CPFS driver:
- If the CPFS driver is not installed on a node, the CPFS driver is automatically installed when you deploy the CSI-CPFS plug-in.
- If the CPFS driver is installed on a node, the CPFS driver is not installed or upgraded when you deploy the CSI-CPFS plug-in.
- Limits of the CSI plug-in
The CSI plug-in is an open source plug-in for ACK clusters. In other types of clusters, such as clusters deployed in third-party clouds and self-managed clusters on Alibaba Cloud, you cannot directly use the CSI plug-in for reasons such as cluster configurations, permission management, and network differences. If you want to use the CSI plug-in in these types of clusters, you must modify the cluster configurations based on the source code. For more information, see alibaba-cloud-csi-driver.
Kubernetes version requirements
To use the CSI plug-in in an ACK cluster, the Kubernetes version of the cluster must
be 1.14 or later. Besides, the kubelet parameter
--enable-controller-attach-detach must be set to
Installation and upgrade of the CSI plug-in
For more information about how to install and upgrade the CSI plug-in, see Install and upgrade the CSI plug-in.
Differences between the CSI and FlexVolume plug-ins
|Flexvolume||FlexVolume is a traditional mechanism to extend Kubernetes storage systems developed
by the Kubernetes community. ACK supports FlexVolume. FlexVolume consists of the following
For more information about FlexVolume, see Overview.
For more information about how to upgrade FlexVolume, see Manage system components.
|CSI||The Kubernetes community recommends the CSI plug-in. The CSI plug-in provided by ACK
is compatible with the features of the community version. CSI consists of the following
- You must select a plug-in when you create an ACK cluster.
- You cannot use CSI and FlexVolume in the same cluster.
- You cannot change the plug-in from FlexVolume to CSI for a cluster.
- For new ACK clusters, we recommend that you use CSI. The ACK technical team will continuously upgrade CSI to support more features of the CSI community version.
- For existing clusters, we recommend that you use the plug-in that is already installed. The ACK technical team will continue its support for FlexVolume.
How to check the storage plug-in used in a cluster
- Method 1: Check node annotations by using the console
- Log on to the ACK console.
- In the left-side navigation pane, click Clusters.
- On the Clusters page, find the cluster that you want to manage, and click the name of the cluster or click Details in the Actions column.
- In the left-side navigation pane of the details page, choose .
- Select a node and click Actions column. in the
- In the Overview section, check Annotations.
volumes.kubernetes.io/controller-managed-attach-detach: trueis displayed, the cluster uses the CSI plug-in. If
volumes.kubernetes.io/controller-managed-attach-detach: trueis not displayed, the cluster uses the FlexVolume plug-in.
- Method 2: Check kubelet parameters
Run the following command to check kubelet parameters:
ps -ef | grep kubelet
If the value of
true, the cluster uses the CSI plug-in. If the value of
false, the cluster uses the FlexVolume plug-in.