All Products
Search
Document Center

Container Service for Kubernetes:CSI overview

Last Updated:Sep 18, 2023

Container Service for Kubernetes (ACK) provides container storage services based on the Container Storage Interface (CSI) plug-in. CSI enables the integration with Elastic Block Storage (EBS), Apsara File Storage NAS (NAS), Object Storage Service (OSS), and local disks of Alibaba Cloud. CSI also enables compatibility with native Kubernetes storage services, such as emptyDir, HostPath, Secret, and ConfigMap. This topic describes the overview, features, and limits of the CSI plug-in, and the permissions that are required to use the CSI plug-in.

ACK container storage architecture

overview

ACK allows you to configure storage services to be automatically mounted to pods. The storage services include Alibaba Cloud disks, NAS, OSS, and local volumes. The following table describes the features and use scenarios of the storage services.

Alibaba Cloud storage service

Statically provisioned volume

Dynamically provisioned volume

Deployed by default

Feature

Scenario

Alibaba Cloud disks

Supported

Supported

Yes

Non-shared storage. A disk can be mounted only to one node.

  • High I/O and low latency

    Disks are block storage devices and are suitable for use in scenarios that require high I/O performance and low latency. For example, databases and middleware services.

  • Non-data sharing

    A disk can be provisioned only for one pod. You can use disk volumes in scenarios that do not require data sharing.

For more information, see Disk volume overview.

NAS

Supported

Supported

Yes

Shared storage that provides high performance and high throughput.

  • Data sharing

    NAS file systems allow multiple pods to access the same data. We recommend that you use NAS file systems if data needs to be shared.

  • Big data analysis

    NAS file systems provide high throughput and meet the requirement of shared storage access when large numbers of jobs are involved.

  • Web applications

    NAS file systems can provision storage for web applications and content management systems.

  • Log storage

    We recommend that you use NAS volumes to store log data.

For more information, see NAS volume overview.

OSS

Supported

Supported

Yes

Shared storage that supports file systems in user space.

  • Read-only media files such as video files and images

    You can use OSS volumes to read the preceding types of files.

  • Read-only configuration files of websites and applications

    ossfs provides limited network performance and can be used to read small files.

Note

OSS volumes are mounted by using ossfs, which is implemented as a file system in user space (FUSE). The write performance is limited when you use OSS volumes. We recommend that you use other storage volumes in scenarios that require high write performance.

For more information, see OSS volume overview.

Note

You can use the CSI plug-in to mount storage resources as statically provisioned volumes and as dynamically provisioned volumes in ACK clusters. To mount a storage resource as a statically provisioned volume, you must manually create a persistent volume (PV) and a persistent volume claim (PVC). When large numbers of PVs and PVCs are required, you can mount storage resources as dynamically provisioned volumes. Definitions of PV and PVC:

  • PV

    • A PV is a piece of storage in the cluster. A PV has a lifecycle that is independent of the pod that uses the PV. Different types of PV can be created based on different types of StorageClass.

  • PVC

    • A PVC is a request for storage in the cluster. PVs are node resources consumed by pods. PVCs are claims that consume PVs. When PVs are insufficient, PVCs can dynamically provision PVs.

ACK container storage features

The following table describes the storage features supported by different types of ACK clusters.

Storage type

Feature

ACK cluster (Linux-based)

ACK Serverless cluster

Registered cluster (hybrid cloud or multicloud)

ACK edge cluster

ACK cluster (Windows-based)

Container Service cluster

ACK cluster that supports sandboxed containers

Block storage

Mount and unmount disks

对对错错对对对

Online resizing

对对错错错对错

Snapshots

对对错错错对对

Container I/O monitoring

对对错错错对错

File systems

XFS, ext4, and dBFS are supported.

XFS and ext4 are supported.

错错

NTFS is supported.

XFS and ext4 are supported.

XFS and ext4 are supported.

Block devices and raw devices

对错错错错对错

Data restoration from snapshots

对对错错错对对

Disk queue settings

对错错错错对错

Customer managed key (CMK)-based encryption and Bring Your Own Key (BYOK) encryption

对对错错对对对

Multi-zone awareness

对对错错对对对

Custom labels

对对错错对对对

Cross-host migration

对对错错对对对

File storage

Create, mount, and unmount NAS file systems

对对对对错对对

Mount and unmount Samba file systems

错错对错对错错

Recycle bin (CNFS)

对错错对错错错

Subdirectories or shared directories of dynamically provisioned volumes (CNFS)

对错对对对对对

CMK-based encryption (CNFS and Extreme NAS file systems)

对错错对错对对

Quota limits (CNFS)

Supported only by ACK managed clusters

错错对错错错

Capacity and I/O monitoring (CNFS)

对错对对错错错

Online resizing (CNFS)

对错对对错错错

Object storage

Mount and unmount OSS buckets

对对对错错对对

BYOK-based encryption

对错对对错对对

Local storage

Linux Volume Manager (LVM)-managed block storage

对错对对错对错

Automated volume groups

对错对对错对错

LVM-managed capacity-aware scheduling

对错对对错对错

PMEM Direct Mem

对错错错错错错

LVM-managed persistent memory (PMEM)

对错错错错错错

CSI deployment architectures

The CSI plug-in consists of two parts: CSI-Plugin and CSI-Provisioner. The following figure shows the deployment architectures of the CSI plug-in in an ACK managed cluster and an ACK dedicated cluster.

Note

The CSI plug-in is automatically installed in ACK managed clusters and ACK dedicated clusters. If you use a ACK Serverless (ASK) cluster or an ACK edge cluster, you must manually install the CSI plug-in.

ACK managed cluster

ACK dedicated cluster

csi managed

In ACK managed clusters, CSI-Provisioner and CSI-Plugin are deployed on worker nodes.

flexvolume

In ACK dedicated clusters, CSI-Provisioner is deployed on master nodes. CSI-Plugin is automatically deployed as DaemonSets on master and worker nodes.

Permissions required to use CSI

Before you can use the CSI plug-in to mount, unmount, create, and delete volumes, you must grant the plug-in the permissions to access other cloud resources. You can use an AccessKey pair or a Resource Access Management (RAM) role to grant permissions to the CSI plug-in. The default method is to use a RAM role. The following table describes the two authorization methods.

Use an AccessKey pair

Use a RAM role

  • You can specify an AccessKey pair in the deployment template of the CSI plug-in.

  • You can also create a Secret to pass an AccessKey pair as environment variables.

The CSI plug-in uses the RAM role AliyunCSManagedCsiRole to access resources of other cloud services. For more information, see ACK default roles. For more information about how to grant permissions to a RAM role, see Grant permissions to a RAM role.

  • ACK managed clusters

    The permission token of the RAM role used by the CSI plug-in is stored in a Secret named addon.csi.token. To grant permissions to the CSI plug-in and allow the plug-in to call API operations, you need only to mount the Secret to the plug-in.

  • ACK dedicated clusters

    The CSI plug-in assumes the RAM role assigned to the Elastic Compute Service (ECS) node that hosts the CSI pods.

Limits on CSI

When you use the CSI plug-in in ACK clusters, take note of the limits on the CSI plug-in and Alibaba Cloud storage services.

  • Limits on Alibaba Cloud storage services

    Alibaba Cloud storage service

    Limits

    Alibaba Cloud disks

    • You can provision a disk as a volume only for one pod.

    • You cannot mount disks of all types to ECS instances of all types. For more information, see Overview of instance families.

    • You cannot mount or unmount subscription disks as volumes.

    • You can mount a disk only to an ECS instance in the same zone as the disk.

    • We recommend that you create StatefulSets instead of Deployments to use disk volumes.

      Note

      Deployments are used to create stateless applications. When a pod is restarted, the start time of the new pod may overlap with the end time of the old pod. If multiple pods are created for a Deployment, no dedicated volume is provisioned for each pod.

    • The minimum capacity of a disk volume is 20 GiB.

    NAS

    • You can mount a NAS volume only to ECS instances in the same virtual private cloud (VPC) as the NAS file system.

    • You cannot use the CSI plug-in to mount Server Message Block (SMB) file systems.

    • The number of NAS file systems that you can create is subject to a quota limit. To request the quota increase, Join the DingTalk group 35532895. for technical support.

    OSS

    We recommend that you do not perform data write operations on OSS volumes. Use other storage media for data write operations.

    Note

    OSS volumes are mounted by using ossfs, which is implemented as a file system in user space (FUSE). The write performance is limited when you use OSS volumes. We recommend that you use other storage volumes in scenarios that require high write performance.

  • Limits on the CSI plug-in

    The CSI plug-in is an open source plug-in for ACK clusters. In other types of clusters, such as clusters deployed in third-party clouds and self-managed clusters on Alibaba Cloud, you cannot directly use the CSI plug-in for reasons such as cluster configurations, permission management, and network differences. If you want to use the CSI plug-in in these types of clusters, you must modify the cluster configurations based on the source code. For more information, see alibaba-cloud-csi-driver.

Requirements on Kubernetes versions

To use the CSI plug-in in an ACK cluster, the Kubernetes version of the cluster must be 1.14 or later. Besides, the kubelet parameter --enable-controller-attach-detach must be set to true.

Installation and upgrade of the CSI plug-in

For more information about how to install and upgrade the CSI plug-in, see Install and upgrade the CSI plug-in.

Differences between the CSI and FlexVolume plug-ins

Plug-in

Feature

References

Flexvolume

FlexVolume is a traditional mechanism to extend the storage systems developed by the Kubernetes community. ACK supports FlexVolume. FlexVolume consists of the following parts:

  • FlexVolume: allows you to mount and unmount volumes. By default, ACK allows you to mount the following types of storage media by using FlexVolume: disks, NAS file systems, and OSS buckets.

  • Disk-Controller: automatically creates disk volumes.

  • Nas-Controller: automatically creates NAS volumes.

For more information about FlexVolume, see Overview.

For more information about how to upgrade FlexVolume, see Manage components.

CSI

The Kubernetes community recommends the CSI plug-in. The CSI plug-in provided by ACK is compatible with the features of the community version. CSI consists of the following two parts:

  • CSI-Plugin: allows you to mount and unmount volumes. By default, ACK allows you to mount the following types of storage media by using CSI-Plugin: disks, NAS file systems, and OSS buckets.

  • CSI-Provisioner: automatically creates disk volumes and NAS volumes.

For more information about CSI, see Overview and alibaba-cloud-csi-driver.

Note
  • You must select a plug-in when you create an ACK cluster.

  • You cannot use CSI and FlexVolume in the same cluster.

  • You cannot change the plug-in from FlexVolume to CSI for a cluster.

Recommendation

  • For newly created ACK clusters, we recommend that you use CSI. The ACK technical team will continuously upgrade CSI to support more features of the CSI community version.

  • For existing clusters, we recommend that you use the plug-in that is already installed. The ACK technical team will continue its support for FlexVolume.

How to check the storage plug-in used in a cluster

  • Method 1: Check node annotations by using the console

    1. Log on to the ACK console.

    2. In the left-side navigation pane, click Clusters.

    3. On the Clusters page, find the cluster to which you want to add a node. Then, click the name of the cluster or click Details in the Actions column.

    4. In the left-side navigation pane of the details page, choose Nodes > Nodes.

    5. Select a node and click More > Details in the Actions column.

    6. In the Overview section, check Annotations.

      If volumes.kubernetes.io/controller-managed-attach-detach: true is displayed, the cluster uses the CSI plug-in. If volumes.kubernetes.io/controller-managed-attach-detach: true is not displayed, the cluster uses the FlexVolume plug-in.

  • Method 2: Check kubelet parameters

    Run the following command to check kubelet parameters:

    ps -ef | grep kubelet

    Expected output:

    --enable-controller-attach-detach=true

    If the value of --enable-controller-attach-detach is true, the cluster uses the CSI plug-in. If the value of --enable-controller-attach-detach is false, the cluster uses the FlexVolume plug-in.