Container Service for Kubernetes:Mount a dynamically provisioned OSS volume using ossfs 2.0 in ACK

By leveraging RRSA, you can achieve fine-grained, PV-level permission isolation for accessing cloud resources, which significantly reduces security risks. For details, see Use RRSA to authorize different pods to access different cloud services.

Prerequisites

• You have an ACK cluster running Kubernetes 1.26 or later. Manually upgrade the cluster if needed.

• The Container Storage Interface (CSI) plugin version is 1.33.1 or later. To upgrade it, see Update csi-plugin and csi-provisioner.

  If you use RRSA in a version earlier than 1.30.4, see [Product Changes] Version upgrade and mounting process optimization of ossfs in CSI to configure the RAM role authorization.

• You have created an OSS bucket in the same Alibaba Cloud account as your cluster.

  To mount an OSS bucket across accounts, we recommend using the RRSA authentication method. For details, see FAQ about ossfs 2.0 volumes

Step 1: Create a RAM role

If you have mounted an OSS volume in the cluster using RRSA, skip this step. If this is your first time, follow these steps:

1. Enable the RRSA feature in the ACK console.

2. Create a RAM role for an OIDC IdP. This role will be assumed using RRSA.

   The following table lists the key parameters for the sample role demo-role-for-rrsa:

   Parameter	Description
   Identity Provider Type	Select OIDC.
   Identity Provider	Select the provider associated with your cluster, such as ack-rrsa-<cluster_id>, where <cluster_id> is your actual cluster ID.
   Condition	oidc:iss: Keep the default value. oidc:aud: Keep the default value. oidc:sub: Manually add the following condition: Key: Select oidc:sub. Operator: Select StringEquals. Value: Enter system:serviceaccount:ack-csi-fuse:csi-fuse-ossfs. In this value, ack-csi-fuse is the namespace where the ossfs client is located, and cannot be customized. csi-fuse-ossfs is the service account name, and can be changed as needed. For more information about how to modify the service account name, see FAQ about ossfs 2.0 volumes
   Role Name	Enter demo-role-for-rrsa.

Step 2: Grant permissions to the demo-role-for-rrsa role

1. Create a custom policy to grant OSS access permissions to the RAM user. For more information, see Create custom policies.

   Select the read-only policy or read-write policy based on your business requirements. Replace mybucket with the name of the bucket you created.

   • Policy that provides read-only permissions on OSS

     Click to view policy content

     {
         "Statement": [
             {
                 "Action": [
                     "oss:Get*",
                     "oss:List*"
                 ],
                 "Effect": "Allow",
                 "Resource": [
                     "acs:oss:*:*:mybucket",
                     "acs:oss:*:*:mybucket/*"
                 ]
             }
         ],
         "Version": "1"
     }

   • Policy that provides read-write permissions on OSS

     Click to view policy content

     {
         "Statement": [
             {
                 "Action": "oss:*",
                 "Effect": "Allow",
                 "Resource": [
                     "acs:oss:*:*:mybucket",
                     "acs:oss:*:*:mybucket/*"
                 ]
             }
         ],
         "Version": "1"
     }

2. Optional. If the objects in the OSS bucket are encrypted by using a specified customer master key (CMK) in Key Management Service (KMS), you need to grant KMS access permissions to the RAM user. For more information, see Encryption.

3. Grant the required permissions to the demo-role-for-rrsa role. For more information, see Grant permissions to a RAM role.

   Note

   If you want to use an existing RAM role that has OSS access permissions, you can modify the trust policy of the RAM role. For more information, see Use an existing RAM role and grant the required permissions to the RAM role.

Step 3: Create a StorageClass

Create a StorageClass to use as a template for dynamically provisioning PVs.

1. Create a file named ossfs2-sc-rrsa.yaml with the following content:

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: ossfs2-sc  # The name of the StorageClass.
   parameters:
     bucket: cnfs-oss-test  # The name of the bucket.
     path: /subpath  # The subdirectory to mount. If left empty, the root directory is mounted.
     url: oss-cn-hangzhou-internal.aliyuncs.com  # The endpoint of the region where the OSS Bucket is located.
     authType: rrsa
     roleName: demo-role-for-rrsa  # The RAM role created earlier.
     fuseType: ossfs2
     volumeAs: sharepath
     otherOpts: "-o close_to_open=false"
   provisioner: ossplugin.csi.alibabacloud.com  # This value is fixed.
   reclaimPolicy: Retain  # The reclaim policy for the dynamically provisioned PV. Only Retain is supported, which means the PV and the data in the OSS Bucket are not deleted when the PVC is deleted.
   volumeBindingMode: Immediate  # The volume binding mode. OSS volumes do not require zone-based node affinity, so you can use the default value Immediate.

   The following table describes the parameters in the parameters field:

   Parameter	Required	Description
   bucket	Yes	The name of the OSS bucket to mount.
   path	Yes	The base path within the bucket. When volumeAs is set to sharepath, each dynamically provisioned PV is assigned a unique subdirectory under this path, such as /ack/<pv-name>.
   url	Yes	The access endpoint for the OSS bucket. Use an internal endpoint if your cluster nodes and the bucket are in the same region, or if a Virtual Private Cloud (VPC) connection is established. Use a public endpoint if the mount node and the bucket are in different regions. The following are common formats for different access endpoints: Internal: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com. The internal access endpoint format vpc100-oss-{{regionName}}.aliyuncs.com is deprecated. Switch to the new format as soon as possible. Public: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com.
   fuseType	Yes	Specifies the client to use for the mount. Must be set to ossfs2 to use the ossfs 2.0 client.
   authType	Yes	Set to rrsa to declare that the RRSA authentication method is used.
   roleName	Yes	Set to the name of the RAM role you created or modified. To assign different permissions for different PVs, create a separate RAM role for each permission set. Then, in each PV's definition, use the roleName parameter to associate it with the corresponding role.
   volumeAs	No	Defines how the PV is provisioned. sharepath indicates that each PV creates a separate subdirectory in the directory specified by path.
   otherOpts	No	Additional mount options passed to the OSS volume, specified as a string of space-separated flags in the format -o *** -o ***. For example, -o close_to_open=false. The close-to-open option controls metadata consistency. When set to false (default), metadata is cached to improve performance for small file reads. When set to true, the client fetches fresh metadata from OSS by sending a GetObjectMeta request every time a file is opened, ensuring real-time consistency at the cost of increased latency and API calls. For more optional parameters, see ossfs 2.0 mount options.

2. Apply the StorageClass.

   kubectl create -f ossfs2-sc-rrsa.yaml

3. Check the StorageClass status.

   kubectl get sc ossfs2-sc

   Expected output:

   NAME        PROVISIONER                      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
   ossfs2-sc   ossplugin.csi.alibabacloud.com   Retain          Immediate                                  10s

Step 4: Create a PVC

Step 5: Create an application and mount the volume

ACK supports authenticating OSS volume mounts by storing a static AccessKey in a Kubernetes Secret. This approach is suitable for scenarios where a specific application requires long-term, fixed access permissions.

• If the AccessKey referenced by the PV is revoked or its permissions are changed, any application using the volume will immediately lose access and encounter permission errors. To restore access, update the credentials in the Secret, then restart the application's pods to force a remount. This process causes a brief service interruption and should only be performed during a scheduled maintenance window.

• To avoid the downtime associated with manual key rotation, we strongly recommend using the RRSA authentication method instead.

Prerequisites

• You have an ACK cluster with CSI V1.33.1 or later installed. To upgrade the component, see Update csi-plugin and csi-provisioner.

• You have an OSS bucket in the same Alibaba Cloud account as your cluster.

  To mount an OSS bucket across accounts, we recommend using RRSA for authentication. For details, see FAQ about ossfs 2.0 volumes

Step 1: Create a RAM user with OSS access permissions and obtain an AccessKey pair

kubectl create -n default secret generic oss-secret --from-literal='akId=xxxxxx' --from-literal='akSecret=xxxxxx'

Step 2: Create a StorageClass

1. Create a file named ossfs2-sc.yaml with the following content:

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: ossfs2-sc
   parameters:
     # Use the Secret created in the preparations.
     csi.storage.k8s.io/node-publish-secret-name: oss-secret  
     csi.storage.k8s.io/node-publish-secret-namespace: default
     fuseType: ossfs2
     bucket: cnfs-oss-test # The name of the bucket.
     path: /subpath  # The subdirectory to mount. If left empty, the root directory is mounted.
     url: oss-cn-hangzhou-internal.aliyuncs.com  # The endpoint of the region where the OSS Bucket is located.
     otherOpts: "-o close_to_open=false"
   provisioner: ossplugin.csi.alibabacloud.com  # This value is fixed.
   reclaimPolicy: Retain  # The reclaim policy for the dynamically provisioned PV. Only Retain is supported, which means the PV and the data in the OSS Bucket are not deleted when the PVC is deleted.
   volumeBindingMode: Immediate  # The volume binding mode. OSS volumes do not require zone-based node affinity, so you can use the default value Immediate.

   The following table describes the parameters in the parameters field:

   • Secret configuration

     Parameter	Required	Description
     csi.storage.k8s.io/node-publish-secret-name	Yes	The name of the Secret that stores the AccessKey information.
     csi.storage.k8s.io/node-publish-secret-namespace	Yes	The namespace where the Secret storing the AccessKey information is located.

   • Volume configuration

     Parameter	Required	Description
     fuseType	Yes	Specifies the client to use for the mount. Must be set to ossfs2 to use the ossfs 2.0 client.
     bucket	Yes	The name of the OSS bucket you want to mount.
     path	No	The mount path of the OSS bucket, which represents the directory structure relative to the bucket root.
     url	Yes	The endpoint of the OSS bucket you want to mount. You can retrieve the endpoint from the Overview page of the bucket in the OSS console. If the bucket is mounted to a node in the same region as the bucket or the bucket can be connected to the node through a virtual private cloud (VPC), use the internal endpoint of the bucket. If the bucket is mounted to a node in a different region, use the public endpoint of the bucket. Public endpoints and internal endpoints have different formats: Format of internal endpoints: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com. Format of public endpoints: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com. Important The vpc100-oss-{{regionName}}.aliyuncs.com format for internal endpoints is deprecated.
     otherOpts	No	Enter custom parameters for the OSS volume in the format -o *** -o ***, for example, -o close_to_open=false. close-to-open: Disabled by default. If enabled, the system sends a GetObjectMeta request to OSS each time a file is opened to get the latest metadata. This ensures metadata is always up-to-date. However, in scenarios with many small file reads, frequent metadata queries can significantly increase access latency. For more optional parameters, see ossfs 2.0 mount options.

2. Create the StorageClass.

   kubectl create -f ossfs2-sc.yaml

Step 3: Create a PVC

Step 4: Create an application and mount the volume