Container Service for Kubernetes:Mount a static OSS volume by using ossfs 2.0 in ACK

1. Create a file named ossfs2-pv.yaml with the following content. This PV definition tells Kubernetes how to access your OSS bucket using the RRSA role.

   The following PV mounts an OSS bucket named cnfs-oss-test as a 20 GB read-only file system for pods in the cluster to use.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv-ossfs2  # PV name
   spec:
     capacity:
       storage: 20Gi  # Define the volume capacity (this value is only for matching PVCs)
     accessModes:  # Access mode
       - ReadOnlyMany
     persistentVolumeReclaimPolicy: Retain
     csi:
       driver: ossplugin.csi.alibabacloud.com
       volumeHandle: pv-ossfs2  # Must be the same as the PV name (metadata.name)
       volumeAttributes:
         fuseType: ossfs2
         bucket: cnfs-oss-test # The name of the OSS Bucket
         path: /subpath  # The subdirectory to mount. Leave it empty to mount the root directory.
         url: oss-cn-hangzhou-internal.aliyuncs.com  # The endpoint of the region where the OSS Bucket is located
         otherOpts: "-o close_to_open=false"
         authType: "rrsa"
         roleName: "demo-role-for-rrsa"  # The RAM role created earlier

   • Parameters in volumeAttributes:

     Parameter	Required	Description
     fuseType	Yes	Specifies the client to use for the mount. Must be set to ossfs2 to use the ossfs 2.0 client.
     bucket	Yes	The name of the OSS bucket you want to mount.
     path	No	The subdirectory within the OSS bucket to mount. If not specified, the root of the bucket will be mounted by default.
     url	Yes	The access endpoint for the OSS bucket. Use an internal endpoint if your cluster nodes and the bucket are in the same region, or if a Virtual Private Cloud (VPC) connection is established. Use a public endpoint if the mount node and the bucket are in different regions. The following are common formats for different access endpoints: Internal: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com. The internal access endpoint format vpc100-oss-{{regionName}}.aliyuncs.com is deprecated. Switch to the new format as soon as possible. Public: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com.
     otherOpts	No	Additional mount options passed to the OSS volume, specified as a string of space-separated flags in the format -o *** -o ***. For example, -o close_to_open=false. The close-to-open option controls metadata consistency. When set to false (default), metadata is cached to improve performance for small file reads. When set to true, the client fetches fresh metadata from OSS by sending a GetObjectMeta request every time a file is opened, ensuring real-time consistency at the cost of increased latency and API calls. For more optional parameters, see ossfs 2.0 mount options.
     authType	Yes	Set to rrsa to declare that the RRSA authentication method is used.
     roleName	Yes	Set to the name of the RAM role you created or modified. To assign different permissions for different PVs, create a separate RAM role for each permission set. Then, in each PV's definition, use the roleName parameter to associate it with the corresponding role.

     To use specified ARNs or a ServiceAccount with the RRSA authentication method, see How do I use specified ARNs or a ServiceAccount with the RRSA authentication method?.

2. Apply the PV definition.

   kubectl create -f ossfs2-pv.yaml

3. Verify the PV status.

   kubectl get pv pv-ossfs2

   The output shows that the PV status is Available.

   NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
   pv-ossfs2   20Gi       ROX            Retain           Available                          <unset>                          15s

Step 4: Create a PVC

Create a PVC to declare the persistent storage capacity required by the application.

You can now mount the PVC to an application.

1. Create a file named ossfs2-test.yaml to define your application.

   The following YAML template creates a StatefulSet that contains one pod. The pod requests storage resources through a PVC named pvc-ossfs2 and mounts the volume to the /data path.

   YAML template

   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: ossfs2-test
     namespace: default
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: ossfs2-test
     template:
       metadata:
         labels:
           app: ossfs2-test
       spec:
         containers:
         - name: nginx
           image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
           ports:
           - containerPort: 80
           volumeMounts:
           - name: pvc-ossfs2
             mountPath: /data
         volumes:
           - name: pvc-ossfs2
             persistentVolumeClaim:
               claimName: pvc-ossfs2  # The PVC to reference

2. Deploy the application.

   kubectl create -f ossfs2-test.yaml

3. Check the pod deployment status.

   kubectl get pod -l app=ossfs2-test

   Expected output:

   NAME            READY   STATUS    RESTARTS   AGE
   ossfs2-test-0   1/1     Running   0          12m

4. Verify that the application can access the data in OSS.

   kubectl exec -it ossfs2-test-0 -- ls /data

   The output should show the data in the OSS mount path.

Create a RAM user and grant permissions

1. Create a RAM user. You can skip this step if you have an existing RAM user. For more information about how to create a RAM user, see Create a RAM user.

2. Create a custom policy to grant OSS access permissions to the RAM user. For more information, see Create custom policies.

   Select the read-only policy or read-write policy based on your business requirements. Replace mybucket with the name of the bucket you created.

   • Policy that provides read-only permissions on OSS

     Click to view policy content

     {
         "Statement": [
             {
                 "Action": [
                     "oss:Get*",
                     "oss:List*"
                 ],
                 "Effect": "Allow",
                 "Resource": [
                     "acs:oss:*:*:mybucket",
                     "acs:oss:*:*:mybucket/*"
                 ]
             }
         ],
         "Version": "1"
     }

   • Policy that provides read-write permissions on OSS

     Click to view policy content

     {
         "Statement": [
             {
                 "Action": "oss:*",
                 "Effect": "Allow",
                 "Resource": [
                     "acs:oss:*:*:mybucket",
                     "acs:oss:*:*:mybucket/*"
                 ]
             }
         ],
         "Version": "1"
     }

3. Optional. If the objects in the OSS bucket are encrypted by using a specified customer master key (CMK) in Key Management Service (KMS), you need to grant KMS access permissions to the RAM user. For more information, see Encryption.

4. Grant OSS access permissions to the RAM user. For more information, see Grant permissions to a RAM user.

5. Create an AccessKey pair for the RAM user. For more information, see Create an AccessKey pair.

Create a Secret to store the AccessKey credentials

Run the following command to create a Secret for OSS authentication. Replace akId and akSecret with your actual credentials.

kubectl create -n default secret generic oss-secret --from-literal='akId=xxxxxx' --from-literal='akSecret=xxxxxx'

1. Create a file named ossfs2-pv-ak.yaml with the following content. This PV definition includes a nodePublishSecretRef that points to the Secret you created.

   The following PV mounts an OSS bucket named cnfs-oss-test as a 20 GB read-only file system for pods in the cluster to use.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv-ossfs2  # PV name
   spec:
     capacity:
       storage: 20Gi  # Define the volume capacity (this value is only for matching PVCs)
     accessModes:  # Access mode
       - ReadOnlyMany
     persistentVolumeReclaimPolicy: Retain
     csi:
       driver: ossplugin.csi.alibabacloud.com
       volumeHandle: pv-ossfs2   # Must be the same as the PV name (metadata.name)
       # Use the Secret created earlier
       nodePublishSecretRef:
         name: oss-secret  # The name of the Secret that stores the AccessKey information
         namespace: default  # The namespace where the Secret is located
       volumeAttributes:
         fuseType: ossfs2
         bucket: cnfs-oss-test  # The name of the OSS Bucket
         path: /subpath  # The subdirectory to mount. Leave it empty to mount the root directory.
         url: oss-cn-hangzhou-internal.aliyuncs.com  # The endpoint of the region where the OSS Bucket is located
         otherOpts: "-o close_to_open=false"

   • Parameters in nodePublishSecretRef:

     Parameter	Required	Description
     name	Yes	The name of the Secret that stores the AccessKey information.
     namespace	Yes	The namespace where the Secret storing the AccessKey information is located.

   • Parameters in volumeAttributes:

     Parameter	Required	Description
     fuseType	Yes	Specifies the client to use for the mount. Must be set to ossfs2 to use the ossfs 2.0 client.
     bucket	Yes	The name of the OSS bucket you want to mount.
     path	No	The mount path of the OSS bucket, which represents the directory structure relative to the bucket root.
     url	Yes	The endpoint of the OSS bucket you want to mount. You can retrieve the endpoint from the Overview page of the bucket in the OSS console. If the bucket is mounted to a node in the same region as the bucket or the bucket can be connected to the node through a virtual private cloud (VPC), use the internal endpoint of the bucket. If the bucket is mounted to a node in a different region, use the public endpoint of the bucket. Public endpoints and internal endpoints have different formats: Format of internal endpoints: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com. Format of public endpoints: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com. Important The vpc100-oss-{{regionName}}.aliyuncs.com format for internal endpoints is deprecated.
     otherOpts	No	Enter custom parameters for the OSS volume in the format -o *** -o ***, for example, -o close_to_open=false. close-to-open: Disabled by default. If enabled, the system sends a GetObjectMeta request to OSS each time a file is opened to get the latest metadata. This ensures metadata is always up-to-date. However, in scenarios with many small file reads, frequent metadata queries can significantly increase access latency. For more optional parameters, see ossfs 2.0 mount options.

2. Apply the PV definition.

   kubectl create -f ossfs2-pv.yaml

3. Check the PV status.

   kubectl get pv pv-ossfs2

   The following output shows that the PV is Available:

   NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
   pv-ossfs2   20Gi       ROX            Retain           Available                          <unset>                          15s

You can now mount the PVC to an application.

1. Create a file named ossfs2-test.yaml to define your application.

   The following YAML template creates a StatefulSet that contains one pod. The pod requests storage resources through a PVC named pvc-ossfs2 and mounts the volume to the /data path.

   YAML template

   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: ossfs2-test
     namespace: default
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: ossfs2-test
     template:
       metadata:
         labels:
           app: ossfs2-test
       spec:
         containers:
         - name: nginx
           image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
           ports:
           - containerPort: 80
           volumeMounts:
           - name: pvc-ossfs2
             mountPath: /data
         volumes:
           - name: pvc-ossfs2
             persistentVolumeClaim:
               claimName: pvc-ossfs2  # The PVC to reference

2. Deploy the application.

   kubectl create -f ossfs2-test.yaml

3. Check the pod deployment status.

   kubectl get pod -l app=ossfs2-test

   Expected output:

   NAME            READY   STATUS    RESTARTS   AGE
   ossfs2-test-0   1/1     Running   0          12m

4. Verify that the application can access the data in OSS.

   kubectl exec -it ossfs2-test-0 -- ls /data

   The output should show the data in the OSS mount path.