Use ossfs 1.0 to mount an OSS bucket as persistent storage by creating a statically provisioned volume with a PV and PVC. This approach is ideal for scenarios requiring concurrent reads, infrequent random writes, and file permission modifications—a common requirement for resources like configuration files, images, or videos.
Requirements
Your cluster and CSI components (csi-plugin and csi-provisioner) must meet the following requirements:
For authentication using RAM Roles for Service Accounts (RRSA): Your cluster must be version 1.26 or later, and the CSI version must be v1.30.4 or later.
If you used the RRSA feature in a version earlier than v1.30.4, refer to [Product Change] CSI ossfs version upgrade and mount process optimization to add the required RAM Role authorization configuration.
For authentication using an AccessKey: To ensure mount stability, we recommend CSI version v1.18.8.45 or later.
To upgrade your cluster, see Manually upgrade a cluster. To upgrade the CSI components, see Upgrade CSI components.
Starting with CSI v1.30.4-*, mounting OSS static volumes depends on the csi-provisioner component.
Step 1: Choose an authentication method
To ensure that your cluster can access OSS Bucket resources securely and compliantly, you must first configure an authentication method.
RRSA authentication: Dynamically grants Pods temporary, automatically rotating RAM roles to enable fine-grained, application-level permission isolation. This method is more secure.
AccessKey authentication: Stores static, long-term keys in a Secret. This method is simpler to configure but less secure.
For clusters running version 1.26 or later, we recommend RRSA authentication to prevent workload restarts and ossfs remounts caused by AccessKey rotation.
This example assumes that the cluster and the OSS Bucket are in the same Alibaba Cloud account. To mount an OSS Bucket that belongs to another account, we recommend using RRSA authentication.
RRSA
1. Enable RRSA for your cluster
-
On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click Cluster Information.
On the Basic Information tab, in the Security and Auditing section, click Enabled to the right of RRSA OIDC. Follow the on-screen instructions to enable RRSA during off-peak hours.
When the cluster status changes from Updating to Running, RRSA is enabled.
ImportantAfter you enable RRSA, the maximum validity period of newly created ServiceAccount tokens in the cluster is limited to 12 hours.
2. Create and authorize a RAM role
Create a RAM role that your Pods can assume for RRSA-authenticated access to the OSS storage volume.
AccessKey
Create a RAM user with permissions to access the target OSS Bucket, and then obtain an AccessKey for that user.
Create a RAM user. You can skip this step if you already have a RAM user.
Go to the Create User page on the RAM Console and follow the on-screen instructions to create a RAM user. You must specify parameters such as the logon name and password.
Create an access policy.
This example follows the principle of least privilege. Create a custom policy to grant access to the target OSS Bucket. You can grant read-only or read/write permissions.
Go to the Create Policy page on the RAM Console, switch to the JSON Editor, and configure the policy script.
Read-only policy
Replace
<myBucketName>with the actual bucket name.{ "Statement": [ { "Action": [ "oss:Get*", "oss:List*" ], "Effect": "Allow", "Resource": [ "acs:oss:*:*:<myBucketName>", "acs:oss:*:*:<myBucketName>/*" ] } ], "Version": "1" }OSS read/write policy
Replace
<myBucketName>with the actual bucket name.{ "Statement": [ { "Action": "oss:*", "Effect": "Allow", "Resource": [ "acs:oss:*:*:<myBucketName>", "acs:oss:*:*:<myBucketName>/*" ] } ], "Version": "1" }If you want to create a PV in the console, you also need the
oss:ListBucketspermission.{ "Effect": "Allow", "Action": "oss:ListBuckets", "Resource": "*" }(Optional) If you use a specific CMK ID managed by KMS to encrypt OSS objects, you must also grant KMS permissions to the RAM user. For more information, see Use a specific CMK ID managed by KMS to encrypt data.
Attach the access policy to the RAM user.
Go to the Users page on the RAM Console, find the user you created, and in the Actions column, click Add Permissions.
In the Policies section, search for and select the access policy you created.
The AccessKey you create will be stored as a Secret for the PV.
Go to the Users page on the RAM Console, click the name of your RAM user, and then in the AccessKey section, click Create AccessKey.
Follow the on-screen instructions to create an AccessKey. Copy and securely store the AccessKey ID and AccessKey Secret.
Step 2: Create a PV
Create a PV to register an existing OSS bucket in your cluster.
Use RRSA
Create a file named
pv-oss-rrsa.yaml.apiVersion: v1 kind: PersistentVolume metadata: # The PV name. name: pv-oss # The PV labels. labels: alicloud-pvname: pv-oss spec: capacity: # The capacity of the persistent volume. storage: 10Gi # The access mode. accessModes: - ReadOnlyMany persistentVolumeReclaimPolicy: Retain csi: driver: ossplugin.csi.alibabacloud.com # Must be the same as the PV name (metadata.name). volumeHandle: pv-oss volumeAttributes: # Replace with your bucket name. bucket: "your-bucket-name" # The root directory or a subdirectory of the bucket to mount. path: / # The endpoint of the region where the bucket is located. url: "http://oss-cn-hangzhou-internal.aliyuncs.com" otherOpts: "-o umask=022 -o max_stat_cache_size=100000 -o allow_other" authType: "rrsa" # The RAM role that you previously created or modified. roleName: "demo-role-for-rrsa" # The signature version for OSS requests. sigVersion: "v4"Parameter
Description
storageThe capacity of the OSS persistent volume. This value is used only to match the PV with a PVC.
accessModesSpecifies the access mode. Valid values include
ReadOnlyManyandReadWriteMany.If you set this parameter to
ReadOnlyMany, ossfs mounts the OSS bucket in read-only mode.persistentVolumeReclaimPolicyThe reclaim policy of the PV. OSS persistent volumes support only
Retain. This means that when a PVC is deleted, the PV and the data in the OSS bucket are retained.driverThe driver type. When you use the Alibaba Cloud OSS CSI plugin, set this field to
ossplugin.csi.alibabacloud.com.volumeHandleMust be the same as the PV name specified in
metadata.name.bucketThe OSS bucket to mount.
pathThe version of the CSI component must be v1.14.8.32-c77e277b-aliyun or later.
The mount path relative to the root directory of the bucket. The default value is
/, which mounts the entire bucket.If the ossfs version is earlier than 1.91, the specified
pathmust exist in the OSS bucket. For more information, see New features in ossfs 1.91 and later.urlThe access endpoint for the OSS bucket.
Use an internal endpoint if your cluster nodes and the bucket are in the same region, or if a Virtual Private Cloud (VPC) connection is established.
Use a public endpoint if the mount node and the bucket are in different regions.
The following are common formats for different access endpoints:
Internal:
http://oss-{{regionName}}-internal.aliyuncs.comorhttps://oss-{{regionName}}-internal.aliyuncs.com.NoteThe internal access endpoint format
vpc100-oss-{{regionName}}.aliyuncs.comis deprecated. Switch to the new format as soon as possible.Public:
http://oss-{{regionName}}.aliyuncs.comorhttps://oss-{{regionName}}.aliyuncs.com.
otherOptsCustom parameters for the OSS persistent volume, in the format of
-o *** -o ***. For example:-o umask=022 -o max_stat_cache_size=100000 -o allow_other.authTypeSet this parameter to
rrsato use RRSA for authentication.roleNameThe RAM role that you previously created or modified.
To configure different permissions for different PVs, you can create different RAM roles and specify different
roleNamevalues in the PVs.sigVersionThe signature version for requests sent to the OSS server.
"v1"(Default): Uses OSS Signature Version 1."v4"(Recommended): Uses OSS Signature Version 4.
If the default RRSA authentication does not meet your requirements (for example, if you use a non-default ServiceAccount or a third-party OIDC), you can modify the PV configuration to specify a specific ARN or ServiceAccount. For more information, see How do I use specified ARNs or ServiceAccounts with RRSA authentication?.
Create the PV.
kubectl create -f pv-oss-rrsa.yaml
Use an AccessKey
Use kubectl
Create a file named
oss-secret.yamlto store the AccessKey from Step 1 as a Kubernetes Secret for the PV.apiVersion: v1 kind: Secret metadata: name: oss-secret # Must match the namespace of the application. namespace: default stringData: # Replace with your AccessKey ID. akId: <your AccessKey ID> # Replace with your AccessKey secret. akSecret: <your AccessKey Secret>Create the Secret.
kubectl create -f oss-secret.yamlCreate a file named
pv-oss-ram.yaml.apiVersion: v1 kind: PersistentVolume metadata: # The PV name. name: pv-oss # The PV labels. labels: alicloud-pvname: pv-oss spec: capacity: storage: 10Gi accessModes: - ReadOnlyMany persistentVolumeReclaimPolicy: Retain csi: driver: ossplugin.csi.alibabacloud.com # Must be the same as the PV name (metadata.name). volumeHandle: pv-oss # Specifies the Secret object that provides the AccessKey credentials. nodePublishSecretRef: name: oss-secret namespace: default volumeAttributes: # Replace with your bucket name. bucket: "your-bucket-name" url: "http://oss-cn-hangzhou-internal.aliyuncs.com" otherOpts: "-o umask=022 -o max_stat_cache_size=100000 -o allow_other" path: "/" # The signature version for OSS requests. "v4" is recommended. sigVersion: "v4"Parameter
Description
storageThe capacity of the OSS persistent volume. This value is used only to match the PV with a PVC.
accessModesSpecifies the access mode. Valid values include
ReadOnlyManyandReadWriteMany.If you set this parameter to
ReadOnlyMany, ossfs mounts the OSS bucket in read-only mode.persistentVolumeReclaimPolicyThe reclaim policy of the PV. OSS persistent volumes support only
Retain. This means that when a PVC is deleted, the PV and the data in the OSS bucket are retained.driverThe driver type. When you use the Alibaba Cloud OSS CSI plugin, set this field to
ossplugin.csi.alibabacloud.com.nodePublishSecretRefThe Secret that contains the AccessKey credentials for mounting the PV.
volumeHandleMust be the same as the PV name specified in
metadata.name.bucketThe OSS bucket to mount.
urlThe access endpoint for the OSS bucket.
Use an internal endpoint if your cluster nodes and the bucket are in the same region, or if a Virtual Private Cloud (VPC) connection is established.
Use a public endpoint if the mount node and the bucket are in different regions.
The following are common formats for different access endpoints:
Internal:
http://oss-{{regionName}}-internal.aliyuncs.comorhttps://oss-{{regionName}}-internal.aliyuncs.com.NoteThe internal access endpoint format
vpc100-oss-{{regionName}}.aliyuncs.comis deprecated. Switch to the new format as soon as possible.Public:
http://oss-{{regionName}}.aliyuncs.comorhttps://oss-{{regionName}}.aliyuncs.com.
otherOptsCustom parameters for the OSS persistent volume, in the format of
-o *** -o ***. For example:-o umask=022 -o max_stat_cache_size=100000 -o allow_other.pathThe version of the CSI component must be v1.14.8.32-c77e277b-aliyun or later.
The mount path relative to the root directory of the bucket. The default value is
/, which mounts the entire bucket.If the ossfs version is earlier than 1.91, the specified
pathmust exist in the OSS bucket. For more information, see New features in ossfs 1.91 and later.sigVersionThe signature version for requests sent to the OSS server.
"v1"(Default): Uses OSS Signature Version 1."v4"(Recommended): Uses OSS Signature Version 4.
Create the PV.
kubectl create -f pv-oss-ram.yaml
Console
Store the AccessKey from Step 1 as a Kubernetes Secret for the PV.
-
On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click .
Click Create from YAML and follow the on-screen instructions to create the Secret.
apiVersion: v1 kind: Secret metadata: name: oss-secret # Must match the namespace of the application. namespace: default stringData: # Replace with your AccessKey ID. akId: <your AccessKey ID> # Replace with your AccessKey secret. akSecret: <your AccessKey Secret>
-
-
On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click .
On the Persistent Volumes page, click Create. Set PV Type to OSS, configure the parameters as prompted, and then submit the configuration.
The following table describes the key parameters.
Parameter
Description
Total capacity
The capacity of the persistent volume.
Access Mode
Specifies the access mode. Valid values include
ReadOnlyManyandReadWriteMany.If you set this parameter to
ReadOnlyMany, ossfs mounts the OSS bucket in read-only mode.Access credential
The Secret that contains the AccessKey ID and AccessKey secret from Step 1.
Optional Parameters:
Custom parameters for the OSS persistent volume, in the format of
-o *** -o ***. For example:-o umask=022 -o max_stat_cache_size=100000 -o allow_other.Bucket ID:
The OSS bucket to use.
The console displays only the buckets that the configured AccessKey can access.
OSS path
The version of the CSI component must be v1.14.8.32-c77e277b-aliyun or later.
The mount path relative to the root directory of the bucket. The default value is
/, which mounts the entire bucket.If the ossfs version is earlier than 1.91, the specified
pathmust exist in the OSS bucket. For more information, see New features in ossfs 1.91 and later.Endpoint
The access endpoint for the OSS bucket.
Use an internal endpoint if your cluster nodes and the bucket are in the same region, or if a Virtual Private Cloud (VPC) connection is established.
Use a public endpoint if the mount node and the bucket are in different regions.
The following are common formats for different access endpoints:
Internal:
http://oss-{{regionName}}-internal.aliyuncs.comorhttps://oss-{{regionName}}-internal.aliyuncs.com.NoteThe internal access endpoint format
vpc100-oss-{{regionName}}.aliyuncs.comis deprecated. Switch to the new format as soon as possible.Public:
http://oss-{{regionName}}.aliyuncs.comorhttps://oss-{{regionName}}.aliyuncs.com.
By default, the console uses HTTP to access OSS over an internal network. To use HTTPS, you must create the PV by using kubectl.
Step 3: Create a PVC
Create a PersistentVolumeClaim (PVC) to request the storage capacity for your application.
kubectl
Create a file named
pvc-oss.yaml.apiVersion: v1 kind: PersistentVolumeClaim metadata: # The name of the PVC. name: pvc-oss namespace: default spec: # Configure the access mode. ReadOnlyMany indicates that ossfs mounts the OSS bucket in read-only mode. accessModes: - ReadOnlyMany resources: requests: # The requested storage capacity. This value cannot exceed the capacity of the PersistentVolume. storage: 10Gi # Add this line to explicitly specify that no StorageClass is used. storageClassName: "" selector: matchLabels: # Match the PV by its label. alicloud-pvname: pv-ossCreate the PVC.
kubectl create -f pvc-oss.yamlCheck the status of the PVC.
kubectl get pvc pvc-ossThe output confirms that the PVC is bound to the PV.
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE pvc-oss Bound pv-oss 10Gi ROX <unset> 6s
Console
-
On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click .
On the Persistent Volume Claims page, click Create, select OSS as the PVC Type, and follow the on-screen instructions to configure the parameters and submit the request.
The following table describes the key parameters.
Parameter
Description
Allocation Mode
Select Use Existing PersistentVolume.
If you have not created a persistent volume, you can set the Allocation Mode to Create PV and configure the parameters for the new volume.
Total capacity
The storage capacity of the PVC, which cannot exceed the capacity of the PersistentVolume.
Step 4: Create an application and mount a volume
To mount the volume, reference the PVC in your application.
kubectl
Create an
oss-workload.yamlfile.apiVersion: apps/v1 kind: Deployment metadata: name: oss-workload labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 ports: - containerPort: 80 volumeMounts: # The mount path in the container. - name: pvc-oss mountPath: "/data" # Configure a health check. livenessProbe: exec: command: - ls - /data initialDelaySeconds: 30 periodSeconds: 30 volumes: - name: pvc-oss persistentVolumeClaim: # Reference the PVC that you created. claimName: pvc-ossCreate the application.
kubectl create -f oss-workload.yamlVerify the mount.
Confirm that the Pods are running.
kubectl get pod -l app=nginxAccess a Pod and check the mount point.
kubectl exec -it <pod-name> -- ls /dataThe output should list the files in the OSS mount path.
Console
-
On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click .
In the upper-right corner of the Deployments page, click Create from Image.
Click Create from Image and follow the on-screen instructions to configure the application parameters.
The key parameters are described in the following table. For other parameters, you can use the default values. For more information, see Create a Deployment.
Configuration page
Parameter
Description
Basic Information
Replicas:
The number of Pod replicas for the Deployment.
Container
Image name
The address of the container image for the application, such as
anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6.Required Resources
The amount of vCPU and memory to allocate to each container.
Volumes
Click Add PVC and then configure the parameters.
Mount Source: Select the PVC that you created earlier.
Container Path: The path within the container where the volume will be mounted, such as
/data.
Labels and Annotations
Pod label
For example, set the key to 'app' and the value to 'nginx'.
Check the application deployment status.
On the Deployments page, click the name of your application. On the Pods tab, verify that the Pods have a Running status.
Step 5: Verify shared and persistent storage
Verify shared storage
To verify shared storage, create a file in one Pod and then view it in another.
Get the names of the application Pods.
kubectl get pod -l app=nginxThis example uses the Pod
oss-workload-66fbb85b67-d****:ReadWriteMany: Create the tmpfile file in the/datapath.kubectl exec oss-workload-66fbb85b67-d**** -- touch /data/tmpfileReadOnlyMany: Upload thetmpfilefile to the corresponding path in the OSS bucket by using the OSS console or by uploading a file with cp.
Check for the file in the mount path of another Pod.
Take a Pod named
oss-workload-66fbb85b67-l****with the mount pathdataas an example.kubectl exec oss-workload-66fbb85b67-l**** -- ls /data | grep tmpfileThe presence of the file in the output confirms that the Pods can share data.
tmpfileIf you do not see the expected output, make sure that your CSI component version is v1.20.7 or later.
Verify persistent storage
To verify persistent storage, delete a Pod and then check that the file remains after the Pod is automatically recreated.
Delete an application Pod. The Deployment controller automatically recreates it.
kubectl delete pod oss-workload-66fbb85b67-d****Check the status of the Pods. Wait for the new Pod to start and become
Running.kubectl get pod -l app=nginxView the files in the
/datapath.This example uses a Pod named
oss-workload-66fbb85b67-z****with the mount pathdata.kubectl exec oss-workload-66fbb85b67-z**** -- ls /data | grep tmpfileThe output shows the
tmpfilefile still exists, which confirms that data persists even after the Pod is recreated.tmpfile
Known impacts
Data integrity risks
Concurrent write consistency risk: To improve write stability, we recommend upgrading the CSI component to v1.28 or later. However, in single-file concurrent write scenarios, the OSS "overwrite upload" feature can still overwrite data. You must ensure data consistency at the application layer.
Data synchronization and accidental deletion risk: When a volume is mounted, deleting or modifying files in the mount path on an application Pod or host node directly synchronizes those changes with the source files in the OSS Bucket. To prevent accidental data deletion, we recommend enabling versioning for your OSS Bucket.
Application stability risks
OOM risk: When you perform a
readdiroperation (such as thelscommand in a shell script) on a large number of files (for example, more than 100,000, depending on the node memory) for the first time, ossfs loads all metadata at once and consumes a large amount of memory. This may cause the process to be OOM Killed and make the mount point unavailable.To mitigate this risk, we recommend mounting a subdirectory of the OSS Bucket or optimizing your directory structure.
Extended mount time: Configuring
securityContext.fsgroupin an application causes kubelet to recursively modify file permissions (chmod/chown) when mounting a volume. If the volume contains a large number of files, the mount time is significantly extended, which can cause severe Pod startup delays.If you must configure this parameter but want to reduce the mount time, see Increased mount time for OSS volumes.
Key invalidation risk (AccessKey authentication): If an AccessKey referenced by a PV becomes invalid or its permissions change, the associated application immediately loses access.
To restore access, you must update the credentials in the Secret and restart the application Pod to force a remount. This causes a service interruption, so perform this operation during a maintenance window. For more information, see Solutions.
Cost risks
Part costs: When uploading files larger than 10 MB, ossfs performs a multipart upload, splitting the file into parts. If an upload is unexpectedly interrupted, for example, due to an application restart, you must manually delete the parts or delete them by using lifecycle rules to avoid storage costs from incomplete parts.
Resource cleanup
To avoid unexpected charges and ensure data security, follow these steps to release unneeded resources.
Delete the workload
Action: Delete all applications, such as Deployments and StatefulSets, that use the PVC. This action stops the Pods and unmounts the volume.
Example command:
kubectl delete deployment <your-deployment-name>
Delete the PVC
Action: Delete the PVC associated with your application. For an OSS static PV, the reclaim policy (
persistentVolumeReclaimPolicy) only supportsRetain. Therefore, when you delete the PVC, its bound PV enters theReleasedstate, but the backend OSS Bucket and its data are retained.Example command:
kubectl delete pvc <your-pvc-name>
Delete the PV
Action: Delete the
ReleasedPV. This action only removes the PV resource from Kubernetes and does not delete the data in the backend OSS Bucket.Example command:
kubectl delete pv <your-pv-name>
Delete the Secret (Optional)
Action: Delete the Secret used to mount the OSS volume. This action only removes the Secret from Kubernetes and does not delete any data from the backend OSS Bucket.
Example command:
kubectl delete secret <your-secret-name>
Related documents
Manage OSS volumes by using CNFS to improve their performance and QoS. For more information, see Manage the lifecycle of OSS volumes.
Enable server-side encryption to protect sensitive data at rest in OSS. For more information, see Encrypt ossfs 1.0 volumes.
For frequently asked questions about ossfs and OSS, see ossfs 1.0 (default) and ossfs 1.0 volume FAQ.
Enable container storage monitoring and configure alerts to promptly detect volume anomalies or performance bottlenecks.
ossfs 1.0 provides stronger data consistency for random and concurrent writes than ossfs 2.0. However, ossfs 2.0 offers better performance for sequential reads and writes.