This topic describes typical scenarios in which an Object Storage Service (OSS) bucket is needed for creating a stateful service, and the procedure for how to use the bucket.
Scenarios and method
Alibaba Cloud OSS provides massive, secure, low-cost, and highly reliable cloud storage services. An OSS bucket can be mounted to multiple pods.
- Disk I/O performance requirements are low.
- Shared services such as files, figures, and short videos are to be configured.
- Manually create a bucket.
- Obtain the AccessKey ID and AccessKey Secret pair.
- Manually create a Persistent Volume (PV) and a Persistent Volume Claim (PVC).
- Upgrading a Kubernetes cluster of Alibaba Cloud Container Service causes kubelet and the OSSFS driver to restart. As a result, the OSS directory becomes unavailable. In this case, the pods that use the OSS bucket must be recreated. We recommend that you add health check settings in the YAML file of your application. If you add health check settings for your application, the pods will be automatically restarted to remount the OSS bucket when the OSS directory within your container becomes unavailable.
- If you use a Kubernetes cluster of the latest version, the preceding issue does not affect you.
Create a PV
- Create a pv-oss.yaml file.
apiVersion: v1 kind: PersistentVolume metadata: name: pv-oss labels: alicloud-pvname: pv-oss spec: capacity: storage: 5Gi accessModes: - ReadWriteMany storageClassName: oss flexVolume: driver: "alicloud/oss" options: bucket: "docker" ////Replace this value with your bucket name. url: "oss-cn-hangzhou.aliyuncs.com" ////Replace this value with your URL. akId: "***" ////Replace this value with your AccessKey ID. akSecret: "***" ////Replace this value with your AccessKey Secret. otherOpts: "-o max_stat_cache_size=0 -o allow_other" ////Replace this value with your specified otherOpts value.
alicloud-pvname: indicates a PV name. This parameter value must be used in the
selectorfield of the PVC associated with the PV.
bucket: indicates a bucket name. Only buckets can be mounted to a Kubernetes cluster. The sub-directories or files in a bucket cannot be mounted to any Kubernetes cluster.
url: indicates a domain name used to access the OSS bucket, namely, an endpoint. For more information, see Regions and endpoints. You can also view the endpoint of the created OSS bucket in the OSS console. That is, log on to the OSS console, select the target bucket, and view Endpoint in the Domain Names area.
akId: indicates your AccessKey ID. In the Container Service console, click in the upper-right corner. For a primary account, select accesskeys. For a RAM user, select AccessKey. Then, you can create your AccessKey ID and AccessKey Secret.
akSecret: indicates your AccessKey Secret. Use the same method to obtain this parameter value as that to obtain the value of the
otherOpts: indicates custom parameters for mounting the OSS bucket. Set this parameter in the format of
-o *** -o ***. For more information, see FAQ.
- Run the following command to create a PV:
$ kubectl create -f pv-oss.yaml
Create a PVC
Create a PVC for the OSS bucket. Specifically, you need to set the
selector field to filter for the created PV so that you can associate the PVC with the correct
PV. Set the
storageClassName parameter to associate the PVC with only the PV of the OSS type.
- Create a pvc-oss.yaml file.
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: pvc-oss spec: accessModes: - ReadWriteMany storageClassName: oss resources: requests: storage: 5Gi selector: matchLabels: alicloud-pvname: pv-oss
- Run the following command to create a PVC:
$ kubectl create -f pvc-oss.yaml
Create an application
- Create an oss-static.yaml file.
apiVersion: apps/v1 kind: Deployment metadata: name: oss-static labels: app: nginx spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx ports: - containerPort: 80 volumeMounts: - name: pvc-oss mountPath: "/data" - name: pvc-oss mountPath: "/data1" livenessProbe: exec: command: - sh - -c - cd /data initialDelaySeconds: 30 periodSeconds: 30 volumes: - name: pvc-oss persistentVolumeClaim: claimName: pvc-ossNote For more information about
livenessProbe, see Use OSS volumes.
- Run the following command to create a deployment:
$ kubectl create -f oss-static.yaml d
Verify that data in the OSS bucket are stored persistently
- Run the following command to view the pod in which the created deployment resides:
$ kubectl get pod NAME READY STATUS RESTARTS AGE oss-static-66fbb85b67-dqbl2 1/1 Running 0 1h
- Run the following command to view the files in the /data path:
$ kubectl exec oss-static-66fbb85b67-dqbl2 ls /data | grep tmpfileNote The /data path is empty.
- Run the following command to create a file named tmpfile in the /data path:
$ kubectl exec oss-static-66fbb85b67-dqbl2 touch /data/tmpfile
- Run the following command to view the file in the /data path:
$ kubectl exec oss-static-66fbb85b67-dqbl2 ls /data | grep tmpfile tmpfile
- Run the following command to remove the pod named oss-static-66fbb85b67-dqbl2:
$ kubectl delete pod oss-static-66fbb85b67-dqbl2 pod "oss-static-66fbb85b67-dqbl2" deleted
- Open another kubectl interface and run the following command to view the process in
which the preceding pod is removed and a new pod is created by Kubernetes:
$ kubectl get pod -w -l app=nginx NAME READY STATUS RESTARTS AGE oss-static-66fbb85b67-dqbl2 1/1 Running 0 78m oss-static-66fbb85b67-dqbl2 1/1 Terminating 0 78m oss-static-66fbb85b67-zlvmw 0/1 Pending 0 <invalid> oss-static-66fbb85b67-zlvmw 0/1 Pending 0 <invalid> oss-static-66fbb85b67-zlvmw 0/1 ContainerCreating 0 <invalid> oss-static-66fbb85b67-dqbl2 0/1 Terminating 0 78m oss-static-66fbb85b67-dqbl2 0/1 Terminating 0 78m oss-static-66fbb85b67-dqbl2 0/1 Terminating 0 78m oss-static-66fbb85b67-zlvmw 1/1 Running 0 <invalid>
- Run the following command to view the pod created by Kubernetes:
$ kubectl get pod NAME READY STATUS RESTARTS AGE oss-static-66fbb85b67-zlvmw 1/1 Running 0 40s
- Run the following command to verify that the created file named tmpfile in the /data path has not been removed, indicating that data in the OSS bucket can be stored persistently:
$ kubectl exec oss-static-66fbb85b67-zlvmw ls /data | grep tmpfile tmpfile