By Chen Xianlu (Yaolu), Senior Technical Expert at Alibaba
The Kubernetes API programming paradigm is also known as CustomResourceDefinition (CRD). A CRD is a type of custom resource. What is CRD used for? This article explores the concept of CRD from the perspective of requirements analysis.
To start, let's take a look at the sources of requirements for the Kubernetes API programming paradigm.
The Kubernetes API programming paradigm is also known as CRD. A CRD is a type of custom resource.
The requirements for custom resources increase with the widespread use of Kubernetes. Kubernetes users require capabilities that go beyond the grouping of subresources. Specifically, users want a type of custom resource able to group subresources. Kubernetes provides CRD to facilitate scale up and the use of its native resources.
First, let's see what CRD is.
CRD was introduced in Kubernetes 1.7, allowing you to add custom Kubernetes object resources as needed. The custom object resources are also native resources of Kubernetes, just like the pods and Deployments provided by Kubernetes. The Kubernetes API server processes custom object resources in the same way as the resources stored in etcd.
Like the native built-in resources of Kubernetes, custom resources can be created and viewed by using kubectl, and their access is regulated by role-based access control (RBAC) and security functions. You can develop custom controllers to detect or perform custom resource changes.
Now let's take a look at a simple CRD use case. The following figure shows the definition of a CRD.
The apiVersion field declares a CRD or defines its schema.
The kind field indicates that the resource type is CRD. The name field indicates a custom name of the CRD. We recommend that you set the name in the format top-level domain name.xxx.APIGroup
, such as foos.samplecontroller.k8s.io.
spec is used to specify the group and version of the CRD. For example, when you create a pod or a Deployment, you may specify the group as apps/v1 or apps/v1beta1. Similarly, you must specify a group when creating a CRD.
The following figure shows the sample code for creating the CRD defined in the preceding figure.
The following code defines a CRD with validation:
This CRD is more complex. Let's take a look at the validation part.
Validation is defined as openAPIV3Schema. The spec part defines specific resources. For example, replicas are defined as integer-type resources, with the minimum number of 1 and maximum number of 10. If you enter replicas not of the integer type or enter -1 or a value greater than 10 when using this CRD, then the CRD is not committed to the Kubernetes API server. In this case, the Kubernetes API server returns an error indicating that the predefined parameter settings are not met.
The following code defines a CRD with the status field:
After you deploy Deployments or pods, you may want to check the deployment status or whether the Deployments or pods have been updated. To do so, you can add the status field. The status field is unavailable in Kubernetes versions earlier than 1.12.
The status field is a subresource of a CRD. Updating this field does not trigger the redeployment of Deployments or pods. If you modify the spec part of a Deployment or a pod, another Deployment or pod is created. However, the status resource is not re-created because it only indicates the overall status of the current pod. In the preceding CRD, subresources.status is simple and in the key:value format. You can enter a custom status in {}.
Take the status field of a Deployment as an example. It includes availableReplicas and current status, such as the latest version and previous version. You can define a CRD to indicate the current status.
The following shows how to create a CRD.
There are two resources: crd.yaml
and example-foo.yaml
.
First, we will create a CRD schema to indicate the usage of the CRD to the Kubernetes API server. Run the kuberctl create -f crd.yaml
command to create a CRD.
Run the kuberctl get crd
command to check that the CRD was created.
Then, create the resource kuberctl create -f example-foo.yaml
as follows.
Run the kubectl get foo example-foo -o yaml
command to view resource details.
This is a Foo resource. The spec part is as previously defined. The selected part is typical of almost all Kubernetes metadata. Unlike the built-in resources of Kubernetes, the Foo resource is a custom resource created in a way similar to creating a pod. The Foo resource is used in the same way and provides the same user experience as the built-in resources of Kubernetes.
After it is created, a CRD is stored in etcd by the Kubernetes API server. A controller ensures that complex operations can be performed based on the CRD and its schema.
Kubernetes provides controllers to scale up or control declarative Kubernetes resources in pluggable mode. Controllers regulate a majority of Kubernetes resources. For instance, Deployments are deployed through kube-controller-manager.
For example, when receiving a request to declare a Deployment with replicas and two pods, kube-controller-manager, while monitoring etcd, creates two pod-mapped replicas and monitors the status changes of the pods in real-time. If the pods are changed, rolled back, restarted, or fail, kube-controller-manager acts accordingly.
Therefore, controllers regulate the final statuses of Kubernetes resources.
After creating a CRD, you need to create a controller to implement the CRD. For example, the preceding Foo resource is intended to create a Deployment with one replica. To implement this CRD, create a controller used to create a Deployment.
Take kube-controller-manager as an example.
The left part of the preceding figure shows the informer used to watch the Kubernetes API server, which monitors the creation, update, and deletion of all resources in etcd. The informer provides two methods: ListFunc and WatchFunc.
kuberctl get pods
.After receiving the object request, the informer calls functions (such as AddFunc
, UpdateFunc
, and DeleteFunc
in the preceding figure) and queues them up in the key format namespace/name
, where name indicates the name of the resource. For example, the Foo resource created in the default namespace is named in the key format default/example-foo
. The controller fetches an object from the queue and performs related operations.
The following figure shows the workflow of the controller.
The Kubernetes API server pushes an event such as Added, Updated, or Deleted. The event enters the controller's loop ListAndWatch(), which has a first-in-first-out (FIFO) queue. The event is popped out through Pop() and sent to the appropriate handler. The handler sends the event to a function, such as Add(), Update(), or Delete().
A function usually has multiple workers. When a multi-worker function receives several objects, the controller starts 5 or 10 workers simultaneously to process the objects in parallel. Each worker can process different object instances.
After an object is created, the worker discards the key to indicate that object processing is complete. If a problem occurs, the worker returns an error, prints an event, and queues up the key again so that the next worker can receive the key and process it.
Let's summarize what we have learned in this article:
Getting Started with Kubernetes | Stateful Application Orchestration with StatefulSets
Getting Started with Kubernetes | Operator and Operator Framework
502 posts | 48 followers
FollowAlibaba Cloud Native Community - August 25, 2022
Alibaba Clouder - May 6, 2019
Alibaba Developer - June 17, 2020
Alibaba Clouder - February 14, 2020
Alibaba Clouder - March 19, 2019
Alibaba Cloud_Academy - July 24, 2023
502 posts | 48 followers
FollowAlibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn MoreProvides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resources
Learn MoreAccelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn MoreA secure image hosting platform providing containerized image lifecycle management
Learn MoreMore Posts by Alibaba Cloud Native Community