Deploy and manage the ack-kserve component in an ACK cluster - Container Service for Kubernetes

KServe is an open source project that provides declarative APIs to simplify the process of deploying and managing machine learning models on Kubernetes. Alibaba Cloud provides the ack-kserve component that is optimized based on open source KServe and integrated with the Alibaba Cloud ecosystem, including the the storage, logging, and network capabilities provided by Alibaba Cloud. ack-kserve simplifies KServe deployment and O&M in Container Service for Kubernetes (ACK) clusters. This topic describes how to deploy and manage ack-kserve in an ACK cluster.

Prerequisites

An ACK managed Pro cluster that runs Kubernetes 1.22 or later is created. For more information, see Create an ACK managed cluster.
The NGINX Ingress controller is installed. For more information, see Manage the NGINX Ingress controller.

Step 1: Install the cert-manager component

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
In the upper-left corner of the Helm page, click Deploy. In the Basic Information step in the Deploy panel, specify the Application Name parameter, select cert-manager in the Chart section, and then click Next.
In the Parameters step, check the value of the Chart Version parameter and the information displayed in the Parameters field and click OK.
After the cert-manager component is deployed, you can view information about the cert-manager component on the Helm page.

Step 2: Install the ack-kserve component

By default, the ack-kserve component is deployed in RawDeployment mode and integrated with the Nginx Ingress controller.

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
In the upper-left corner of the Helm page, click Deploy. In the Basic Information step in the Deploy panel, specify the Application Name parameter, select ack-kserve in the Chart section, and then click Next.
In the Parameters step, check the value of the Chart Version parameter and the information displayed in the Parameters field and click OK.
After the cert-manager component is deployed, you can view information about the ack-kserve component on the Helm page.
Check whether the ack-kserve component is running.
Run the following command to query the status of the ack-kserve pod:
```
kubectl get pod -n kserve
```
If running is returned for the STATUS parameter in the output, the ack-kserve component is installed.

(Optional) Step 3: View or update the ack-kserve ️component

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
View the details of the ack-kserve component.
On the Helm page, find the ack-kserve component and click View Details in the Actions column. On the details page of the ack-kserve component, you can view the information on the Basic Information, Parameters, and History tabs.
Update the ack-kserve component.
On the Helm page, find the ack-kserve component and click Update in the Actions column. In the Update Release panel, change the version of the component and modify the parameter settings.

(Optional) Step 4: Delete resources and uninstall components

Delete the KServe custom resources and relevant CustomResourceDefinitions (CRDs) before you uninstall the ack-kserve component. This prevents resource waste.
Important
Before you delete the custom resources and CRDs, make sure that they are no longer required by your business. If you delete a CRD, the relevant custom resources are also deleted. Custom resources cannot be restored after they are deleted.
1. Before you delete all KServe custom resources, make sure that they are no longer required. You can run the following commands to delete the custom resources:
```
# View all custom resources in the cluster. 
kubectl get isvc --all-namespaces

# Save all custom resources in the cluster. 
kubectl get isvc --all-namespaces -oyaml > isvc.yaml.bak

# Delete the custom resources after you make sure that you no longer require them. 
kubectl delete isvc --all
```
2. Delete the KServe CRDs in the cluster.
  Before you delete a CRD, you must delete all relevant custom resources. Otherwise, you fail to be delete the CRD.
```
kubectl delete crd clusterservingruntimes.serving.kserve.io
kubectl delete crd clusterstoragecontainers.serving.kserve.io
kubectl delete crd inferencegraphs.serving.kserve.io
kubectl delete crd inferenceservices.serving.kserve.io
kubectl delete crd predictors.serving.kserve.io
kubectl delete crd servingruntimes.serving.kserve.io
kubectl delete crd trainedmodels.serving.kserve.io
```
Uninstall the ack-kserve component.
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
3. On the Helm page, find the ack-kserve component and click Delete in the Actions column. In the Delete dialog box, click OK.
Uninstall the cert-manager component.
Warning
Before you uninstall cert-manager, make sure that cert-manager is not used by other components in the cluster. This prevents service interruptions caused by the uninstallation of cert-manager.
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
3. On the Helm page, find the cert-manager component and click Delete in the Actions column. In the Delete dialog box, click OK.

Run the following commands to delete the relevant CRDs used by cert-manager:

kubectl delete crd certificaterequests.cert-manager.io
kubectl delete crd certificates.cert-manager.io
kubectl delete crd challenges.acme.cert-manager.io
kubectl delete crd clusterissuers.cert-manager.io
kubectl delete crd issuers.cert-manager.io
kubectl delete crd orders.acme.cert-manager.io

FAQ

Issue: The following error message appears when the ack-kserve component is being installed: failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": tls: failed to verify certificate: x509: certificate signed by unknown authority.

Cause: The ack-kserve component is highly dependent on the cert-manager component. If it is not installed or is not ready in your cluster, the preceding error occurs when you install the ack-kserve component.

Solution:

Run the following command to check whether the cert-manager component is installed in your cluster:
```
kubectl get crd |grep certificates.cert-manager.io
```
If the following output is returned, the cert-manager component is installed in the cluster.
```
certificates.cert-manager.io                         2024-05-06T07:09:17Z
```
If no CRD of the cert-manager component is available in your cluster, install the cert-manager component. For more information, see the Step 1: Install the cert-manager component section of this topic.
Run the following command to check whether the cert-manager component is ready:
```
kubectl -n cert-manager get po
```
If the following output is returned, the pods of the cert-manager component are ready.
```
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-7f4bb44d5b-jrrfn              1/1     Running   0          23h
cert-manager-cainjector-79544456cc-qp5pp   1/1     Running   0          23h
cert-manager-webhook-f74ccb647-7m5dt       1/1     Running   0          23h
```
If all pods are in the Ready state, you can uninstall the ack-kserve component and then reinstall it. For more information, see the "Uninstall the ack-kserve component" section of this topic.