All Products
Search
Document Center

Container Service for Kubernetes:View control plane component dashboards in ACK Pro clusters

Last Updated:Apr 15, 2024

This topic describes how to view control plane component dashboards in Container Service for Kubernetes (ACK) Pro clusters. This topic also provides best practices for accessing control plane components.

Prerequisites

  • The control plane component monitoring feature applies to ACK Pro clusters whose version is 1.16 or later.

  • Application Real-Time Monitoring Service (ARMS) is activated. For more information, see Activate ARMS.

  • The ack-arms-prometheus component is installed. For more information, see Manage components.

View component plane component dashboards

  1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

  2. On the Clusters page, click the name of the cluster that you want to manage. In the left-side navigation pane, choose Operations > Prometheus Monitoring.

  3. On the Prometheus Monitoring page, click the Key Component Monitoring tab.

    You can view the ACK Pro API server, ACK Pro ETCD, ACK Pro Scheduler, ACK Pro Cloud Controller Manager, and ACK Pro Kube Controller Manager dashboards on this page.

    Note

    To use the latest Prometheus dashboards, update ack-arms-prometheus to the latest version from the Add-ons page in the console. For more information, see Manage components.

Best practices for accessing control plane components

We recommend that you follow our best practice rules when you access control plane components in an ACK Pro cluster that contains more than 100 nodes and a large amount of Kubernetes resources. This helps improve the stability of the cluster.

  • Retrieve data from the API server by using Informer or Lister. This helps reduce the loads of the API server and etcd.

  • If you want to list all data, we recommend that you add resourceVersion=0 to the request to retrieve data from the API server cache in case the etcd is overloaded. If you want to retrieve data from the etcd, we recommend that you add the limit option to limit the number of entries returned per page.

  • Use Protobuf as the API serialization protocol, which requires less memory resources and data transfer compared with JSON. For more information, see Alternate representations of resources. The following code block shows an example:

    kubeConfig, err := clientcmd.BuildConfigFromFlags(s.Master, s.Kubeconfig)
    if err != nil {
        return nil, err
    }
    kubeConfig.AcceptContentTypes = strings.Join([]string{runtime.ContentTypeProtobuf, runtime.ContentTypeJSON}, ",")
    kubeConfig.ContentType = runtime.ContentTypeProtobuf
    client, err := clientset.NewForConfig(restclient.AddUserAgent(kubeConfig, "content-type-example"))
    ...
  • Delete idle Kubernetes resources, such as ConfigMaps, Secrets, and persistent volume claims (PVCs), at the earliest opportunity. These idle resources can create pending pods. When the number of pending pods in the cluster exceeds 1000, the stability of kube-apsierver, kube-controller-manager, and kube-scheduler is affected.

  • Pay close attention to the resource usage of the control plane components, especially the CPU and memory utilization metrics, to prevent out-of-memory errors due to continuous high resource usage. If the resource usage remains high, we recommend that you delete invalid resources, optimize the actions of the client, and separate the workloads in the cluster.

  • Some open source components can greatly increase the loads of the control plane. You can find optimization solutions on the official websites of these components. For example, Argo Workflows provides a solution to resolve the issue of overwhelmed Kubernetes API when Argo is busy. For more information, see Running At Massive Scale.

References

Control plane component

Dashboard

Reference introduction

Link

kube-apiserver

ACK Pro APIServer

Describes the metrics supported by kube-apiserver, provides usage notes for the dashboards of kube-apiserver, and suggests how to troubleshoot common metric anomalies.

kube-apiserver

cloud-controller-manager

ACK Pro Cloud Controller Manager

Describes the metrics supported by cloud-controller-mananger, provides usage notes for the dashboards of cloud-controller-manager, and suggests how to troubleshoot common metric anomalies.

Monitor cloud-controller-mananger

etcd

ACK Pro ETCD

Describes the metrics supported by etcd, provides usage notes for the dashboards of etcd, and suggests how to troubleshoot common metric anomalies.

Monitor etcd

kube-controller-manager

ACK Pro Kube Controller Manager

Describes the metrics supported by kube-controller-manager and provides usage notes for the dashboards of kube-controller-manager.

Monitor kube-controller-manager

kube-scheduler

ACK Pro Scheduler

Describes the metrics supported by kube-scheduler, provides usage notes for the dashboards of kube-scheduler, and suggests how to troubleshoot common metric anomalies.

Monitor kube-scheduler

Custom Prometheus monitoring and alerting

Custom dashboard name

Describes how to collect the metrics from kube-apiserver, etcd, kube-scheduler, cloud-controller-manager, and kube-controller-manager in an ACK Pro cluster to a self-managed Prometheus system and provides recommended alerting configurations.

Use a self-managed Prometheus system to configure monitoring and alerting for control plane components