This topic describes how to create a Kubernetes event center. After you create a Kubernetes event center, you can view event statistics, query event details, view pod lifecycle, configure alerts, and perform custom queries.

Background information

Kubernetes event centers record the status changes of Kubernetes clusters. A status change is recorded when you create, run, or delete a pod, or when a component exception occurs. Kubernetes event centers aggregate all events in Kubernetes clusters in real time and provides storage, search, analysis, visualization, and alert capabilities.

Billing

After a Logstore is associated with a Kubernetes event center, you can use the Logstore free of charge if you store event data in the Logstore for 90 days. During this period, you can write 256 MB of data each day to the Logstore. The data size is equivalent to 250,000 events. In most cases, each online Kubernetes cluster generates 1,000 events each day. By default, events are retained for 90 days. Therefore, you can use the Kubernetes event center free of charge if you do not extend the event retention period. For example:
  • If you do not extend the event retention period and the Kubernetes cluster generates 1,000 events each day, you can use the Kubernetes event center free of charge.
  • If you extend the event retention period to 105 days and the Kubernetes cluster generates 1,000 events each day, you will be charged at a rate of USD 0.014 per day after 90 days. For more information, see Pay-as-you-go.

Step 1: Create an event center

  1. Log on to the Log Service console.
  2. In the Log Application section, click Start under K8s Event Center.
  3. On the Event Centers page, click Add.
  4. In the Add Event Center dialog box, set the required parameters.
    • Select Existing Project, and then select a project from the Project drop-down list.
    • Select Kubernetes Cluster, and then select a Kubernetes cluster from the ClusterId drop-down list. If you use this method to create an event center, a project named k8s-log-{cluster-id} is automatically created for the event center.
  5. Click Next.
    Note After you create an event center, a Logstore named k8s-event is automatically created in the associated project. Relevant dashboards and alerts are also created for the event center.

Step 2: Deploy the eventer and node-problem-detector components

Before you can use the Kubernetes event center, you must deploy the eventer and node-problem-detector components in the Kubernetes cluster.

  • Deploy the eventer and node-problem-detector components in an Alibaba Cloud Kubernetes cluster

    In Alibaba Cloud Marketplace, the ack-node-problem-collector component of Kubernetes clusters includes the eventer and node-problem-detector subcomponents. For more information, see Scenario 3: Use node-problem-detector with kube-eventer to raise alerts upon node anomalies.

    1. Log on to the Alibaba Cloud Container Service for Kubernetes console.
    2. In the left-side navigation pane, choose Marketplace > App Catalog.
    3. On the page that appears, click the Alibaba Cloud Apps tab. On this tab, click ack-node-problem-detector.
    4. On the page that appears, click the Parameters tab. On this tab, modify the parameter settings under the eventer field.
      • enabled: Set the value of the enabled parameter under eventer > sinks > sls to true.
      • topic: Optional. Set this parameter to your cluster name. The name can contain letters, underscores (_), and hyphens (-).
      • project: Set this parameter to the name of the project that is associated with the event center.
      • logstore: Set this parameter to k8s-event.
        sinks:
     sls:
       enabled: true
       # If you want the monitoring results to be notified by sls, set enabled to true.
       topic: "my-cluster"
       project:  "{sls-project-name}"
       # You can view the project information by logging in to the
       # SLS console. Please fill in the name of the project here.
       # eg: your project name is k8s-log-cc18a5f3443dhdss22654da,
       # then you can fill k8s-log-cc18a5f3443dhdss22654da to project label.
       logstore: "k8s-event"
       # You can view the project information by logging in to the
       # SLS console. Please fill the logstore address in here.
    5. Click Create to complete the deployment.
  • Deploy the eventer and node-problem-detector components in a user-created Kubernetes cluster
    1. For more information about how to deploy the eventer component, see Collect Kubernetes events.
    2. For more information about how to deploy the node-problem-detector component, visit GitHub.

Step 3: Use the event center

After you create a Kubernetes event center and deploy the eventer and node-problem-detector components, you can use the Kubernetes event center. You can use the event center to view event statistics, query event details, view pod lifecycle, configure alerts, and perform custom queries.

In the left-side navigation pane of the K8s Event Center page, find the target event center and click Kubernetes event center - 002 to perform the following operations.

Operation Description
View event statistics Click Event Overview to view the statistics of important events. For example, you can view the number of error events (compared with yesterday or last week), statistics of warning events, trends of top 10 important events, and statistics of pod out of memory (OOM) events.
Note If an OOM event occurs, you cannot view the information about the relevant pod. You can view only the node, process name, and process ID of the event. However, you can query the pod restart event around the time when the pod OOM event occurred. Then, you can find the pod.
Query event details Click Event Details. On the page that appears, set one or more search conditions, including level, type, target, host, namespace, and name. Then, you can view the statistics and details of events.
View the lifecycle of a pod Click Pod Lifecycle. On the page that appears, set the search conditions. The information of all events that occurred on the pod are displayed in a chart. You can filter warning events or error events of the pod by severity level.
Configure an alert Click Alert Configuration to configure an event alert. For more information, see Procedure.
Perform custom queries Click Custom Query to query events by using custom search conditions. For information about the query syntax, see Overview.

All events in the event center are stored in the Logstore. You can use all features of the Logstore. For example, you can perform custom queries, process consumption events, and create custom reports or alerts.

To obtain the name of the project that is associated with the event center, use one of the following two methods:
Configure custom alerts A Kubernetes event center supports system alerts. The Kubernetes event center also allows you to configure custom alerts.

Click Custom Query. On the page that appears, enter a statement to query a Kubernetes event, and click Save as Alert. In the dialog box that appears, set the required parameters of the alert. For more information, see Alert Overview.

The following example shows how to create an alert named FailedPreStopHook. Enter the query statement * and FailedPreStopHook | SELECT "object-namespace", "object-name", "reason", "message" in the search box of the Custom Query page, and click Save as Alert. In the dialog box that appears, set the required parameters of the alert.

Note If the alert name is prefixed with K8s, you can view the alert in the Events section of the page that appears after you click Alert Configuration. Otherwise, you can view the alert only in the Modify Alert dialog box.

The following procedure describes how to configure an alert:

  1. In the left-side navigation pane of the K8s Event Center page, find the target event center and click Kubernetes event center - 002.
  2. Click Alert Configuration. The Events page appears.
  3. Add a notification method.
    1. Click Add Notification Method.
    2. In the Add Notification Method dialog box, set the required parameters.
      Parameter Description
      Name The name of the notification method.
      Alert Interval The interval between two successive alert notifications. Default value: 5. Unit: minutes.
      Note To avoid receiving excessive alerts, we recommend that you set the alert interval to at least 2 minutes.
      ActionType The notification type. Valid values: SMS, Voice, Email, WebHook-DingTalk Bot, WebHook-Custom, and Notifications. You can select one or more notification methods. For more information, see Configure a notification method.
    3. Click OK.
  4. Enable an alert notification.
    1. In the Events section, click Modify.
    2. Find the event for which you want to enable an alert notification, click the Enable icon, and then select an alert notification.
      Note We recommend that you first enable the alert notification for all alerts. If excessive alerts are reported, you can disable some alerts or increase the notification interval.
    3. Click Save.

Delete an event center

Choose K8s Event Center > Event Center Management. On the Event Centers page, find the target event center, and click the Kubernetes Event Center icon in the Actions column to delete the event center.

FAQ

  • Why is data unavailable in a Kubernetes event center?
    After you deploy a Kubernetes event center, new events are collected to the event center. You can click Custom Query to query the events. We recommend that you set the query time range to 1 Day. If no data is available in the Kubernetes event center, the possible causes are:
    • After you deploy the Kubernetes event center, no events are generated in the associated Kubernetes cluster.

      You can run the kubectl get events --all-namespaces command to check whether new events are generated in the cluster.

    • The specified parameter values are invalid when you deploy the eventer and node-problem-detector components.
      • If the associated cluster is an Alibaba Cloud Kubernetes cluster, log on to the Alibaba Cloud Container Service for Kubernetes console. Choose Applications > Releases. On the page that appears, select the associated cluster from the Cluster drop-down list, and click Update to view the configurations of the ack-node-problem-detector component. For more information, see Step 2: Deploy the eventer and node-problem-detector components.
      • If the associated cluster is a user-created Kubernetes cluster, follow the instructions in Collect Kubernetes events to check the parameters.
  • How do I view the logs of the pod that is associated with an event?
    • If the associated cluster is an Alibaba Cloud Kubernetes cluster, log on to the Alibaba Cloud Container Service for Kubernetes console. Choose Applications > Pods. On the page that appears, select the associated cluster from the Cluster drop-down list, and select kube-system from the Namespace drop-down list. Enter the keyword eventer in the search box to search for the target pod. Click View Details to view the logs of the pod.
    • If the associated cluster is a user-created Kubernetes cluster, you can view the logs of the target pod whose name is prefixed with event-sls. In addition, the pod is in the kube-system namespace.