Log Service and Alibaba Cloud Server Load Balancer (SLB) jointly launch the CloudLens for CLB application. You can use the application to analyze Classic Load Balancer (CLB) access logs, analyze metrics in seconds, and generate alerts in real time. The application also provides AIOps-based automated anomaly detection. The CloudLens for CLB application allows you to analyze the behavior, geographical distribution, request success rate, and response latency of clients. This topic describes the features, benefits, assets, billing, and limits of the CloudLens for CLB application.

Features

The CloudLens for CLB application automatically aggregates real-time access logs and provides features such as intelligent inspection and real-time alerting. Features
  • The application allows you to manage all CLB instances within your Alibaba Cloud account in a centralized manner. Features
  • The application allows you to enable the log collection feature for CLB access logs with a few clicks and manage the log collection status of the CLB instances in a centralized manner. Features
  • The application allows you to store, query, and analyze CLB access logs in real time. Features
  • The application provides various reports, including Monitoring Center and Instance Inspection. You can subscribe to these reports and configure settings to send the reports by using emails or DingTalk group messages. Features
  • The application supports custom alert settings and can send alert notifications by using the following methods: Message Center, text messages, emails, voice calls, DingTalk, and custom webhooks.
  • The application extracts various metrics in real time based on raw access logs. The metrics include page views (PVs), request success rates, average latency, P50 latency, P99 latency, and P9999 latency, and inbound and outbound traffic. The application can extract metrics from one or more of the following dimensions: slbid, host, method, and status.
  • The application provides the intelligent inspection feature and supports global inspection and slbid-based inspection. You can label anomalies in reports.

Benefits

  • Easy-to-use: You can enable the application with a few clicks and use centralized storage for the application. You do not need to focus on the collection, storage, computing, and visualization of logs. This allows developers and O&M personnel to focus on business development and technical research without the need to worry about tedious and time-consuming log processing.
  • Capable of processing large amounts of data: The number of CLB access logs increases with the number of PVs for CLB instances. This way, a large number of access logs are accumulated. When you process the large number of access logs, you must balance performance and costs. In this case, you can use the CloudLens for CLB application, which can improve query performance. The application supports custom settings for the pre-aggregation feature and can use the feature to calculate and aggregate metrics in real time. After the aggregation, the data volume significantly reduces.
  • Real-time: Real-time data is required in scenarios such as DevOps, monitoring, and alerting. The application can analyze and process real-time logs in seconds by integrating Alibaba Cloud SLB with the big data computing capabilities of Log Service.
  • Flexible: You can enable or disable the access log management feature based on the specifications of your CLB instances. You can also specify a custom retention period for logs. The storage capacity of a Logstore can be dynamically scaled to meet service requirements.
  • Intelligent: The application automatically inspects CLB metrics to identify and locate errors in an efficient and accurate manner based on the AIOps algorithms that are developed by Alibaba DAMO Academy.

Assets

You can view the assets of the CloudLens for CLB application in the project that you specify. The following assets are included:
  • Logstores
    • The Logstore that is used to store CLB access logs. You can use a custom Logstore.
    • The Logstore that is used to store inspection results. After you enable the log collection feature for access logs, Log Service automatically generates a dedicated Logstore named Logstore for access logs-metrics-result.
    Note
    • Do not delete the Logstore that is used to store CLB access logs. If you delete the Logstore, access logs cannot be collected to Log Service.
    • Do not delete the indexes of specific fields in the Logstore that is used to store CLB access logs. If you delete the indexes, metric conversion fails.
  • Metricstore
    The Metricstore is used to store the metrics that are generated after raw metrics are aggregated. After you enable the log collection feature for access logs, Log Service automatically generates a dedicated Metricstore named Logstore for access logs-metrics.
    Note The Metricstore stores the metrics that are generated after raw metrics are aggregated. The data volume significantly reduces after aggregation, and you can store aggregated metrics for a long period of time.
  • Aggregation rules
    Rule name Time granularity Dimension New metric
    total 10 seconds total
    • pv
    • body_bytes_sent_avg
    • body_bytes_sent_sum
    • request_length_avg
    • request_length_sum
    • upstream_response_time_avg
    • upstream_response_time_p50
    • upstream_response_time_p90
    • upstream_response_time_p99
    • upstream_response_time_p9999
    • request_time_avg
    • request_time_p50
    • request_time_p90
    • request_time_p99
    • request_time_p9999
    slbid 10 seconds slbid
    • pv:slb
    • body_bytes_sent_avg:slb
    • body_bytes_sent_sum:slb
    • request_length_avg:slb
    • request_length_sum:slb
    • upstream_response_time_avg:slb
    • upstream_response_time_p50:slb
    • upstream_response_time_p90:slb
    • upstream_response_time_p99:slb
    • upstream_response_time_p9999:slb
    • request_time_avg:slb
    • request_time_p50:slb
    • request_time_p90:slb
    • request_time_p99:slb
    • request_time_p9999:slb
    slbid_host_status 10 seconds slbid+host+status
    • pv:slbid:host:status
    • body_bytes_sent_avg:slbid:host:status
    • body_bytes_sent_sum:slbid:host:status
    • request_length_avg:slbid:host:status
    • request_length_sum:slbid:host:status
    • upstream_response_time_avg:slbid:host:status
    • upstream_response_time_p50:slbid:host:status
    • upstream_response_time_p90:slbid:host:status
    • upstream_response_time_p99:slbid:host:status
    • upstream_response_time_p9999:slbid:host:status
    • request_time_avg:slbid:host:status
    • request_time_p50:slbid:host:status
    • request_time_p90:slbid:host:status
    • request_time_p99:slbid:host:status
    • request_time_p9999:slbid:host:status
    slbid+host+status+request_method+upstream_status+url 10 seconds slbid+host+status+request_method+upstream_status+url
    • pv:slbid:host:status:method:upstream_status
    • body_bytes_sent_avg:slbid:host:status:method:upstream_status
    • body_bytes_sent_sum:slbid:host:status:method:upstream_status
    • request_length_avg:slbid:host:status:method:upstream_status
    • request_length_sum:slbid:host:status:method:upstream_status
    • upstream_response_time_avg:slbid:host:status:method:upstream_status
    • upstream_response_time_p50:slbid:host:status:method:upstream_status
    • upstream_response_time_p90:slbid:host:status:method:upstream_status
    • upstream_response_time_p99:slbid:host:status:method:upstream_status
    • upstream_response_time_p9999:slbid:host:status:method:upstream_status
    • request_time_avg:slbid:host:status:method:upstream_status
    • request_time_p50:slbid:host:status:method:upstream_status
    • request_time_p90:slbid:host:status:method:upstream_status
    • request_time_p99:slbid:host:status:method:upstream_status
    • request_time_p9999:slbid:host:status:method:upstream_status
  • Inspection rules
    Rule name Status Algorithm Metric
    slb-patrol-total This rule is enabled by default. Time2Graph
    • pv
    • body_bytes_sent_avg
    • body_bytes_sent_sum
    • request_length_avg
    • request_length_sum
    • upstream_response_time_avg
    • upstream_response_time_p50
    • upstream_response_time_p90
    • upstream_response_time_p99
    • upstream_response_time_p9999
    • request_time_avg
    • request_time_p50
    • request_time_p90
    • request_time_p99
    • request_time_p9999
    slb-patrol-slb This rule is enabled by default. Time2Graph
    • pv:slb
    • body_bytes_sent_avg:slb
    • body_bytes_sent_sum:slb
    • request_length_avg:slb
    • request_length_sum:slb
    • upstream_response_time_avg:slb
    • upstream_response_time_p50:slb
    • upstream_response_time_p90:slb
    • upstream_response_time_p99:slb
    • upstream_response_time_p9999:slb
    • request_time_avg:slb
    • request_time_p50:slb
    • request_time_p90:slb
    • request_time_p99:slb
    • request_time_p9999:slb
    slb-patrol-slbid_host_status This rule is disabled by default. Time2Graph
    • pv:slbid:host:status
    • body_bytes_sent_avg:slbid:host:status
    • body_bytes_sent_sum:slbid:host:status
    • request_length_avg:slbid:host:status
    • request_length_sum:slbid:host:status
    • upstream_response_time_avg:slbid:host:status
    • upstream_response_time_p50:slbid:host:status
    • upstream_response_time_p90:slbid:host:status
    • upstream_response_time_p99:slbid:host:status
    • upstream_response_time_p9999:slbid:host:status
    • request_time_avg:slbid:host:status
    • request_time_p50:slbid:host:status
    • request_time_p90:slbid:host:status
    • request_time_p99:slbid:host:status
    • request_time_p9999:slbid:host:status
    slb-patrol-slbid_host_status_request_method_upstream_status This rule is disabled by default. Time2Graph
    • pv:slbid:host:status:method:upstream_status
    • body_bytes_sent_avg:slbid:host:status:method:upstream_status
    • body_bytes_sent_sum:slbid:host:status:method:upstream_status
    • request_length_avg:slbid:host:status:method:upstream_status
    • request_length_sum:slbid:host:status:method:upstream_status
    • upstream_response_time_avg:slbid:host:status:method:upstream_status
    • upstream_response_time_p50:slbid:host:status:method:upstream_status
    • upstream_response_time_p90:slbid:host:status:method:upstream_status
    • upstream_response_time_p99:slbid:host:status:method:upstream_status
    • upstream_response_time_p9999:slbid:host:status:method:upstream_status
    • request_time_avg:slbid:host:status:method:upstream_status
    • request_time_p50:slbid:host:status:method:upstream_status
    • request_time_p90:slbid:host:status:method:upstream_status
    • request_time_p99:slbid:host:status:method:upstream_status
    • request_time_p9999:slbid:host:status:method:upstream_status
  • Dedicated dashboards
    Dashboard Associated Logstore or Metricstore Description
    Monitoring Overview Logstore for access logs-metrics Displays the overall information about the metrics of CLB instances. The metrics include Core Indicators, Error Code, Traffic, Exception Event, PVs, Access Success Rate, and Avg Latency.
    Monitoring Center
    • Logstore for access logs-metrics
    • Logstore for access logs
    Displays the real-time monitoring data of CLB instances. The data includes PVs, Request Success Rate, Average Latency, Requests with Status Code 4xx, Status Distribution, Traffic, P50 Latency, P90 Latency, P99 Latency, P9999 Latency, Hosts with Most Requests, Hosts with Highest Latency, Hosts with Highest Failure Rate, URLs with Most Requests, URLs with Highest Latency, URLs with Highest Failure Rate, Backends with Most Requests, Backends with Highest Latency, and Backends with Highest Failure Rate.
    Real-time Monitoring Logstore for access logs Displays the metrics of CLB instances that are accurate to seconds. The metrics include QPS, Access Latency, Upstream Latency, Success Rate, Request Traffic, Response Body Traffic, Status Code 2xx, Status Code 3xx, Error Codes, Upstream Status Code 2xx, Upstream Status Code 3xx, and Upstream Error Codes.
    Instance Inspection
    • Logstore for access logs-metrics
    • Logstore for access logs-metrics-result
    Displays information about anomalies that are detected by Log Service. The information includes Exceptions, High Exceptions, Exception Distributions, Middle Exceptions, Low Exceptions, Exception List, and Exception Events. Log Service automatically detects anomalies in CLB instances based on the machine learning algorithms that are provided by Log Service.
    Access Overview Logstore for access logs Displays the overall status of CLB instances. The status information includes PVs (Day-on-day), PVs (Week-on-week), UVs (Day-on-day), UVs (Week-on-week), PV Distribution, UV Distribution, PVs Today, PVs of 7 Days, Top 10 States with Most Requests, Percentage of Mobile Users, TOP 10 Hosts with Most Requests, TOP 10 User Agents with Most Requests, and IP Addresses with Most Requests.

Billing

  • You are not charged for the access log management feature of SLB.
  • After CLB access logs are collected to Log Service, you are charged based on the storage space, read traffic, number of requests, data transformation, and data shipping. The fees are included in your Log Service bills. For more information, see Billable items.

Limits

  • You can enable the collection of access logs only on the CLB instances on which Layer 7 listeners are configured.
  • The CLB instance that you use must reside in the same region as the project that you specify in Log Service.
  • CloudLens for CLB is supported in the following regions.
    Cloud type Region
    Alibaba Cloud public cloud China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), Singapore (Singapore), Australia (Sydney), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Philippines (Manila), Japan (Tokyo), India (Mumbai), US (Silicon Valley), US (Virginia), Germany (Frankfurt), UK (London), UAE (Dubai), and Russia (Moscow)

Precautions

If you enable a CloudLens application, Log Service automatically checks whether a project whose name is in the aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan format exists within your Alibaba Cloud account. If the project does not exist, Log Service automatically creates the project.

If you want to delete the project, open the Cloud Shell and run the aliyunlog log delete_project --project_name=aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan --region-endpoint=cn-heyuan.log.aliyuncs.com command. Replace Alibaba Cloud account ID based on your business scenario.
Notice If you delete the project, all CloudLens applications become unavailable. Proceed with caution.