All Products
Search
Document Center

Simple Log Service:Usage notes

Last Updated:Aug 25, 2023

Simple Log Service and Server Load Balancer (SLB) jointly launch the CloudLens for ALB application. You can use the application to analyze the Layer 7 access logs of Application Load Balancer (ALB), analyze the metrics that are aggregated at one-second intervals, and generate alerts in real time. The application also provides AIOps-based automated anomaly detection. The application allows you to analyze the behavior, geographical distribution, request success rates, and response latency of clients. This topic describes the features, assets, billing, and limits of the CloudLens for ALB application.

Features

The CloudLens for ALB application automatically aggregates real-time access logs and provides features such as intelligent inspection and real-time alerting.

  • The application allows you to manage all ALB instances within your Alibaba Cloud account in a centralized manner.

  • The application allows you to enable the data collection feature for the access logs of ALB instances with a few clicks and manage the collection status of the instances in a centralized manner.

  • The application allows you to store, query, and analyze ALB access logs in real time.

  • The application extracts various metrics in real time based on raw access logs. The metrics include page views (PVs), request success rates, average latency, P50 latency, P99 latency, and inbound and outbound traffic. The application can extract metrics from one or more of the following dimensions: app_lb_id, host, and status.

  • The application provides various reports, including Monitoring Center, Real-time Monitoring, and Instance Inspection. You can subscribe to the reports by using emails or the webhooks of DingTalk groups.

  • The application provides the intelligent inspection feature and supports global inspection and app_lb_id-based inspection. You can label anomalies in reports.

  • The application supports custom alert settings and can send alert notifications by using the following methods: Message Center, text messages, emails, voice calls, DingTalk, and custom webhooks.

Benefits

  • Easy-to-use: You can enable the application with a few clicks and use centralized storage for the application. You do not need to focus on the collection, storage, computing, or visualization of logs. This allows developers and O&M personnel to focus on business development and technical research without the need to worry about tedious and time-consuming log processing.

  • Capable of processing large amounts of data: The number of ALB access logs increases with the number of PVs for ALB instances. As a result, a large number of access logs are accumulated. When you process the large number of access logs, you must balance performance and costs.

  • Real-time: Real-time data is required in scenarios such as DevOps, monitoring, and alerting. The application integrates Alibaba Cloud SLB with the big data computing capabilities of Simple Log Service. This allows the application to analyze and process real-time logs in seconds.

  • Flexible: You can enable or disable the data collection feature for each of your ALB instances. You can also specify a custom retention period for logs. The storage capacity of a Logstore can be dynamically scaled to meet service requirements.

  • Intelligent: The application automatically inspects ALB metrics to identify and locate errors in an efficient and accurate manner. The inspection is based on the AIOps algorithms that are developed by Alibaba DAMO Academy.

Assets

You can view the assets of the CloudLens for ALB application in the project that you specify when you enable the data collection feature. The following assets are included:

  • Logstores

    • A Logstore that is used to store the Layer 7 access logs of ALB instances. You can create the Logstore.

    • A Logstore that is used to store inspection results. After you enable the data collection feature, Simple Log Service automatically generates a dedicated Logstore named The name of the Logstore for access logs-metrics-result.

    Important
    • Do not delete the Logstore that is used to store the Layer 7 access logs of ALB instances. If you delete the Logstore, access logs cannot be collected or sent to Simple Log Service.

    • Do not delete the indexes of specific fields in the Logstore that is used to store access logs. If you delete the indexes, metric conversion fails.

  • Metricstore

    A Metricstore that is used to store aggregated data about the collected metrics. After you enable the data collection feature, Simple Log Service automatically generates a dedicated Metricstore named The name of the Logstore for access logs-metrics.

    Note

    The Metricstore stores aggregated data about the collected metrics. The data volume significantly reduces after aggregation, and you can store the aggregated data for a long period of time.

  • Aggregation rules

    Rule name

    Time granularity

    Dimension

    New metric

    total

    10 seconds

    total

    • pv

    • body_bytes_sent_avg

    • body_bytes_sent_sum

    • request_length_avg

    • request_length_sum

    • upstream_response_time_avg

    • upstream_response_time_p50

    • upstream_response_time_p90

    • upstream_response_time_p99

    • request_time_avg

    • request_time_p50

    • request_time_p90

    • request_time_p99

    app_lb_id

    10 seconds

    app_lb_id

    • pv:app_lb_id

    • body_bytes_sent_avg:app_lb_id

    • body_bytes_sent_sum:app_lb_id

    • request_length_avg:app_lb_id

    • request_length_sum:app_lb_id

    • upstream_response_time_avg:app_lb_id

    • upstream_response_time_p50:app_lb_id

    • upstream_response_time_p90:app_lb_id

    • upstream_response_time_p99:app_lb_id

    • request_time_avg:app_lb_id

    • request_time_p50:app_lb_id

    • request_time_p90:app_lb_id

    • request_time_p99:app_lb_id

    app_lb_id_host_status

    10 seconds

    app_lb_id+host+status

    • pv:app_lb_id:host:status

    • body_bytes_sent_avg:app_lb_id:host:status

    • body_bytes_sent_sum:app_lb_id:host:status

    • request_length_avg:app_lb_id:host:status

    • request_length_sum:app_lb_id:host:status

    • upstream_response_time_avg:app_lb_id:host:status

    • upstream_response_time_p50:app_lb_id:host:status

    • upstream_response_time_p90:app_lb_id:host:status

    • upstream_response_time_p99:app_lb_id:host:status

    • request_time_avg:app_lb_id:host:status

    • request_time_p50:app_lb_id:host:status

    • request_time_p90:app_lb_id:host:status

    • request_time_p99:app_lb_id:host:status

  • Inspection rules

    Rule name

    Algorithm

    Metric

    alb-patrol-total

    Time2Graph

    • pv

    • body_bytes_sent_avg

    • body_bytes_sent_sum

    • request_length_avg

    • request_length_sum

    • upstream_response_time_avg

    • upstream_response_time_p50

    • upstream_response_time_p90

    • upstream_response_time_p99

    • request_time_avg

    • request_time_p50

    • request_time_p90

    • request_time_p99

    alb-patrol-alb

    Time2Graph

    • pv:alb

    • body_bytes_sent_avg:alb

    • body_bytes_sent_sum:alb

    • request_length_avg:alb

    • request_length_sum:alb

    • upstream_response_time_avg:alb

    • upstream_response_time_p50:alb

    • upstream_response_time_p90:alb

    • upstream_response_time_p99:alb

    • request_time_avg:alb

    • request_time_p50:alb

    • request_time_p90:alb

    • request_time_p99:alb

  • Dedicated dashboards

    Dashboard

    Associated Logstore or Metricstore

    Description

    Monitoring Overview

    The name of the Logstore for access logs-metrics

    Displays the overall information about the metrics of an ALB instance. The metrics include Core Indicators, Error Code, Traffic, Exception Event, PVs, Access Success Rate, Traffic, and Avg Latency.

    Monitoring Center

    • The name of the Logstore for access logs-metrics

    • The name of the Logstore for access logs

    Displays the real-time monitoring data of an ALB instance. The data includes PVs, Request Success Rate, Average Latency, Requests with Status Code 4xx, Status Distribution, Traffic, P50 Latency, P90 Latency, P99 Latency, Hosts with Most Requests, Hosts with Highest Latency, Hosts with Highest Failure Rate, URLs with Most Requests, URLs with Highest Latency, URLs with Highest Failure Rate, Backends with Most Requests, Backends with Highest Latency, and Backends with Highest Failure Rate.

    Real-time Monitoring

    The name of the Logstore for access logs

    Displays the metrics that are aggregated at one-second intervals. You can use the metrics to identify exceptions that are related to transient jitters. The metrics include QPS, Access Latency, Upstream Latency, Success Rate, Request Traffic, Response Body Traffic, Status Code 2xx, Status Code 3xx, Error Codes, Upstream Status Code 2xx, Upstream Status Code 3xx, and Upstream Error Codes.

    Instance Inspection

    • The name of the Logstore for access logs-metrics

    • The name of the Logstore for access logs-metrics-result

    Displays the information about anomalies that are detected by Simple Log Service in an ALB instance. The detection is based on the machine learning algorithms that are provided by Simple Log Service. The information includes Exceptions, High Exceptions, Exception Distributions, Middle Exceptions, Low Exceptions, Exception Distributions, Exception List, and Exception Events.

    Access Overview

    The name of the Logstore for access logs

    Displays the status of an ALB instance. The status information includes PVs (Day-on-day), PVs (Week-on-week), UVs (Day-on-day), UVs (Week-on-week), PV Distribution, UV Distribution, PVs Today, PVs of 7 Days, Top 10 States with Most Requests, Percentage of Mobile Users, TOP 10 Hosts with Most Requests, TOP 10 User Agents with Most Requests, and IP Addresses with Most Requests.

Billing

  • You are not charged for the log management feature of SLB.

  • After ALB access logs are collected and sent to Simple Log Service, you are charged for billable items such as the storage space, read traffic, number of requests, data transformation, and data shipping. The fees are included in your Simple Log Service bills. For more information, see Billable items of pay-by-feature.

Limits

  • The project that you specify in Simple Log Service must reside in the same region as the ALB instance that you use.

  • CloudLens for ALB is supported in the following regions.

    Cloud type

    Region

    Alibaba Cloud public cloud

    China (Qingdao), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Hangzhou), China (Shanghai), China (Nanjing - Local Region), China (Shenzhen), China (Guangzhou), China (Chengdu), China (Hong Kong), Singapore (Singapore), Australia (Sydney), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo), India (Mumbai), US (Silicon Valley), US (Virginia), and Germany (Frankfurt)

Precautions

If you enable a CloudLens application, Log Service automatically checks whether a project whose name is in the aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan format exists within your Alibaba Cloud account. If the project does not exist, Log Service automatically creates the project.

If you want to delete the project, open the Cloud Shell and run the aliyunlog log delete_project --project_name=aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan --region-endpoint=cn-heyuan.log.aliyuncs.com command. Replace Alibaba Cloud account ID based on your business scenario.
Warning If you delete the project, all CloudLens applications become unavailable. Proceed with caution.