Simple Log Service:Usage notes

The CloudLens for CLB application automatically aggregates real-time access logs and provides features such as intelligent inspection and real-time alerting.

• The application allows you to manage all CLB instances within your Alibaba Cloud account in a centralized manner.

• The application allows you to enable the log collection feature for instance logs and global logs with a few clicks and manage the log collection status in a centralized manner. Instance logs are access logs, and global logs consist of Cloud Config logs and CloudMonitor events. For more information, see Enable the data collection feature.

  • Access logs record detailed information about all requests that are sent to CLB instances. The information includes the time when the request is initiated, client IP address, latency, request path, and response from the server.

  • Cloud Config logs consist of change logs and resource non-compliance logs of Cloud Config.

  • CloudMonitor events are the events that are related to SLB certificate expiration.

  

• The application allows you to store, query, and analyze CLB access logs in real time.

• The application provides various reports, including Monitoring Center and Instance Inspection. You can subscribe to these reports and configure settings to send the reports by using emails or DingTalk group messages.

• The application supports custom alert settings and can send alert notifications by using the following methods: Message Center, text messages, emails, voice calls, DingTalk, and custom webhooks.

• The application extracts various metrics in real time based on raw access logs. The metrics include page views (PVs), request success rates, average latency, P50 latency, P99 latency, and inbound and outbound traffic. The application can extract metrics from one or more of the following dimensions: slbid, host, method, and status.

• The application provides the intelligent inspection feature and supports global inspection and slbid-based inspection. You can label anomalies in reports.

You can view the assets of the CloudLens for CLB application in the project that you specify. The following assets are included:

• Logstores

  • The Logstore that is used to store CLB access logs. You can use a custom Logstore.

  • The Logstore that is used to store Cloud Config logs. You can use a custom Logstore.

  • The Logstore that is used to store CloudMonitor events. You can use a custom Logstore.

  • The Logstore that is used to store inspection results. After you enable the log collection feature for access logs, Simple Log Service automatically generates a dedicated Logstore named Logstore for access logs-metrics-result.

  Warning

  • Do not delete the preceding Logstores. If you delete the preceding Logstores, related logs cannot be collected to Simple Log Service.

  • Do not delete the indexes of specific fields in the Logstore that is used to store CLB access logs. If you delete the indexes, metric conversion fails.

• Metricstore

  The Metricstore is used to store the metrics that are generated after raw metrics are aggregated. After you enable the log collection feature for access logs, Simple Log Service automatically generates a dedicated Metricstore named Logstore for access logs-metrics.

  Note

  The Metricstore stores the metrics that are generated after raw metrics are aggregated. The data volume significantly reduces after aggregation, and you can store aggregated metrics for a long period of time.

• Aggregation rules

  Rule name	Time granularity	Dimension	New metric
  total	10 seconds	total	pv body_bytes_sent_avg body_bytes_sent_sum request_length_avg request_length_sum upstream_response_time_avg upstream_response_time_p50 upstream_response_time_p90 upstream_response_time_p99 request_time_avg request_time_p50 request_time_p90 request_time_p99
  slbid	10 seconds	slbid	pv:slb body_bytes_sent_avg:slb body_bytes_sent_sum:slb request_length_avg:slb request_length_sum:slb upstream_response_time_avg:slb upstream_response_time_p50:slb upstream_response_time_p90:slb upstream_response_time_p99:slb request_time_avg:slb request_time_p50:slb request_time_p90:slb request_time_p99:slb
  slbid_host_status	10 seconds	slbid+host+status	pv:slbid:host:status body_bytes_sent_avg:slbid:host:status body_bytes_sent_sum:slbid:host:status request_length_avg:slbid:host:status request_length_sum:slbid:host:status upstream_response_time_avg:slbid:host:status upstream_response_time_p50:slbid:host:status upstream_response_time_p90:slbid:host:status upstream_response_time_p99:slbid:host:status request_time_avg:slbid:host:status request_time_p50:slbid:host:status request_time_p90:slbid:host:status request_time_p99:slbid:host:status
  slbid+host+status+request_method+upstream_status+url	10 seconds	slbid+host+status+request_method+upstream_status+url	pv:slbid:host:status:method:upstream_status body_bytes_sent_avg:slbid:host:status:method:upstream_status body_bytes_sent_sum:slbid:host:status:method:upstream_status request_length_avg:slbid:host:status:method:upstream_status request_length_sum:slbid:host:status:method:upstream_status upstream_response_time_avg:slbid:host:status:method:upstream_status upstream_response_time_p50:slbid:host:status:method:upstream_status upstream_response_time_p90:slbid:host:status:method:upstream_status upstream_response_time_p99:slbid:host:status:method:upstream_status request_time_avg:slbid:host:status:method:upstream_status request_time_p50:slbid:host:status:method:upstream_status request_time_p90:slbid:host:status:method:upstream_status request_time_p99:slbid:host:status:method:upstream_status

• Inspection rules

  Rule name	Status	Algorithm	Metric
  slb-patrol-total	This rule is enabled by default.	Time2Graph	pv body_bytes_sent_avg body_bytes_sent_sum request_length_avg request_length_sum upstream_response_time_avg upstream_response_time_p50 upstream_response_time_p90 upstream_response_time_p99 request_time_avg request_time_p50 request_time_p90 request_time_p99
  slb-patrol-slb	This rule is enabled by default.	Time2Graph	pv:slb body_bytes_sent_avg:slb body_bytes_sent_sum:slb request_length_avg:slb request_length_sum:slb upstream_response_time_avg:slb upstream_response_time_p50:slb upstream_response_time_p90:slb upstream_response_time_p99:slb request_time_avg:slb request_time_p50:slb request_time_p90:slb request_time_p99:slb
  slb-patrol-slbid_host_status	This rule is disabled by default.	Time2Graph	pv:slbid:host:status body_bytes_sent_avg:slbid:host:status body_bytes_sent_sum:slbid:host:status request_length_avg:slbid:host:status request_length_sum:slbid:host:status upstream_response_time_avg:slbid:host:status upstream_response_time_p50:slbid:host:status upstream_response_time_p90:slbid:host:status upstream_response_time_p99:slbid:host:status request_time_avg:slbid:host:status request_time_p50:slbid:host:status request_time_p90:slbid:host:status request_time_p99:slbid:host:status
  slb-patrol-slbid_host_status_request_method_upstream_status	This rule is disabled by default.	Time2Graph	pv:slbid:host:status:method:upstream_status body_bytes_sent_avg:slbid:host:status:method:upstream_status body_bytes_sent_sum:slbid:host:status:method:upstream_status request_length_avg:slbid:host:status:method:upstream_status request_length_sum:slbid:host:status:method:upstream_status upstream_response_time_avg:slbid:host:status:method:upstream_status upstream_response_time_p50:slbid:host:status:method:upstream_status upstream_response_time_p90:slbid:host:status:method:upstream_status upstream_response_time_p99:slbid:host:status:method:upstream_status request_time_avg:slbid:host:status:method:upstream_status request_time_p50:slbid:host:status:method:upstream_status request_time_p90:slbid:host:status:method:upstream_status request_time_p99:slbid:host:status:method:upstream_status

• Dedicated dashboards

  Dashboard	Associated Logstore or Metricstore	Description
  Monitoring Overview	Logstore for access logs-metrics	Displays the overall information about the metrics of CLB instances. The metrics include Core Indicators, Error Code, Traffic, Exception Event, PVs, and Access Success Rate.
  Monitoring Center	Logstore for access logs-metrics Logstore for access logs	Displays the real-time monitoring data of CLB instances. The data includes PVs, Request Success Rate, Average Latency, Requests with Status Code 4xx, Status Distribution, Traffic, P50 Latency, P90 Latency, P99 Latency, Hosts with Most Requests, Hosts with Highest Latency, Hosts with Highest Failure Rate, URLs with Most Requests, URLs with Highest Latency, URLs with Highest Failure Rate, Backends with Most Requests, Backends with Highest Latency, and Backends with Highest Failure Rate.
  Real-time Monitoring	Logstore for access logs	Displays the metrics of CLB instances that are accurate to seconds. The metrics include QPS, Access Latency, Upstream Latency, Success Rate, Request Traffic, Response Body Traffic, Status Code 2xx, Status Code 3xx, Error Codes, Upstream Status Code 2xx, Upstream Status Code 3xx, and Upstream Error Codes.
  Instance Inspection	Logstore for access logs-metrics Logstore for access logs-metrics-result	Displays information about anomalies that are detected by Simple Log Service. The information includes Exceptions, High Exceptions, Exception Distributions, Middle Exceptions, Low Exceptions, Exception List, and Exception Events. Simple Log Service automatically detects anomalies in CLB instances based on the machine learning algorithms that are provided by Simple Log Service.
  Access Overview	Logstore for access logs	Displays the overall status of CLB instances. The status information includes PVs (Day-on-day), PVs (Week-on-week), UVs (Day-on-day), UVs (Week-on-week), PV Distribution, UV Distribution, PVs Today, PVs of 7 Days, Top 10 States with Most Requests, Percentage of Mobile Users, TOP 10 Hosts with Most Requests, TOP 10 User Agents with Most Requests, and IP Addresses with Most Requests.

If you want to delete the project, open the Cloud Shell and run the aliyunlog log delete_project --project_name=aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan --region-endpoint=cn-heyuan.log.aliyuncs.com command. Replace Alibaba Cloud account ID based on your business scenario.