All Products
Search
Document Center

Simple Log Service:Usage notes

Last Updated:Feb 23, 2024

Simple Log Service provides the CloudLens for SLS application to help you monitor and manage assets such as projects and Logstores. This way, you can manage the assets and obtain information about asset consumption in a more efficient manner.

Features

CloudLens for SLS supports the following features:

  • The application allows you to manage all projects and Logstores that meet specified conditions within your Alibaba Cloud account in a centralized manner. For more information about the conditions, see Limits.CloudLens for SLS

  • The application allows you to enable log collection for instance logs and global logs with a few clicks and manage the log collection status in a centralized manner. Instance logs are classified into important logs, detailed logs, and job operational logs. Global logs are classified into audit logs, billing logs, error logs, and metrics.

    Note

    You can view only instance logs and global logs that are generated after you enable the log collection feature. Instance logs are classified into important logs, detailed logs, and job operational logs. Global logs are classified into audit logs, error logs, and metric-related logs.

    • Important logs record the consumption delay of consumer groups for each Logstore, and record the errors, heartbeats, and statistics of Logtail. The logs are stored in a Logstore named internal-diagnostic_log of a specified project.

    • Detailed logs record the operations that create, modify, update, and delete resources in each project. Detailed logs also record data read and write operations. The logs are stored in a Logstore named internal-operation_log of a specified project.

    • Job operational logs record the error and latency information about jobs such as data import jobs of the new version, data shipping jobs of the new version, and Scheduled SQL jobs. The logs are stored in a Logstore named internal-diagnostic_log of a specified project.

    • Audit logs record the operations that create, modify, update, and delete resources in all projects within your Alibaba Cloud account. The logs are stored in a Logstore named internal-audit_log of a specified project.

    • Billing logs record billable resource data. The logs are generated by Cost Manager, stored in Tablestore (OTS), and automatically updated on a daily basis. For more information, see Use the new Cost Manager.

      Note
      • The first time you enable the log collection feature for billing logs, the feature requires approximately 1 hour to 2 hours to import logs. The amount of time that is required to import logs varies based on the amount of billing logs.

      • Only Alibaba Cloud public cloud users can enable the log collection feature for billing logs.

      • After you enable the log collection feature for billing logs, you cannot disable the feature.

    • Error logs record the errors of the operations that create, modify, update, and delete resources and the errors of data read and write operations in all projects within your Alibaba Cloud account. The logs are stored in a Logstore named internal-error_log of a specified project.

    • The metrics of each project and each Logstore are recorded, including traffic, requests, and latency. The metrics are stored in a Metricstore named internal-monitor-metric of a specified project.

    For more information, see Service logs.

    CloudLens for SLS

  • CloudLens for SLS provides alert monitoring rules such as baseline alert monitoring rules, interval-valued comparison alert monitoring rules, and periodicity-valued comparison alert monitoring rules. You can use the alert monitoring rules to monitor the usage of resources such as projects, Logtail, and consumer groups. CloudLens for SLS supports different notification methods, such as text messages, DingTalk messages, emails, voice calls, and custom webhooks. For more information, see Configure alerts.告警

  • The application provides the following types of reports: Overview, Access Monitoring, Collection Monitoring, Operation Monitoring, Job Monitoring, Quota Monitoring, and Billable Resource Monitoring. For more information, see View data reports.CloudLens for SLS

  • The application allows you to view quota information about all projects and Logstores that meet specified conditions within your Alibaba Cloud account. For more information about conditions, see Limits.

    Note

    CloudLens for SLS updates quota information at an interval of 12 hours.

    CloudLens for SLS

Scenarios

  • Development O&M

    Developers can use CloudLens for SLS to monitor and manage assets such as projects and Logstores in an efficient manner. In addition, the developers can query and analyze the instance logs of a project in CloudLens for SLS to efficiently troubleshoot errors. Instance logs are classified into detailed logs, important logs, and job operational logs.

  • IT O&M

    IT O&M personnel can use the alerting feature of CloudLens for SLS to identify and handle the exceptions of project service status, traffic, quotas, and Logtail metrics at the earliest opportunity.

  • Security audit

    The audit logs of CloudLens for SLS record the operations that create, update, and delete resources in a project. This allows security auditors to analyze and audit operations within a specific period of time.

Assets

  • Projects, Logstores, and Metricstores

    • When you enable log collection for important logs or job operational logs, you can select a project based on your business requirements. After log collection is enabled for important logs or job operational logs, Simple Log Service generates a Logstore named internal-diagnostic_log in the project.

    • When you enable log collection for detailed logs, you can select a project based on your business requirements. After log collection is enabled for detailed logs, Simple Log Service generates a Logstore named internal-operation_log in the project.

    • When you enable log collection for audit logs, you can select a project based on your business requirements. After log collection is enabled for audit logs, Simple Log Service generates a Logstore named internal-audit_log in the project.

    • When you enable log collection for error logs, you can select a project based on your business requirements. After log collection is enabled for error logs, Simple Log Service generates a Logstore named internal-error_log in the project.

    • When you enable log collection for metrics, you can select a project based on your business requirements. After log collection is enabled for metrics, Simple Log Service generates a Metricstore named internal-error_log in the project.

  • Dashboards

    Dashboard

    Description

    Overview

    Access Overview

    Displays information about Simple Log Service projects in charts. The charts include Number of requests, Request Success Rate, Requests with Status Code 4xx, Requests with Status Code 5xx, Write traffic (raw size), Write traffic (after compression), Write traffic compression ratio, and Read Traffic.

    Access Monitoring

    Access Traffic Monitoring

    Displays information about access to Simple Log Service in charts. The charts include Total Requests, Clients, Users, Top 10 SourceIPs with Most Read Traffic, Top 10 SourceIPs with Most Write Traffic, Requests, Write Traffic, Trend of Write Traffic to Shards, Trend of Read Traffic from Shards, Read/Write Traffic/Minute, and Request IP Distribution.

    Access Exception Monitoring

    Displays information about abnormal access to Simple Log Service in charts. The charts include Total Requests, Percentage of Failed Requests, Quota Exceeded, Error Status Distribution, Logstore with Excessive Write Traffic, Error Requests, and Trend of Request Latency.

    Consumer Group Monitoring

    Displays information about consumer groups in charts. The charts include Consumer Group, Logstore, Shard, Fall Behind, Data, Consumer Group, Top 10 Latency, and Trend of Consumer Group Latency.

    Collection Monitoring

    Logtail Overall Status

    Displays information about Logtail in charts. The charts include Active Logtail, Data Traffic, Status, CPU, Logtail Overall Status, Trend of CPU, Trend of Memory, and Data Sending Traffic.

    Logtail File Collection Monitoring

    Displays information about the files from which you want to collect logs in charts. The charts include Logtail File Collection, Machine Collection, Collection File Distribution, Trend of Log Collection, Trend of Average Latency, Trend of Parse Failure, and Trend of Send.

    Logtail Exception Monitoring

    Displays information about Logtail exceptions in charts. The charts include Active Logtail, Restart Logtail List, Restart Clients, and All Alarm.

    Operation monitoring

    Operation monitoring

    Displays information about Simple Log Service operation records in charts. The charts include Logstore Related Operation Count, Logstore Related Fail Operation, and Clients.

    Job Monitoring

    Data Processing Monitor

    Displays information about data import jobs of the new version, data shipping jobs of the new version, and Scheduled SQL jobs in charts. The charts include Ingest Success, Ingest Failed, Deliver Success, Deliver Failed, Ingest PubNet, Deliver PubNet, Process Speed, Process Lags, Process Error, and Process State.

    Quota Monitoring

    Quota Monitoring

    Displays information about quota exceeded events in charts. The charts include Overview of Resource Quota (80%) Warning, QuotaExceed, LogStore Usage Distribution, MachineGroup Usage Distribution, Logtail Config Usage Distribution, Project Write Inflow Usage Distribution, Project Overview of Basic Quota Usage, Project Overview of Write/Read Quota Usage, and Project Quota Exceeded Statistics.

    Billable Resource Monitoring

    Billable Resource Monitoring

    Displays information about the billable resources of Simple Log Service in charts. The charts include Storage Size(Hot Storage), Index Size(Log), Read/Write Traffic, Billing Items, Top Project, and Top Logstore.

Billing

You can use CloudLens for SLS free of charge. The following list describes the billing details for different types of logs:

  • You are not charged when you import, store, query, and analyze important logs, job operational logs, and audit logs. You are charged for data shipping and data transformation based on the pay-as-you-go billing method.

  • The billing method for all Logstores is the same regardless of whether a Logstore contains detailed logs.

  • You are not charged for billing logs. The related projects and Tablestore instances are free of charge.

For more information, see Billable items of pay-by-feature.

Limits

The following table describes the regions where CloudLens for SLS allows you to monitor and manage assets.

Cloud type

Region

Alibaba Cloud public cloud

China (Hangzhou), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shanghai), China (Chengdu), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Hong Kong), Indonesia (Jakarta), Malaysia (Kuala Lumpur), Philippines (Manila), Singapore, South Korea (Seoul), Thailand (Bangkok), Japan (Tokyo), India (Mumbai), UK (London), Australia (Sydney), Germany (Frankfurt), UAE (Dubai), US (Silicon Valley), and US (Virginia)

Precautions

If you enable a CloudLens application, Simple Log Service automatically checks whether a project whose name is in the aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan format exists within your Alibaba Cloud account. If the project does not exist, Simple Log Service automatically creates the project.

If you want to delete the project, open the Cloud Shell and run the aliyunlog log delete_project --project_name=aliyun-product-data-<Alibaba Cloud account ID>-cn-heyuan --region-endpoint=cn-heyuan.log.aliyuncs.com command. Replace Alibaba Cloud account ID based on your business scenario.

Warning

If you delete the project, all CloudLens applications become unavailable. Proceed with caution.