The service log feature of Log Service helps you record log data about the operations that are performed in a project. This feature also provides dashboards that allow you to analyze data from multiple dimensions. You can use this feature to view the service status of Log Service in real time and improve O&M efficiency.

Default configuration

Default configuration item Description
Logstore When the service log feature is enabled for a project, generated log data is classified and stored in one of the following Logstores. Two Logstores are automatically created for the project that you specify to store service logs.
  • internal-operation_log: stores operations log data. A log entry corresponds to an API request. By default, log data in the Logstore is retained for 30 days. The Logstore is charged as a common Logstore.
  • internal-diagnostic_log: stores consumption delay logs of consumer groups and Logtail heartbeat logs. The logs are classified by topic. By default, log data in the Logstore is retained for 30 days. The Logstore is free of charge.
For more information about log types and fields, see Log types.
Note
  • The automatically created Logstores are used to store only the logs generated by Log Service. You cannot write other data to these Logstores. However, you can perform other operations on the automatically created Logstores. For example, you can query and analyze data in the Logstores, configure alerts, or use the Logstore data for streaming consumption.
  • You are charged for log data recorded by the service log feature on a pay-as-you-go basis. For more information, see Billable items.
Region
  • If you select Automatic creation (recommended), a project is automatically created in the same region as the project for which the service log feature is enabled.
  • You can also select another option from the Log Storage Location drop-down list. If you specify a project to store service logs, the specified project must reside in the same region as the project for which the service log feature is enabled.
Shard By default, two shards are created for each Logstore and the automatic sharding feature is enabled. For more information, see Enable automatic sharding.
Log retention period By default, log data is retained for 30 days. You can modify the retention period. For more information, see Manage a Logstore.
Index By default, the index feature is enabled for all collected log data. If you do not need to query and analyze data or configure alerts, you can disable the index feature by using Index Attributes in the upper-right corner of the Search & Analysis page.
Dashboard The following five dashboards are automatically created:
  • Operation statistics dashboard
  • Logtail log collection statistics dashboard
  • Logtail status monitoring dashboard
  • Consumer group monitoring dashboard
For more information about dashboards, see Service log dashboards.

Scenarios

  • Check whether data is evenly written and consumed among shards

    You can use predefined dashboards to view the data write and consumption trends of shards and check whether data is evenly written or consumed among shards.

    Multiple Logstores under a project may share the same shards. To view the data writes to multiple shards of a Logstore, you can add filter conditions on the dashboard to filter Logstores.

  • Monitor API request status

    You can call API operations to write log data, consume log data, and create projects or Logstores. A log entry is generated in the internal-operation_log Logstore when an operation is performed. If an API request fails, the value of the Status field in the generated log entry is an integer greater than 200, such as 404. You can monitor API requests by viewing the number of log entries whose Status field value is greater than 200.

  • View Logtail status

    By default, two Logtail-related dashboards are created after the service log feature is enabled. They are Logtail status monitoring and Logtail log collection statistics. The Logtail status monitoring dashboard helps you monitor Logtail exceptions such as regular expression mismatches and log data parsing failures.