This topic describes typical scenarios where you can use Log Service for your business. These log scenarios include data collection and consumption, data extract, transform, and load (ETL) and stream processing, communication with data warehouses, and real-time query and analysis.

Data collection and consumption

The LogHub feature provided by Log Service allows you to collect massive volumes of log data such as metric data, log events, binary logs, text logs, and clickstream data.

Benefits:

  • Ease of use: provides a powerful platform that allows you to configure data collection from more than 30 data sources in real time and thus reduces your O&M workloads.
  • Elastic scalability: capable of handling peak hours and business growth.
Figure 1. Data collection and consumption

ETL and stream processing

LogHub supports integration with other services such as real-time computing services and provides progress monitoring, alert, and other features. LogHub also allows you to consume data by using SDK and API.

  • Ease of use: provides comprehensive SDKs and programming frameworks for seamless integration with multiple stream computing engines.
  • Monitoring and alerts: provides comprehensive monitoring and latency alert mechanisms.
  • Elastic scalability: capable of scaling to handle petabytes of data with zero latency.
Figure 2. ETL and stream processing

Integration with data warehouses

The LogShipper feature ships LogHub data to storage services and supports various storage formats, such as compressed files, user-defined partitions, rows, and columns.

  • Large data capacity: supports unlimited amount of data to be shipped to storage services.
  • Multiple formats: supports various storage formats, such as rows, columns, and text files.
  • Flexible configurations: supports multiple configurations including user-defined partitions.
Figure 3. Integration with data warehouses

Log search and analytics

The log search and analytics feature allows you to index LogHub data in real time and query data by using keywords, fuzzy match, contextual query, and SQL aggregate functions. You can also query data within a specific range.

  • Timeliness: supports real-time query after data is written to LogHub.
  • High efficiency at low cost: supports indexing petabytes of data each day at a cost that is only 15% of that of a self-built solution.
  • Strong analysis capability: provides multiple query methods, SQL aggregate functions for analysis, visualized dashboards and charts, and alert features.
Figure 4. Log search and analytics