Log Service

An all-in-one service for log-type data

Log Service is a complete real-time data logging service that has been developed by Alibaba Group. Log Service supports collection, consumption, shipping, search, and analysis of logs, and improves the capacity of processing and analyzing large amounts of logs.

Benefits

Ease of use
Completes data collections from more than 30 data sources within five minutes.
Enriched features
Fully supports real-time and offline computing, and seamlessly connects to Alibaba Cloud software, open-source software, and commercial software.
Stable and reliable
Deploys reliable high-availability service nodes in data centers around the world.
Four Fully-managed
You can set the access permissions for individual rows so that the same report is displayed differently for each user role.

Features

  • Fully-managed Service

    Completes connections to data sources within five minutes without any management cost.


    Ease of use

    Completes connections to data sources over any networks within five minutes by using the Logtail agent.


    LogHub

    Log Service fully supports Kafka, elastic scaling, delay alarms, and all streaming computing systems, such as Spark Streaming, Storm, StreamCompute, Flink, and Consumer Library (automatic load balancing). Log Service can integrate with third-party services, such as CloudMonitor and Application Real-Time Monitoring Service (ARMS). The usage cost is reduced by more than 50%, compared with deploying your own system.


    LogSearch/Analytics

    Log Service saves search results, and provides dashboards and reports. The usage cost is reduced by more than 50%, compared with deploying your own system.

  • Integrations and Configurations

    Integrated with Alibaba Cloud software, open-source software, and commercial software.


    LogHub

    LogHub supports more than 30 data collection tools, such as Logstash and Fluent, for easy integration with embedded devices, webpages, servers, and programs. LogHub integrates with various consumer systems, such as Spark Streaming, Storm, Flink, StreamCompute, CloudMonitor, and ARMS.


    LogShipper

    Supports user-defined partitions and multiple data formats, such as TextFile, SequenceFile, and Parquet, and ships data to storage engines, such as Presto, Hive, Spark, Hadoop, E-MapReduce, MaxCompute, and HybridDB.


    LogSearch/Analytics

    Supports the complete search and analysis syntax that is compatible with SQL-92, and supports connections using Restful APIs and Java Database Connectivity (JDBC), and visual analysis tools, such as Grafana, DataV, Zipkin, and Jaeger.

  • Real-time Response

    Processes large amounts of data in real time.


    LogHub

    LogHub processes written data in real time. Collection agent Logtail collects and transmits data to Log Service in real time, with an availability of 99.9%.


    LogSearch/Analytics

    Writes, searches, and analyzes data in real time. Multiple search conditions are used to search more than one billion items of data within one second, and multiple aggregation conditions are used to analyze more than 100 million items of data within one second.

  • Comprehensive APIs and SDKs

    Easily supports custom management and secondary development.


    APIs

    APIs and SDKs support multiple features and programming languages, to easily manage and serve more than one million devices.


    Search and analysis

    Simple syntax that is compatible with SQL-92 and multiple APIs for easy integrations with related service software. The integration with Grafana will be available soon.

How it works

  • Real-time Data Collection
  • Real-time Search and Analysis
  • Real-time Computing and Data Scrubbing
  • Connections with the Data Warehouse
Real-time Data Collection

Real-time data collection

Real-time data collection

LogHub collects large amounts of logs in real time, such as Metric, Event, BinLog, TextLog, and Click. This service is cost-effective.

Benefits

  • Ease of use

    Provides more than 30 real-time data collection methods to allow you to quickly build a Log Service platform. Powerful configuration and management capabilities help to reduce the workload of operating and maintaining nodes around the world.

  • Elastic scaling

    Easily handles peak-hour traffic and incremental service data.

Related Products & Services

Real-time Search and Analysis

Real-time search and analysis

Scenario title

LogSearch/Analytics supports indexing data in LogHub in real time, and provides multiple search methods based on keywords, fuzzy match, context, range, and SQL aggregation.

Benefits

  • Real-time response

    Writes and searches data in real time.

  • Mass data

    Supports PB-level daily indexing.

  • Powerful analysis

    Supports multiple search methods such as SQL aggregation and analysis.

Integrations

Real-time Computing and Data Scrubbing

Real-time computing and data scrubbing

StreamCompute and data scrubbing

LogHub supports integrations with various real-time computing services, provides comprehensive progress monitoring and alarms, and customizes data consumption using SDKs and APIs.

Benefits

  • Ease of use

    Provides comprehensive SDKs and programming frameworks for seamless connections with multiple stream computing engines.

  • Comprehensive features

    Provides enriched monitoring data and delay alarms.

  • Elastic scaling

    Real-time PB-level elastic scaling.

Integrations

Connections with the Data Warehouse

Connections with the data warehouse

Data warehouse

LogShipper ships data from LogHub to storage services, and supports various storage methods, such as compression, user-defined partitions, row storage, and column storage.

Benefits

  • Mass data

    Processes unlimited scales of data.

  • Multiple storage formats

    Supports multiple storage formats, such as row, column, and TextFile storage formats.

  • Flexible configuration

    Supports user-defined partitions and other configurations.