Log Service is a complete real-time data logging service that has been developed by Alibaba Group. Log Service supports collection, consumption, shipping, search, and analysis of logs, and improves the capacity of processing and analyzing large amounts of logs.
- Ease of use
Completes data collections from more than 30 data sources within five minutes.
- Enriched features
Fully supports real-time and offline computing, and seamlessly connects to Alibaba Cloud software, open-source software, and commercial software.
- Stable and reliable
Deploys reliable high-availability service nodes in data centers around the world.
You can set the access permissions for individual rows so that the same report is displayed differently for each user role.
Completes connections to data sources within five minutes without any management cost.
Ease of use
Completes connections to data sources over any networks within five minutes by using the Logtail agent.
Log Service fully supports Kafka, elastic scaling, delay alarms, and all streaming computing systems, such as Spark Streaming, Storm, StreamCompute, Flink, and Consumer Library (automatic load balancing). Log Service can integrate with other products, such as CloudMonitor and Application Real-Time Monitoring Service (ARMS). The usage cost is reduced by more than 50%, compared with deploying your own system.
Log Service saves search results, and provides dashboards and reports. The usage cost is reduced by more than 50%, compared with deploying your own system.
Integrations and Configurations
Integrated with Alibaba Cloud software, open-source software, and commercial software.
LogHub supports more than 30 data collection tools, such as Logstash and Fluent, for easy integration with embedded devices, webpages, servers, and programs. LogHub integrates with various consumer systems, such as Spark Streaming, Storm, Flink, StreamCompute, CloudMonitor, and ARMS.
Supports user-defined partitions and multiple data formats, such as TextFile, SequenceFile, and Parquet, and ships data to storage engines, such as Presto, Hive, Spark, Hadoop, E-MapReduce, MaxCompute, and HybridDB.
Supports the complete search and analysis syntax that is compatible with SQL-92, and supports connections using Restful APIs and Java Database Connectivity (JDBC), and visual analysis tools, such as Grafana, DataV, Zipkin, and Jaeger.
Processes large amounts of data in real time.
LogHub processes written data in real time. Collection agent Logtail collects and transmits data to Log Service in real time, with an availability of 99.9%.
Writes, searches, and analyzes data in real time. Multiple search conditions are used to search more than one billion items of data within one second, and multiple aggregation conditions are used to analyze more than 100 million items of data within one second.
Comprehensive APIs and SDKs
Easily supports custom management and secondary development.
APIs and SDKs support multiple features and programming languages, to easily manage and serve more than one million devices.
Search and analysis
Simple syntax that is compatible with SQL-92 and multiple APIs for easy integrations with related service software. The integration with Grafana will be available soon.
How it works
- Real-time Data Collection
- Real-time Search and Analysis
- Real-time Computing and Data Scrubbing
- Connections with the Data Warehouse
Real-time data collection
Real-time data collection
LogHub collects large amounts of logs in real time, such as Metric, Event, BinLog, TextLog, and Click. This service is cost-effective.
Ease of use
Provides more than 30 real-time data collection methods to allow you to quickly build a Log Service platform. Powerful configuration and management capabilities help to reduce the workload of operating and maintaining nodes around the world.
Easily handles peak-hour traffic and incremental service data.
Real-time search and analysis
LogSearch/Analytics supports indexing data in LogHub in real time, and provides multiple search methods based on keywords, fuzzy match, context, range, and SQL aggregation.
Writes and searches data in real time.
Supports PB-level daily indexing.
Supports multiple search methods such as SQL aggregation and analysis.
Real-time computing and data scrubbing
StreamCompute and data scrubbing
LogHub supports integrations with various real-time computing services, provides comprehensive progress monitoring and alarms, and customizes data consumption using SDKs and APIs.
Ease of use
Provides comprehensive SDKs and programming frameworks for seamless connections with multiple stream computing engines.
Provides enriched monitoring data and delay alarms.
Real-time PB-level elastic scaling.
Connections with the data warehouse
LogShipper ships data from LogHub to storage services, and supports various storage methods, such as compression, user-defined partitions, row storage, and column storage.
Processes unlimited scales of data.
Multiple storage formats
Supports multiple storage formats, such as row, column, and TextFile storage formats.
Supports user-defined partitions and other configurations.