Differences between log consumption and log search

Last Updated: Jun 15, 2017

The Log Service provides two functions related to the read operation.

Log collection and consumption (LogHub): provides public channels for log collection and distribution, sequential (FIFO) read and write of full data, and functions similar to Kafka.

  • Each LogStore has one or more shards. Data is written to a specific shard at random.
  • You can read logs in batches from the specified shard according to the log write sequence.
  • You can set the starting point (cursor) of batch log pulling from shards based on the time when the Log Service receives logs.
  • By default, logs are retained in LogHub for two days, during which logs can be consumed.

LogSearch (index): The LogSearch function is provided based on LogHub and supports massive log query. Data is queried randomly based on keywords.

  • Acquisition of only keyword-matched data
  • Boolean combination of keywords AND, NOT, and OR
  • Data is queried based on all shards.

Difference:

Function LogSearch Log collection and consumption (based on LogHub)
Keyword search Supported Not supported
Reading of small data volumes Fast Fast
Full data reading Slow (100 logs every 100 ms, not recommended) Fast (1-MB logs every 10 ms, recommended)
Reading by topic Yes No. Data is read by shard.
Reading by shard No. Data is queried based on all shards. Yes. A shard must be specified for the read operation.
Fee Relatively high Low
Application scenario Data filter is performed for monitoring and troubleshooting. Full processing scenarios such as stream computing and batch processing
Thank you! We've received your feedback.