Overview

Last Updated: Oct 26, 2017

The Logtail access service facilitates quick retrieval of logs from servers.

Logtail configuration process

process

The Log Service console provides easy access to Logtail.

  1. Install Logtail. Install Logtail on servers that you need to collect logs from, check Install Logtail on Windows and Install Logtail on Linux for more information.

  2. Configure user ID for non-Alibaba Cloud ECS. No need to perform this step if you wish to collect Logs from ECS.

  3. Create Logtail machine group. Log Service manage all the servers that need to collect logs through Logtail by using machine group. Log Service support defining machine groups through IPs and user IDs. You can also create machine group with tips when you apply logtail config to machine group.

  4. Create Logtail machine group and apply the Logtail configuration to Logtail machine groups. You can create Logtail config to collect text files or collect syslog data, and apply logtail config to machine group.

After the preceding process is completed, logs of a specific type on the ECS servers are collected and sent to the selected Logstore. (Historical logs are not collected. For details, refer to the Basic functions.) You can view the collected logs through the Log Service console or SDKs and APIs. To check whether collection is normal or whether an error occurs, log on to the Log Service console to view the Logtail collection status on each ECS server.

For more information of complete operation on console, refer to Use Logtail to collect text files.

Docker

  • Alibaba Cloud Container Service
  • ECS/IDC self-built Docker (The log directories in containers must be mounted to the host machine.)
    1. Install Logtail for Windows or Install Logtail for Linux.
    2. Mount the log directories in containers to the directory on the host machine.
      • Method 1: Use the following command (For example, the directory on the host machine is /log/webapp, and the directory in a container is /opt/webapp/log.)
        1. docker run -d -P --name web -v /src/webapp:/opt/webapp training/webapp python app.py
      • Method 2: Use orchestration template

Core concepts

  • Machine group: A machine group contains one or more machines on which logs of a specific type are collected. After a set of Logtail configurations is applied to a machine group, Logtail collects logs of the specified type on all machines in the machine group according to the same Logtail configuration. You can create and delete machine groups as well as add and remove machines in machines groups through the Log Service console. A single machine group cannot contain both Windows and Linux machines but may have machines of different Windows Server versions or different Linux releases.

  • Logtail agent: Logtail is the agent that runs on your machines to collect logs. Refer to Install Logtail on Windows and Install Logtail on Linux. After Logtail is installed on ECS servers, add the intranet IP addresses of the servers to a machine group.

    • In Linux, the agent is installed in the /usr/local/ilogtail directory and launches two independent processes (a collection process and a daemon, whose names start with ilogtail). The program running log is /usr/local/ilogtail/ilogtail.LOG.
    • In Windows, the agent is installed in the C:\Program Files\Alibaba\Logtail directory (for 32-bit systems) or the C:\Program Files (x86)\Alibaba\Logtail directory (for 64-bit systems). Two Windows services exist in Windows Management Tools > Services. One service is LogtailWorker for log collection and the other is LogtailDaemon. The program running log is logtail_*.log in the installation directory.
  • Logtail configuration: Logtail configuration describes how to collect a specific type of logs on machines, parse the collected logs, and send the logs to the specified Logstore of Log Service. You can use the console to add Logtail configurations to a Logstore to enable the Logstore to receive logs that are collected based on the configurations.

Basic functions

Currently, the Logtail access service provides the following functions.

  • Real-time log collection: Logtail dynamically monitors log files and reads and parses incremental logs in real-time. There is a delay of less than 3s between log discovery and transfer of logs to Log Service.

    Note: Logtail does not support the collection of historical data. Logs with an interval of more than 5 minutes between the read time and generation time are discarded.

  • Automatic log rotation processing: Many applications rotate log files according to the file size or date. During the rotation process, the original log file is renamed and a new blank log file is created with data to be written in. (For example, the monitored app.LOG is rotated to generate app.LOG.1 and app.LOG.2.) You can specify the file (for example, app.LOG) to which collected logs are written. Logtail automatically detects the log rotation process and ensures that no log data is lost during this process.

    Note: Data may be lost if log files are rotated multiple times within several seconds.

  • Automatic handling of collection exceptions: Logtail performs proactive retry based on specific scenarios in the case of data transfer failures due to exceptions (such as Log Service errors, network measures, and quota overruns). If retry fails, Logtail writes the data to the local cache and then resends the data after a time.

    Note: The local cache is located in the disk of your server. If the cached data is not received by Log Service within 24 hours, it is discarded and deleted from the cache.

  • Flexible collection policy configuration: You can perform Logtail configuration to flexibly specify how logs are collected on an ECS server. Specifically, you can select log directories and files (by means of exact match or fuzzy match using wildcards) based on the actual scenario. You can customize an extraction method for log collection and set the names of extracted fields. (Log extraction by regular expression is supported.) Because the log data models of Log Service require that each log have precise timestamp information, Logtail provides custom log time formats, allowing you to extract the required timestamp information from log data of different formats.

  • Automatic synchronization of collection configurations: After you create or update configurations on the Log Service console, Logtail automatically accepts and applies the configurations within 3 minutes. No collected data is lost during the configuration update process.

  • Automatic agent upgrade: After you manually install Logtail on a server, Log Service automatically performs agent O&M and upgrade. No log data is lost during the agent upgrade process.

  • Status monitoring: To prevent the Logtail agent from consuming too many resources and thus affecting your services, Logtail monitors its resource (CPU and memory) consumption in real time. The Logtail agent automatically restarts when the resource usage limit is exceeded to avoid impact on the ongoing operations on the machine. The agent proactively limits network traffic to prevent excessive bandwidth consumption.

    Note:

    • Log data may be lost when the Logtail agent restarts.
    • If the Logtail agent is exited due to an exception of its processing logic, the corresponding protective mechanism is triggered and the agent is restarted to continue log collection. However, log data may be lost before restart.
  • Transferred data signature: To prevent data tampering during the transfer process, the Logtail agent proactively obtains your Alibaba Cloud access key to sign all log data packets before they are sent.

    Note: The Logtail agent obtains your Alibaba Cloud access key over HTTPS to ensure key security.

Features

  • Non-invasive log collection based on log files.

    You do not need to modify any application code, and log collection does not affect the operating logic of your applications.

  • Exception handling during the log collection process in a stable manner.

    Logtail takes data security measures (such as proactive retry and local caching) when the network or Log Service has an exception or when user data temporarily exceeds the reserved write bandwidth limit.

  • Centralized management capability based on Log Service.

    After installing Logtail, you can configure the data sources, collection modes and other parameters on the client for all the servers without the need to log on to the servers and make configurations separately.

  • Comprehensive management mechanism.

    To ensure that the collection agent running on your machine does not significantly impact the performance of your services, Logtail protects and limits the use of CPU, memory, and network resources.

Processing capabilities and constraints

The following table lists the per-server processing capabilities and constraints of the Logtail access service.

Item Processing capability and constraints
File encoding UTF-8/GBK-encoded log files are supported. If log files are encoded in other formats, undefined behaviors such as garbled characters and lost data occur. UTF-8 encoding is recommended for improved processing performance.
Log processing throughput capacity The raw log processing traffic is limited to 1 MB/s by default. Data is sent through the Alibaba Cloud intranet. Logs may be lost when the traffic limit is exceeded. You can set the limit to about 50 MB/s at most in accordance with this document.
Network error handling Local cache storage up to 500 MB is supported. Logtail caches data locally when the network or Log Service has an exception or when user data temporarily exceeds the reserved write bandwidth limit. Logtail then retries sending as soon as possible.
Configuration update The delay of applying updated configurations is 30s.
Status management detection Logtail automatically restarts in the case of an exception (abnormal program exit or resource limit overruns).
Monitored directory count Logtail proactively restricts the formats of directories that can be monitored to prevent user resource overconsumption. When the monitoring upper limit is reached, Logtail stops monitoring more directories and log files. Up to 3,000 directories (including subdirectories) can be monitored.
Soft link support Monitored directories can be soft links.
Log file size Unrestricted.
Single log size The maximum size of a single log is 512 KB. Each line of a multiline log, after the pattern that delineates it, may be up to 512 KB.
Regular expression type Perl-compatible regular expressions are used.
Thank you! We've received your feedback.