This topic introduces the Logtail configurations and their scenarios.
Configuration overview
The Logtail configuration process involves three steps: installing Logtail, creating a machine group, and creating a collection configuration. Each step allows for fine-grained custom configurations.
Logtail
Logtail is a log collection agent designed to collect logs from Alibaba Cloud Elastic Compute Service (ECS) instances, self-managed data centers, or servers from other cloud service providers. To use Logtail, you must first install it on the target machine.
For information on installing, running, checking status, upgrading, and uninstalling Logtail in various environments, see Install Logtail.
Logtail has different network requirements for data transmission depending on the type of target machine. For more information, see Select a network type.
Logtail uses a series of configuration files and generates record files during runtime. For more information, see Logtail configuration files and record files.
To prevent excessive server resource consumption, Simple Log Service limits Logtail's collection performance. To enhance Logtail's collection capabilities, configure the startup parameters of Logtail.
To accelerate data collection, enable log collection acceleration.
For real-time monitoring of important logs, Simple Log Service offers built-in alerting and monitoring rules with notifications through channels such as DingTalk.
When collecting logs with Logtail, you may encounter issues such as regular expression parsing failures, incorrect file paths, or traffic exceeding shard capacity. Simple Log Service offers Logtail diagnostics to help identify these errors. For Linux servers, a self-service diagnostic tool of Logtail is available.
Machine group
A machine group is a virtual collection of multiple target machines. Simple Log Service utilizes machine groups for batch configuration and management of log collection rules, accommodating dynamic environments.
For information about machine group concepts, such as user identifier, custom identifier-based machine group, and IP address-based machine group, see Introduction to machine groups.
To use Logtail for collecting logs from a server in a different Alibaba Cloud account, a third-party cloud service provider, or a self-managed data center, you must configure a user identifier.
Simple Log Service offers two types of machine groups: IP address and custom identifier. The IP address type is easier to create, while the custom identifier type supports automatic scaling without address conflicts.
Collection configuration items
These are essential rules for log data collection and processing. They facilitate efficient log collection, structured parsing, and filtering through flexible configurations.
For information about how to create, view, modify, and delete Logtail configurations in the console, see Manage Logtail configurations for log collection.
Logtail configuration items mainly consist of three parts:
Global configurations are used to organize collected text logs into different log topics.
Input configurations are used to define the collection details.
Processor configurations are used to transform raw logs into structured data.
References
In addition to console operations, the Logtail configuration process also supports API and SDK methods.