Simple Log Service:Runtime management

Common LoongCollector commands

Start LoongCollector

If you have specific performance requirements for data collection, modify the startup parameter configuration file (ilogtail_config.json) before you start LoongCollector.

sudo /etc/init.d/loongcollectord start

Stop LoongCollector

sudo /etc/init.d/loongcollectord stop

Check the LoongCollector status

sudo /etc/init.d/loongcollectord status # A return value of "loongcollector is running" indicates a successful start.

Check the LoongCollector version

cat /usr/local/ilogtail/app_info.json # The version information is stored in the loongcollector_version field.

Restart LoongCollector

sudo /etc/init.d/loongcollectord restart

Uninstall LoongCollector

1. In the sample code, replace ${region_id} with cn-hangzhou. To accelerate the download, replace ${region_id} with the region of your ECS instance. For more information, see Regions.

   wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;

2. Run the uninstall command.

   chmod +x loongcollector.sh; sudo ./loongcollector.sh uninstall;

Upgrade LoongCollector

Upgrade LoongCollector to a later version or upgrade from Logtail to LoongCollector. The upgrade commands are the same for both scenarios.

Upgrade LoongCollector or upgrade from Logtail to LoongCollector

1. Run the download command on your server to obtain the latest installation package. In the sample code, replace ${region_id} with cn-hangzhou. To accelerate the download, replace ${region_id} with the region of your ECS instance. For more information, see Regions.

   wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh -O loongcollector.sh;

2. Run the upgrade command. Use the upgrade command. If you use the install command, the installation is overwritten and your original configuration is lost.

   chmod +x loongcollector.sh; sudo ./loongcollector.sh upgrade;

3. If the following information is displayed, the upgrade is successful.

   Upgrade loongcollector files successfully.
   Starting loongcollector ...
   Upgrade loongcollector successfully.

Roll back from LoongCollector to Logtail

1. Run the download command on your server to obtain the installation package. In the sample code, replace ${region_id} with cn-hangzhou. To accelerate the download, replace ${region_id} with the region of your ECS instance. For more information, see Regions.

   wget https://logtail-release-${region_id}.oss-${region_id}.aliyuncs.com/linux64/logtail.sh -O logtail.sh; 

2. Run the rollback command. To specify a version, such as 1.8.7, refer to the comment and replace the version number as needed.

   chmod +x logtail.sh; sudo ./logtail.sh upgrade;
   #chmod +x logtail.sh; sudo ./logtail.sh upgrade -v -v 1.8.7;

Startup parameter configuration file (ilogtail_config.json)

• Description: This file configures the startup parameters for LoongCollector. Incorrect parameter settings can affect collection performance or cause collection errors.

• Path: /usr/local/ilogtail/ilogtail_config.json.

• Scenario: Simple Log Service (SLS) limits the collection performance of LoongCollector to avoid consuming excessive server resources. If the collection performance does not meet your needs, adjust the startup parameters.

• File example:

  {
      "primary_region" : "cn-beijing", // Default region. You can ignore this parameter.
      "config_servers" : // The address used to obtain collection configurations.
      [
          "http://logtail.cn-beijing.log.aliyuncs.com"
      ],
      "data_servers" : // The address for data transmission with SLS.
      [
          {
              "region" : "cn-beijing",
              "endpoint_list": [
                  "cn-beijing.log.aliyuncs.com"
              ]
          }
      ],
      "cpu_usage_limit" : 0.4,
      "mem_usage_limit" : 384,
      "max_bytes_per_sec" : 20971520,
      "bytes_per_sec" : 1048576,
      "buffer_file_num" : 25,
      "buffer_file_size" : 20971520,
      "buffer_map_num" : 5
  }

• Parameters: The following table lists only the common startup parameters that you might need to configure. Add or modify them as needed. Keep the default values for parameters not listed in the table.

  ilogtail_config.json startup parameters

  Parameter	Type	Description	Example
  cpu_usage_limit	double	The CPU usage threshold, calculated per core. The details are described as follows: Valid values: 0.1 to the number of CPU cores on the machine. Default value: 0.4. Warning cpu_usage_limit is a soft limit. The actual CPU usage may exceed this value. If the limit is exceeded for five minutes, a circuit breaker is triggered and LoongCollector automatically restarts. For example, a value of 0.4 means SLS attempts to limit the collector's CPU usage to 40% of a single CPU core. If the usage exceeds this limit, LoongCollector restarts. Typically, a single core can process about 100 MB/s in minimalist mode or 20 MB/s in full regular expression mode.	"cpu_usage_limit" : 0.4
  mem_usage_limit	int	The memory usage threshold. The details are described as follows: Valid values: 128 MB to 8192 MB. Default value: 384 MB for hosts and 2048 MB for ACK components. Warning mem_usage_limit is a soft limit. The actual memory usage may exceed this value. If the limit is exceeded for five minutes, a circuit breaker is triggered and LoongCollector automatically restarts. The collection rate, the number of monitored directories and files, and sending blockages are related to the mem_usage_limit parameter. For more information, see Limits.	"mem_usage_limit" : 384
  max_bytes_per_sec	int	The maximum traffic for sending raw data per second. The details are described as follows: Valid values: 1024 Byte/s to 52428800 Byte/s. Default value: 20971520 Byte/s. Important A value greater than 20971520 Byte/s (20 MB/s) indicates no speed limit. For example, a value of 2097152 sets the data sending rate to 2 MB/s.	"max_bytes_per_sec" : 2097152
  process_thread_count	int	The number of threads for data processing. The details are described as follows: Valid values: 1 to 64. Default value: 1. A single thread can typically handle 24 MB/s of data writes in minimalist mode or 12 MB/s in full regular expression mode. You do not need to adjust this parameter under normal circumstances.	"process_thread_count" : 1
  send_request_concurrency	int	The number of asynchronous concurrent requests. The details are described as follows: Valid values: 1 to 50. Default value: 20. If you have a high write TPS, set a higher concurrency value. Estimate that one concurrent request supports 0.5 MB/s to 1 MB/s of network throughput, depending on network latency. Note Setting the concurrency too high can occupy too many network ports. You may need to adjust TCP-related parameters.	"send_request_concurrency" : 4
  buffer_file_num	int	The maximum number of cache files. The details are described as follows: Valid values: 1 to 100. Default value: 25. If network issues or write quota overruns occur, the collector caches the parsed logs to local files in the installation directory. It retries sending them after the issue is resolved.	"buffer_file_num" : 25
  buffer_file_size	int	The maximum size of a single cache file in bytes. The details are described as follows: Valid values: 1048576 Byte to 104857600 Byte. Default value: 20971520 Byte. The maximum disk space for cache files is buffer_file_size × buffer_file_num.	"buffer_file_size" : 20971520
  buffer_file_path	String	The directory where cache files are stored. The default value is empty, which means cache files are stored in the installation directory /usr/local/ilogtail. If you set this parameter, you must manually move files named logtail_buffer_file_* from the original directory to the new directory. This ensures the collector can read the cache files and delete them after they are sent.	"buffer_file_path" : ""
  bind_interface	String	The name of the bound network interface card. The default value is empty, which means LoongCollector automatically binds to an available network interface card. If you set it to a specific network interface card, such as eth1, LoongCollector is forced to use that card to upload logs.	"bind_interface" : ""
  check_point_filename	String	The path where the checkpoint file is saved. Default value: /tmp/logtail_check_point.	"check_point_filename" : /tmp/logtail_check_point
  check_point_dump_interval	int	The interval for updating the Checkpoint file, in seconds. Default value: 900. By default, the Checkpoint file is updated every 15 minutes.	"check_point_dump_interval" : 900
  user_config_file_path	String	The path where the configuration file is saved. The default is the directory of the process binary. The default filename is user_log_config.json.	"user_config_file_path" : user_log_config.json
  discard_old_data	Boolean	Specifies whether to discard old logs. Default value: true. If set to true, logs older than 12 hours are discarded.	"discard_old_data" : true
  ilogtail_discard_interval	int	The age threshold for discarding old logs, in seconds. Default value: 43200 (12 hours).	"ilogtail_discard_interval": 43200
  working_ip	String	The default value is empty, which means the IP address is automatically obtained from the server. If you set a value, that value is reported as the server's IP address.	"working_ip" : ""
  working_hostname	String	The hostname reported for this server. The default value is empty, which means the hostname is automatically obtained from the server.	"working_hostname" : ""
  max_read_buffer_size	long	The maximum size for a single log entry, in bytes. Default value: 524288 (512 KB). Maximum value: 8388608 (8 MB). If a single log entry exceeds 524288 bytes, increase this value.	"max_read_buffer_size" : 524288
  oas_connect_timeout	long	The connection timeout for requests to get collection configurations or AccessKeys, in seconds. Default value: 5. Increase this value if you have a poor network connection that causes long connection times.	"oas_connect_timeout" : 5
  oas_request_timeout	long	The total request timeout for requests to get collection configurations or AccessKeys, in seconds. Default value: 10. Increase this value if you have a poor network connection.	"" : 10
  data_server_port	long	If you set data_server_port to 443, data is sent to SLS over HTTPS.	"data_server_port": 443
  enable_log_time_auto_adjust	Boolean	If you set enable_log_time_auto_adjust to true, the log time automatically adjusts to the server's local time. For data security, SLS validates the timestamp in each request, including requests from LoongCollector. It rejects requests with a timestamp that differs from the server time by more than 15 minutes. The collector uses the server's local time for requests. If the server's local time is changed, for example, to a future time for testing, requests are rejected and data writes fail. Use this parameter to allow the log time to adapt to the server's local time. Important When this feature is enabled, the offset between the SLS time and the server's local time is added to the log time. Because the offset is updated only when a request is rejected, the log time you query in SLS might not match the actual write time. Some LoongCollector logic relies on the system time to increase monotonically. Restart the collector after you adjust the machine's time.	"enable_log_time_auto_adjust": true
  accept_multi_config	Boolean	Specifies whether to allow multiple collection configurations to collect the same file. Default value: false. By default, a file can be collected by only one collection configuration. Use this parameter to remove this restriction. The processing for each collection configuration is independent. Allowing multiple configurations to collect the same file consumes more CPU and memory.	"accept_multi_config": true
  enable_checkpoint_sync_write	Boolean	Specifies whether to enable the sync write feature. Default value: false. The sync write feature is mainly used with the ExactlyOnce write feature. When ExactlyOnce is enabled, the collector records fine-grained, file-level Checkpoint information to the local disk. For performance reasons, the `sync` call is not used by default when writing Checkpoints. If the machine restarts before buffer data is written to disk, the Checkpoint may be lost. In this case, set enable_checkpoint_sync_write to true to enable sync writes. For more information, see Logtail configuration (Old version).	"enable_checkpoint_sync_write": false
  enable_env_ref_in_config	Boolean	Specifies whether to enable environment variable replacement in collection configurations. Default value: false. When this feature is enabled, use ${xxx} in the collection configuration in the console as a placeholder for the environment variable xxx. For example, if you set the collection path to /${xxx}/logs and the environment variable is xxx=user, the effective collection path is /user/logs. If your configuration needs to use the literal characters ${ or }, escape them as $${ and $}, respectively.	"enable_env_ref_in_config": false
  logreader_max_rotate_queue_size	Int	The maximum length of the rotation queue. Default value: 20. When log collection is blocked or delayed, pending files hold their file handles and wait in this queue. If collection is delayed and you need to control maximum disk usage, consider reducing this value. Warning If the number of delayed files exceeds this value, the collection of new files is skipped.	"logreader_max_rotate_queue_size" : 10
  data_endpoint_policy	string	The switching policy for SLS endpoints. The possible values are: Note Check the data_server parameter in the ilogtail_config.json file to see if a default domain name is configured. designated_first (Default) If a default domain name is specified for a region and is available, the system uses the default domain name. If the specified default domain name is unavailable, the system automatically selects an available domain name. If no default domain name is specified, the system automatically selects an available domain name. designated_locked If a default domain name is specified for a region, the system uses only that domain name, regardless of its availability. If no default domain name is specified, the system automatically selects an available domain name.	"data_endpoint_policy" : "designated_first"
  inotify_black_list	Array<String>	A blacklist for the inotify listener. This is an exact-match list. Directories in this list are not monitored by inotify.	"inotify_black_list": ["/tmp"]
  host_path_blacklist	String	A global blacklist for host paths, matched by substring. On Linux, separate multiple substrings with a colon (:). For example, "host_path_blacklist" : "/volumes/kubernetes.io~csi/nas-" prevents the collection of data from NAS mounts.	"host_path_blacklist" : "/volumes/kubernetes.io~csi/nas-"
  LOGTAIL_LOG_LEVEL	String	The log level. This must be configured as an environment variable. The default value is empty, which corresponds to info. Possible values are trace, debug, info, warning, error, and fatal.	LOGTAIL_LOG_LEVEL=info
  FORCE_RELEASE_STOP_CONTAINER_FILE	Boolean	Configuration: This can only be configured as an environment variable. Description: When this parameter is set to true, the collector immediately releases the container's file handle when the application container stops. This operation prevents the container from being unable to exit because the file handle is not released. Note: With this setting, the completeness of data collection from the container cannot be guaranteed. Add a delay of a few seconds before your application exits to ensure all logs are collected.	"FORCE_RELEASE_STOP_CONTAINER_FILE" : "true"

• Scenarios: The following examples show common scenarios and operations.

Collection performance configuration planning

To prevent the collector from consuming excessive server resources and affecting other services, SLS limits its collection performance. To improve collection performance, modify the startup parameters.

High network latency or unstable log transmission

Large single log entries are truncated

Logs older than 15 minutes are lost during transmission

Switch to a project in another region

Optimize long log transmission delays

Other common configuration files

The following are other configuration and information files used by LoongCollector during runtime:

LoongCollector collection configuration file (user_log_config.json)

• Description: This file records the collection configuration information that LoongCollector obtains from SLS. The file is updated whenever the collection configuration is updated. Do not modify this file, except to manually configure sensitive information such as AccessKey details or database passwords.

• Path: /usr/local/ilogtail/user_log_config.json.

• Scenario: Check this file to confirm whether a collection configuration has been delivered to the server. If this file exists and its content matches the collection configuration in SLS, the configuration has been successfully delivered.

AppInfo record file (app_info.json)

• Description: The app_info.json file records information such as the LoongCollector startup time, the retrieved IP address, and the hostname. This file is for recording purposes only. Any modifications to it do not take effect.

• Path: /usr/local/ilogtail/app_info.json.

• Scenario: View the server IP information identified by SLS during collection to verify that it matches the IP information in the IP-based machine group in the SLS console. This is typically used to resolve heartbeat failures in IP-based machine groups.

  If you have set a hostname-to-IP binding in the server's /etc/hosts file, the bound IP address is automatically retrieved. If no hostname binding is set, the IP address of the first network interface card is automatically retrieved. If the working_ip parameter in ilogtail_config.json is set, its value is used as the server's IP address.

LoongCollector runtime log (loongcollector.LOG)

• Description: The loongcollector.LOG file records the runtime logs of LoongCollector. The log levels, from lowest to highest, are INFO, WARN, and ERROR.

• Path: /usr/local/ilogtail/loongcollector.LOG.

• Scenario: If a collection error occurs, first refer to Diagnose and monitor LoongCollector to check for errors. Then, troubleshoot the issue by referring to the log collection error types and the LoongCollector runtime log.

User ID file

• Description: This file contains the Alibaba Cloud account ID of the project to which the collected logs belong. The filename is the account ID, with no suffix. It indicates that this account has permission to access and collect logs from this server.

• Path: /etc/ilogtail/users/{Alibaba Cloud account ID}.

• Scenario: You only need to configure a user ID when collecting logs from non-Alibaba Cloud ECS instances, self-managed servers, or servers from other cloud providers. If multiple accounts need to collect logs from the same server, create multiple user ID files on that server.

Custom identifier file

• Description: This file is used to configure a custom identifier. This identifier is used as the content for a custom-identifier machine group, which helps SLS discover LoongCollector on the server and establish a heartbeat.

• Path: /etc/ilogtail/user_defined_id.

• Scenario: This configuration is required when using a custom-identifier machine group. For more information, see Associate a machine group with a LoongCollector collection configuration.

CheckPoint file

• Description: The checkpoint mechanism records the current collection position to ensure log integrity.

• Path: The default path is /tmp/logtail_checkpoint.

• Scenario: Manage the checkpoint file by modifying the startup parameter file. For more information, see Limits.

FAQ

How to upgrade when the server cannot connect to the internet

1. On the server where you need to upgrade LoongCollector, run uname -m to determine the system architecture. Then, on a server that can access the public network, run the corresponding command. Replace ${region_id} with the region of your project.

   • ARM architecture:

     wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh;wget http://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/aarch64/main/loongcollector-linux64.tar.gz;

   • x86-64 architecture:

     wget https://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/loongcollector.sh;wget http://aliyun-observability-release-${region_id}.oss-${region_id}.aliyuncs.com/loongcollector/linux64/latest/x86_64/main/loongcollector-linux64.tar.gz;

2. Copy the installation script and package to the server where you need to upgrade LoongCollector, and then run the following command:

   chmod +x loongcollector.sh;./loongcollector.sh upgrade-local;