This topic describes the basic configuration files and record files of Logtail. When Logtail is active, it uses the configuration files and generates record files.

Startup configuration file (ilogtail_config.json)

The ilogtail_config.json file is used to set Logtail startup parameters. For more information, see Configure the startup parameters of Logtail.
Note
  • The file must be a valid JSON file. Otherwise, Logtail cannot be started.
  • If you modify the file, you must restart Logtail to validate your modifications.
After Logtail is installed on a server, you can perform the following operations on the ilogtail_config.json file:
  • Modify the Logtail runtime parameters.
  • Check whether the command to install Logtail is correct.

    The values of the config_server_address and data_server_list parameters in the ilogtail_config.json file depend on the command that you select. If the region in the command is inconsistent with the region where the Log Service project is located or the URL in the command is inaccessible, the command is incorrect. In this case, Logtail cannot collect logs and must be reinstalled.

  • File path
    • Linux: /usr/local/ilogtail/ilogtail_config.json
    • Windows:
      • 64-bit: C:\Program Files (x86)\Alibaba\Logtail\ilogtail_config.json
      • 32-bit: C:\Program Files\Alibaba\Logtail\ilogtail_config.json
    • Containers: The file is stored in a Logtail container. The file path is specified in the environment variable ALIYUN_LOGTAIL_USER_ID of the Logtail container. You can run the command docker inspect ${logtail_container_name} | grep ALIYUN_LOGTAIL_USER_ID to view the file path. Example: /etc/ilogtail/conf/cn_hangzhou/ilogtail_config.json.
  • Sample file 
    $cat /usr/local/ilogtail/ilogtail_config.json
{
    "config_server_address" : "http://logtail.cn-hangzhou-intranet.log.aliyuncs.com",
    "data_server_list" :
    [
        {
            "cluster" : "cn-hangzhou",
            "endpoint" : "cn-hangzhou-intranet.log.aliyuncs.com"
        }
    ],
    "cpu_usage_limit" : 0.4,
    "mem_usage_limit" : 100,
    "max_bytes_per_sec" : 2097152,
    "process_thread_count" : 1,
    "send_request_concurrency" : 4,
    "streamlog_open" : false
}

Configuration file of user identity

The configuration file of user identity contains the ID of your Alibaba Cloud account. The file is used to indicate that the Alibaba Cloud account is authorized to access the server and collect logs. For more information, see Configure a user identifier.

Note
  • You must configure the user identity file when you collect logs from ECS instances that do not belong to your current Alibaba Cloud account, servers in your on-premises, or servers that are provided by third-party cloud service providers.
  • The configuration file of user identity must contain the ID of your Alibaba Cloud account. You cannot configure the ID of a RAM user for the file.
  • You need to configure the name of the user identity file. You do not need to configure the file extension.
  • You can configure multiple user identity files for a server. However, you can configure only one user identity file for a Logtail container.
  • File path
    • In a Linux-based server: The file is stored in the /etc/ilogtail/users/ directory.
    • In a Windows-based server: The file is stored in the C:\LogtailData\users\ directory.
    • Containers: The file is stored in a Logtail container. The file path is specified in the environment variable ALIYUN_LOGTAIL_USER_ID of the Logtail container. You can run the command docker inspect ${logtail_container_name} | grep ALIYUN_LOGTAIL_USER_ID to view the file path.
  • Sample file 
    $ls /etc/ilogtail/users/
**************** *****************

Custom identifier file (user_defined_id)

The user_defined_id file is used to configure a custom ID for a machine group. For more information, see Create a custom identifier-based machine group.

Note If you want to configure a custom identifier for a machine group, you must configure the user_defined_id file.
  • File path
    • In a Linux-based server: The file is stored in the /etc/ilogtail/user_defined_id directory.
    • In a Windows-based server: The file is stored in the C:\LogtailData\user_defined_id directory.
    • Containers: The file is stored in a Logtail container. The file path is specified in the environment variable ALIYUN_LOGTAIL_USER_DEFINED_ID of the Logtail container. You can run the command docker inspect ${logtail_container_name} | grep ALIYUN_LOGTAIL_USER_DEFINED_ID to view the file path.
  • Sample file 
    $cat /etc/ilogtail/user_defined_id
aliyun-ecs-rs1e16355

Logtail configuration file for log collection (user_log_config.json)

The user_log_config.json file records the Logtail configurations for log collection. The configurations are synchronized from Log Service. The file is updated whenever the Logtail configurations for log collection are updated. The file type is JSON. You can use the user_log_config.json file to check whether Logtail configurations for log collection are synchronized to the server. If the file exists and the configurations in the file are the same as the log collection configurations in the Log Service console, the synchronization succeeds.

Note We recommend that you do not modify the file unless you need to specify sensitive information, such as the AccessKey pair and database password.
  • File path
    • In a Linux-based server: The file is stored in the /usr/local/ilogtail/user_log_config.json directory.
    • Windows
      • 64-bit: The file is stored in the C:\Program Files (x86)\Alibaba\Logtail\user_log_config.json directory.
      • 32-bit: The file is stored in the C:\Program Files\Alibaba\Logtail\user_log_config.json directory.
    • Containers: The file is stored in the /usr/local/ilogtail/user_log_config.json directory.
  • Sample file 
    $cat /usr/local/ilogtail/user_log_config.json
{
   "metrics" : {
      "##1.0##k8s-log-c12ba2028*****939f0b$app-java" : {
         "aliuid" : "16542189*****50",
         "category" : "app-java",
         "create_time" : 1534739165,
         "defaultEndpoint" : "cn-hangzhou-intranet.log.aliyuncs.com",
         "delay_alarm_bytes" : 0,
         "enable" : true,
         "enable_tag" : true,
         "filter_keys" : [],
         "filter_regs" : [],
         "group_topic" : "",
         "local_storage" : true,
         "log_type" : "plugin",
         "log_tz" : "",
         "max_send_rate" : -1,
         "merge_type" : "topic",
         "plugin" : {
            "inputs" : [
               {
                  "detail" : {
                     "IncludeEnv" : {
                        "aliyun_logs_app-java" : "stdout"
                     },
                     "IncludeLable" : {
                        "io.kubernetes.container.name" : "java-log-demo-2",
                        "io.kubernetes.pod.namespace" : "default"
                     },
                     "Stderr" : true,
                     "Stdout" : true
                  },
                  "type" : "service_docker_stdout"
               }
            ]
         },
         "priority" : 0,
         "project_name" : "k8s-log-c12ba2028c*****ac1286939f0b",
         "raw_log" : false,
         "region" : "cn-hangzhou",
         "send_rate_expire" : 0,
         "sensitive_keys" : [],
         "tz_adjust" : false,
         "version" : 1
      }
   }
}

AppInfo record file (app_info.json)

The app_info.json file records the information about Logtail, such as the startup time, retrieved IP address, and hostname.

If you associate the IP address of the server with the hostname in the /etc/hosts file, Logtail automatically retrieves the IP address of the server. If you do not associate the IP address of the server with the hostname, Logtail retrieves the IP address of the first network interface controller (NIC) on the server.

Note
  • The AppInfo record file records only the information about Logtail.
  • If you modify the hostname of the server or other network configurations, you must restart Logtail to retrieve a new IP address.
  • File path
    • In a Linux-based server: The file is stored in the /usr/local/ilogtail/app_info.json directory.
    • Containers: The file is stored in the /usr/local/ilogtail/app_info.json directory.
    • Windows
      • 64-bit: The file is stored in the C:\Program Files (x86)\Alibaba\Logtail\app_info.json directory.
      • 32-bit: The file is stored in the C:\Program Files\Alibaba\Logtail\app_info.json directory.
  • Sample file 
    $cat /usr/local/ilogtail/app_info.json
{
   "UUID" : "",
   "hostname" : "logtail-ds-slpn8",
   "instance_id" : "E5F93BC6-B024-11E8-8831-0A58AC14039E_1**. ***. ***.***_1536053315",
   "ip" : "1**. ***. ***.***",
   "logtail_version" : "0.16.13",
   "os" : "Linux; 3.10.0-693.2.2.el7.x86_64; #1 SMP Tue Sep 12 22:26:13 UTC 2017; x86_64",
   "update_time" : "2018-09-04 09:28:36"
}
    Field Description
    UUID The serial number of the server.
    hostname The hostname of the server.
    instance_id The unique identifier of Logtail. This identifier is randomly generated.
    ip The IP address that is retrieved by Logtail. If Logtail does not retrieve an IP address, the value of this field is null. Logtail cannot run as expected. You must specify an IP address for the server and then restart Logtail.
    Note If you configure an IP address-based identifier for a machine group, make sure that the IP address-based identifier includes the IP address indicated by the field. If the IP address is not included in the IP address-based identifier, log on to the Log Service console and modify the IP address. Wait for 1 minute and then view the value of this field.
    logtail_version The version of Logtail.
    os The version of the operating system.
    update_time The last startup time of Logtail.

Logtail operational log file (ilogtail.LOG)

The ilogtail.LOG file records the operational logs of Logtail. The severity levels of logs in ascending order include INFO, WARN, ERROR. Logs with the INFO severity level can be ignored.

If an exception occurs during log collection, troubleshoot the exception based on the exception type and Logtail operational logs. For more information, see How do I view Logtail collection errors?.

Note If you submit a ticket for a log collection error, upload Logtail operational logs.
  • File path
    • In a Linux-based server: The file is stored in the /usr/local/ilogtail/ilogtail.LOG directory.
    • Containers: The file is stored in the /usr/local/ilogtail/ilogtail.LOG directory.
    • Windows
      • 64-bit: The file is stored in the C:\Program Files (x86)\Alibaba\Logtail\ilogtail.LOG directory.
      • 32-bit: The file is stored in the C:\Program Files\Alibaba\Logtail\ilogtail.LOG directory.
  • Sample file 
    $tail /usr/local/ilogtail/ilogtail.LOG
[2018-09-13 01:13:59.024679]    [INFO]    [3155]    [build/release64/sls/ilogtail/elogtail.cpp:123]    change working dir:/usr/local/ilogtail/
[2018-09-13 01:13:59.025443]    [INFO]    [3155]    [build/release64/sls/ilogtail/AppConfig.cpp:175]    load logtail config file, path:/etc/ilogtail/conf/ap-southeast-2/ilogtail_config.json
[2018-09-13 01:13:59.025460]    [INFO]    [3155]    [build/release64/sls/ilogtail/AppConfig.cpp:176]    load logtail config file, detail:{
   "config_server_address" : "http://logtail.ap-southeast-2-intranet.log.aliyuncs.com",
   "data_server_list" : [
      {
         "cluster" : "ap-southeast-2",
         "endpoint" : "ap-southeast-2-intranet.log.aliyuncs.com"
      }
]

Operational logs of Logtail plug-ins (logtail_plugin.LOG)

The logtail_plugin.LOG file records the operational logs of Logtail plug-ins. The severity levels of logs in ascending order include INFO, WARN, ERROR. Logs with the INFO severity level can be ignored.

If an exception such as CANAL_RUNTIME_ALARM occurs, you can troubleshoot the exception based on the logtail_plugin.LOG file.

Note If you submit a ticket for a Logtail plug-in exception, upload Logtail operational logs.
  • File path
    • In a Linux-based server: The file is stored in the /usr/local/ilogtail/logtail_plugin.LOG directory.
    • Containers: The file is stored in the /usr/local/ilogtail/logtail_plugin.LOG directory.
    • In a Windows-based server:
      • 64-bit: The file is stored in the C:\Program Files (x86)\Alibaba\Logtail\logtail_plugin.LOG directory.
      • 32-bit: The file is stored in the C:\Program Files\Alibaba\Logtail\logtail_plugin.LOG directory.
  • Sample file 
    $tail /usr/local/ilogtail/logtail_plugin.LOG
2018-09-13 02:55:30 [INF] [docker_center.go:525] [func1] docker fetch all:start
2018-09-13 02:55:30 [INF] [docker_center.go:529] [func1] docker fetch all:stop
2018-09-13 03:00:30 [INF] [docker_center.go:525] [func1] docker fetch all:start
2018-09-13 03:00:30 [INF] [docker_center.go:529] [func1] docker fetch all:stop
2018-09-13 03:03:26 [INF] [log_file_reader.go:221] [ReadOpen] [##1.0##sls-zc-test-hz-pub$docker-stdout-config,k8s-stdout]    open file for read, file:/logtail_host/var/lib/docker/containers/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624-json.log    offset:40379573    status:794354-64769-40379963
2018-09-13 03:03:26 [INF] [log_file_reader.go:221] [ReadOpen] [##1.0##k8s-log-c12ba2028cfb444238cd9ac1286939f0b$docker-stdout-config,k8s-stdout]    open file for read, file:/logtail_host/var/lib/docker/containers/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624-json.log    offset:40379573    status:794354-64769-40379963
2018-09-13 03:04:26 [INF] [log_file_reader.go:308] [CloseFile] [##1.0##sls-zc-test-hz-pub$docker-stdout-config,k8s-stdout]    close file, reason:no read timeout    file:/logtail_host/var/lib/docker/containers/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624-json.log    offset:40379963    status:794354-64769-40379963
2018-09-13 03:04:27 [INF] [log_file_reader.go:308] [CloseFile] [##1.0##k8s-log-c12ba2028cfb444238cd9ac1286939f0b$docker-stdout-config,k8s-stdout]    close file, reason:no read timeout    file:/logtail_host/var/lib/docker/containers/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624/7f46afec6a14de39b59ee9cdfbfa8a70c2fa26f1148b2e2f31bd3410f5b2d624-json.log    offset:40379963    status:794354-64769-40379963
2018-09-13 03:05:30 [INF] [docker_center.go:525] [func1] docker fetch all:start
2018-09-13 03:05:30 [INF] [docker_center.go:529] [func1] docker fetch all:stop

Container path mapping file (docker_path_config.json)

The docker_path_config.json file is created when you collect container logs. The file records the path mapping between container log files and host log files. The file type is JSON.

If the DOCKER_FILE_MAPPING_ALARM message appears when you troubleshoot a log collection exception, it indicates that Docker files fail to be mapped to host files. You can use the docker_path_config.json file to troubleshoot the exception.

Note
  • The docker_path_config.json file cannot be modified. If you delete the file, another file is automatically created. This action does not affect your business.
  • If you submit a ticket for a container log collection exception, upload this file.
  • File path

    /usr/local/ilogtail/docker_path_config.json

  • Sample file 
    $cat /usr/local/ilogtail/docker_path_config.json
{
   "detail" : [
      {
         "config_name" : "##1.0##k8s-log-c12ba2028cfb444238cd9ac1286939f0b$nginx",
         "container_id" : "df19c06e854a0725ea7fca7e0378b0450f7bd3122f94fe3e754d8483fd330d10",
         "params" : "{\n   \"ID\" : \"df19c06e854a0725ea7fca7e0378b0450f7bd3122f94fe3e754d8483fd330d10\",\n   \"Path\" : \"/logtail_host/var/lib/docker/overlay2/947db346695a1f65e63e582ecfd10ae1f57019a1b99260b6c83d00fcd1892874/diff/var/log\",\n   \"Tags\" : [\n      \"nginx-type\",\n      \"access-log\",\n      \"_image_name_\",\n      \"registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest\",\n      \"_container_name_\",\n      \"nginx-log-demo\",\n      \"_pod_name_\",\n      \"nginx-log-demo-h2lzc\",\n      \"_namespace_\",\n      \"default\",\n      \"_pod_uid_\",\n      \"87e56ac3-b65b-11e8-b172-00163f008685\",\n      \"_container_ip_\",\n      \"172.20.4.224\",\n      \"purpose\",\n      \"test\"\n   ]\n}\n"
      }
   ],
   "version" : "0.1.0"
}