This topic describes how to create a Logstash collection configuration.

Background information

Related plug-ins

  • logstash-input-file
    This plug-in collects log files in tail mode. For more information, see logstash-input-file.
    Note The path parameter indicates a file path. When you specify a file path, you must use delimiters in the UNIX format, for example, C:/test/multiline/*.log. Otherwise, fuzzy match cannot be used.
  • logstash-output-logservice
    This plug-in sends the logs collected by the logstash-input-file plug-in to Log Service.
    Parameter Description
    endpoint The endpoint of Log Service, for example, http://regionid.example.com.
    project The name of a Log Service project.
    logstore The name of a Logstore.
    topic The topic of logs. The default value is NULL.
    source The log source. If the value of this parameter is NULL, the IP address of the current server is used as the log source. Otherwise, the log source is the specified parameter value.
    access_key_id The AccessKey ID of the Alibaba Cloud account.
    access_key_secret The AccessKey secret of the Alibaba Cloud account.
    max_send_retry The maximum number of retries performed when a data packet fails to be sent to Log Service. Data packets that fail to be sent within the retry period are discarded. The retry interval is 200 ms.

Procedure

  1. Create a collection configuration.
    Add a configuration file to the C:\logstash-2.2.2-win\conf\ directory. Restart Logstash to make the configuration take effect.

    You can create a configuration file for each type of logs. The file name format is *.conf. We recommend that you save all logs in the C:\logstash-2.2.2-win\conf\ directory to facilitate log management.

    Note Configuration files must be encoded in UTF-8 without BOM. You can use Notepad++ to change the file encoding format.
  2. Verify the configuration.
    1. Run PowerShell or cmd.exe to go to the Logstash installation directory and run the following command to verify the configuration:
      PS C:\logstash-2.2.2-win\bin> .\logstash.bat agent --configtest --config C:\logstash-2.2.2-win\conf\iis_log.conf
    2. Modify the collection configuration file by adding the temporary configuration item rubydebug in the output phase to send the collected results to the console. Set the type field in the configuration as needed.
      output {
      if [type] == "***" {
        stdout { codec => rubydebug }
        logservice {
        ...
        }
       }
      }
    3. Run PowerShell or cmd.exe to go to the Logstash installation directory and run the following command:
      PS C:\logstash-2.2.2-win\bin> .\logstash.bat agent -f C:\logstash-2.2.2-win\conf
    After the verification, stop the logstash.bat process and remove the temporary configuration item rubydebug.

What to do next

Start logstash.bat in PowerShell. The logstash process works in the frontend. Configuring Logstash as a Windows service is usually required for configuration testing and collection debugging. We recommend that you configure Logstash as a Windows service after debugging so that Logstash can work in the backend and start automatically upon the power-on of the server where Logstash is installed. For more information about how to set Logstash to a Windows service, see Configure Logstash as a Windows service.