Log Service is a platform provided by Alibaba Cloud for collecting, storing, and querying massive logs. You can use Log Service to centrally collect all the logs from the service cluster. It also supports real-time consumption and query.
This document demonstrates the basic workflow of configuring Logtail to collect Alibaba Cloud Elastic Compute Service (ECS) logs in the Windows environment. This case is related to the basic functions of Log Service, such as collecting logs and querying logs in real time, and is an entry-level user guide of Log Service.
Log service operation process
Step 1. Getting started
1. Activate Log Service
Use a registered Alibaba Cloud account to log on to the Log Service product page and click Get it Free.
2. Create an AccessKey (optional)
3. Create a project
If you have logged on to the Log Service console for the first time, the system prompts you to create a project. You can also click Create Project in the upper-right corner to create a project.
4. Create a Logstore
Step 2. Install Logtail client on ECS instance
1. Download the installation package
Download the Logtail installation package to an ECS instance. Click here to download the Windows installation package.
2. Install Logtail
Extract the installation package to the current directory and then enter the
logtail_installer directory. Run cmd as an administrator, and run the installation command
.\logtail_installer.exe install cn_hangzhou.
Step 3. Configure data import wizard
In the Log Service console, click the project name to go to the Logstore List page. Click 1 at the right of the Logstore to enter the Logtail configuration. You can also click Manage at the right of the Logstore to create a configuration in the Logtail configuration list.
Logtail configuration process includes the following steps: Select Data Source, Configure Data Source , Search, Analysis, and Visualization, Shipper & ETL. The last two steps are optional.
1. Select data source
Log Service supports the log collection of many cloud products, self-built softwares, and custom data. This document uses collecting text logs as an example. For more information, see Collect text logs.
Click Text in Other Sources, and then click Next.
2. Configure data source
- Specify the Configuration name and Log path.
Follow the page prompts to enter the configuration name, log path, and log file name. The log file name can be a full name, and supports wildcard matching at the same time.
- Specify the log collection mode.
- Enter the log sample.
You must enter the log sample if Delimiter Mode or Full Mode is selected as the log collection mode. Log Service supports parsing the log sample according to selected configuration when configuring Logtail. If the log sample failed to be parsed, you must modify the delimiter configurations or regular expressions. Enter the log sample to be parsed in the Log Sample field.
- Specify the delimiter.
You can specify the delimiter as a tab, a vertical line, or a space. You can also customize the delimiter. Select the delimiter according to your log format. Otherwise, logs fail to be parsed.
- Specify the key in the log extraction results.
After you enter the log sample and select the delimiter, Log Service extracts log fields according to your selected delimiter, and defines them as Value. You must specify the corresponding Key for the Value.
- Configure the advanced options as needed.
Generally, keep the default configurations of the advanced options. For how to configure the advanced options, see the related descriptions in Collect text logs.
- Apply to the machine group.
If you have not created a machine group before, create a machine group according to the page prompts. Then, apply the Logtail configuration to the machine group.Note To create Armory to associate with the machine group, jump to the specified internal link as instructed on the page.
After completing these steps, Log Service begins to collect logs from the Alibaba Cloud ECS instance immediately. You can consume the collected logs in real time in the console and by using API/SDK.
To query, analyze, ship, or consume the logs, click Next.
Search, analysis, and visualization
After the collection is configured, your ECS logs are collected in real time. To query and analyze the collected logs, configure the indexes in the data import wizard as follows.
You can click Search on the Logstore List page to go to the query page. Click Enable in the upper-right corner and configure the indexes on the displayed Search & Analysis page.
- Full text index attributes
You can enable the Full Text Index Attributes. Confirm whether to enable Case Sensitive, and confirm the Token contents.
- Key/value index attributes
Click the plus icon at the right of Key to add a line. Configure the Key, Type, Alias, Case Sensitive, and Token, and select whether to enable analytics.
- Full text or key/value indexes attributes at least one must be enabled. When both types are enabled, key/value index attributes prevail.
- When the index type is long or double, the Case Sensitive and Token attributes are unavailable.
- For how to configure indexes, see Overview.
- To use Nginx template or MNS template, configure the attributes on the Search & Analysis page after clicking Enable on the query page.
After configuring the query and analysis, click Next if you want to configure the log shipping. To experience the query and analysis, go back to the Logstore List page and click Search to go to the query page. You can enter the keyword, topic, or query & analysis statement, and select the time range to query logs. Log Service provides intuitive histograms to preview the query results. You can click the histogram to query logs in a more detailed time range. For more information, see Overview.
Log Service also supports querying and analyzing logs in many ways such as quick query and statistical graphs. For more information, see Other functions.
For example, to query all the logs within the last 15 minutes, you can set an empty query condition and select 15 min as the time range.
Log Service not only supports collecting data with multiple sources and formats in batch, managing and maintaining the data, but also supports shipping log data to cloud products such as Object Storage Service (OSS) for calculation and analysis.
To ship logs to OSS, click Enable.
This document uses OSS storage as an example. See Ship logs to OSSto complete the authentication.
Besides the basic functions such as accessing, querying, and analyzing logs, Log Service also provides many ways to consume logs. For more information, see User Guide.