All Products
Search
Document Center

System fields in log cleansing

Last Updated: Apr 28, 2018

In the log cleansing process, besides custom fields, some default fields are added, such as _line, _hostIp, and _sysTime.

_line

The _line field indicates each line of the log. For example, the log format is as follows:

  1. 2016-11-08 11:00:01|user_abc|123456|PlaceOrder
  2. 2016-11-08 11:00:02|user_abc|123456|Payment
  3. 2016-11-08 11:10:01|user_abc|123456|Return

In the first splitting, the input key is _line of each line of the log. The custom splitting form is as follows:

line

_hostIp

The _hostIp field indicates the source IP address of each line of the log. ARMS currently supports the ECS, LogHub, and SDK data sources. The following table lists the relationship between ARMS and the data sources.

Data source Supported or not Note
ECS data source Supported None
LogHub data source Depends Unsupported if data in LogHub is written using the SDK, and supported if data in LogHub is captured by Log Service
SDK data source Unsupported None

Currently, _hostIp is available only in custom splitting mode, but not in intelligent splitting mode. The preceding log is used as an example. Click Log Splitting Preview:

hostIp

In the preceding figure, the _hostIp field is set to 127.0.0.1 because the local mode is applied. After you click Log Splitting Preview, the value of the _hostIp field is 127.0.0.1 for all data sources that ARMS is connected to. When a job runs, real data is generated.

_sysTime

The _sysTime field indicates the log processing time. If your log does not contain any business time, you can use the _sysTime field for aggregation computing.