ConsumerGroup
The following table describes the data structure of a consumer group.
Parameter | Type | Example | Description |
name | String | test-consumer-group | The name of the consumer group. |
timeout | Integer | 300 | The timeout period. If Log Service does not receive heartbeats from a consumer within the timeout period, Log Service deletes the consumer. Unit: seconds. |
order | Boolean | false | Specifies whether to consume data in sequence. Valid values:
|
EncryptConf
The following table describes the data structure of an encryption configuration.
Parameter | Type | Example | Description |
enable | Boolean | true | Specifies whether to enable data encryption. Valid values:
|
encrypt_type | String | default | The encryption algorithm. Valid values: default and sm4. This parameter must be specified when enable is set to true. |
user_cmk_info | { "cmk_key_id" : "f5136b95-2420-ab31-xxxxxxxxx" "arn" : "acs:ram::13234:role/logsource" "region_id" : "cn-hangzhou" } | Optional. If you specify this parameter, the bring-your-own-key (BYOK) key is used. If you do not specify this parameter, the service key of Log Service is used. |
EncryptUserCmkConf
The following table describes the data structure of a user-defined encryption configuration.
Parameter | Type | Example | Description |
cmk_key_id | String | f5136b95-2420-ab31-xxxxxxxxx | The ID of the customer master key (CMK) that uses the BYOK. |
arn | String | acs:ram::13234:role/logsource | The Alibaba Cloud Resource Name (ARN) of the RAM role. |
region_id | String | cn-hangzhou | The ID of the region where the CMK resides. |
Histogram
The following table describes the data structure of log distribution.
Parameter | Type | Example | Description |
from | Integer | 1409529600 | The start time of the subinterval. The value is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970. |
to | Integer | 1409569200 | The end time of the subinterval. The value is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970. |
count | Long | 2 | The number of logs in the subinterval. |
progress | String | Complete | Indicates whether the query result in the subinterval is complete. Valid values:
|
LogtailConfig
The following table describes the data structure of a Logtail configuration.
Parameter | Type | Example | Description |
configName | String | test-config | The name of the Logtail configuration. The name must meet the following requirements:
|
logSample | String | 2022-06-14 11:13:29.796 | DEBUG | __main__:<module>:1 - hello world | A sample of the logs that Logtail needs to collect. |
inputType | String | file | The type of the data source from which Logtail collects logs. |
inputDetail | Map | The configurations of the data source. | |
outputType | String | LogService | The type of the destination to which the collected logs are stored. Set the value to LogService. |
outputDetail | Object | The configurations of the destination. | |
endpoint | String | cn-hangzhou-intranet.log.aliyuncs.com | The endpoint of the project. |
logstoreName | String | test-logstore | The name of the Logstore to which the collected logs are stored. |
region | String | cn-hangzhou | The region. |
createTime | Long | 1655176807 | The time at which the Logtail configuration was created. The value is a UNIX timestamp. |
lastModifyTime | Long | 1655176807 | The time at which the Logtail configuration was last modified. The value is a UNIX timestamp. |
SavedSearch
The following table describes the data structure of a saved search.
Parameter | Type | Example | Description |
savedsearchName | String | savedsearch-name | The name of the saved search. |
searchQuery | String | status: 401 | SELECT remote_addr,COUNT(*) as pv GROUP by remote_addr ORDER by pv desc limit 5 | The search statement or the query statement. |
logstore | String | aliyun-test-logstore | The name of the Logstore. |
topic | String | topic | The topic of the logs. Default value: empty string (""). |
displayName | String | Method statistics | The display name of the saved search. |
chart
The following table describes the data structure of a chart.
Parameter | Type | Example | Description |
title | String | my-chart | The title of the chart. |
type | String | linepro | The type of the chart. |
search | Map | (*)| SELECT date_format(__time__ - __time__ % 60, '%H:%i:%s') as time, count(1) as count GROUP BY time ORDER BY time | The query configuration. |
display | Map | The display configuration of the chart. | |
action | Map | action |
dashboard
The following table describes the data structure of a dashboard.
Parameter | Type | Example | Description |
dashboardName | String | dashboard-1609294922657-434834 | The internal display name of the dashboard, which is not displayed on the dashboard page. |
displayName | String | test-alert | The display name of the dashboard. |
description | String | This is a dashboard. | The description of the dashboard. |
attribute | Map | The attribute values of the dashboard. | |
charts | Array of chart | The charts on the dashboard. |
etlJob
The following table describes the data structure of a data transformation job.
Parameter | Type | Example | Description |
etlJobName | String | etl-name | The name of the data transformation job. |
sourceConfig | Object | The configurations of the data source. | |
logstoreName | String | my-logstore | The name of the Logstore. |
triggerConfig | Object | The trigger configuration of the data transformation job. | |
roleArn | String | acs:ram::13791******44:role/aliyunlogetlrole | The ARN of the role that is used for authorization. Example: |
triggerInterval | Integer | 5 | The interval at which the data transformation job is triggered. |
maxRetryTime | Integer | 10 | The maximum number of retries. |
startingPosition | String | at-unixtime | The position at which the data transformation job starts to run. |
startingUnixtime | Long | 1655176807 | The time at which the data transformation job starts to run. This parameter takes effect when startingPosition is set to at-unixtime. |
functionConfig | Object | The configuration of the functions that are used. | |
functionProvider | String | FunctionCompute | The function provider. |
endpoint | String | cn-hangzhou.log.aliyuncs.com | The Log Service endpoint. |
accountId | String | 13791******44 | The ID of the Alibaba Cloud account. |
regionName | String | cn-hangzhou | The region. |
serviceName | String | my-service | The name of the service. |
functionName | String | hello-world | The name of the function. |
roleArn | String | acs:ram::13791******44:role/aliyunlogetlrole | The ARN of the role that is used for authorization. Example: |
functionParameter | Map | The parameters. | |
enable | Boolean | true | Specifies whether to enable the data transformation job. |
logConfig | Object | The logging configuration. | |
endpoint | String | cn-hangzhou.log.aliyuncs.com | The Log Service endpoint. |
projectName | String | ali-test-project | The name of the project. |
logstoreName | String | ali-test-logstore | The name of the Logstore. |
etlMeta
The following table describes the data structure of etlMeta.
Parameter | Type | Example | Description |
etlMetaName | String | MetaName | MetaName |
etlMetaKey | String | MetaKey | MetaKey |
etlMetaTag | String | MetaTag | MetaTag |
enable | Boolean | true | Enable |
etlMetaValue | String | MetaValue | MetaValue |
externalStore
The following table describes the data structure of an external store.
Parameter | Type | Example | Description |
externalStoreName | String | rds_store | The name of the external store. The name must be unique in a project and must be different from Logstore names. |
storeType | String | rds-vpc | The storage type. Set the value to rds-vpc, which indicates an ApsaraDB RDS for MySQL database in a virtual private cloud (VPC). |
parameter | Map | { "vpc-id": "vpc-bp1aevy8sofi8mh1q****", "instance-id": "i-bp1b6c719dfa08exf****", "host": "192.168.XX.XX", "port": "3306", "username": "root", "password": "sfdsfldsfksfls****", "db": "meta", "table": "join_meta", "region": "cn-qingdao" } | The parameters that are configured for the external store. |
index
The following table describes the data structure of an index configuration.
Parameter | Type | Example | Description |
ttl | Integer | 30 | The log retention period. Unit: days. |
max_text_len | Integer | 2048 | The maximum length of a field value that can be retained. Unit: bytes. Default value: 2048. The default value is equal to 2 KB. You can change the value of the max_text_len parameter. Valid values: 64 to 16384. Unit: bytes. |
log_reduce_white_list | Array of String | name | The fields in the whitelist. |
log_reduce_black_list | Array of String | time | The fields in the blacklist. |
line | Object | The configuration of full-text indexing. You must configure at least one of the following parameters: line and keys. For more information, see Example. | |
chn | Boolean | false | Specifies whether to include Chinese characters. Valid values:
|
caseSensitive | Boolean | true | Specifies whether to enable case sensitivity. Valid values:
|
token | Array of String | , | The delimiter. Examples: ",", ".", "\r", and "\n". |
include_keys | Array of String | includeField | The included fields. |
exclude_keys | Array of String | excludeField | The excluded fields. |
keys | Map | The configuration of field indexes. A field index is a key-value pair in which the key specifies the name of the field and the value specifies the index configuration of the field. You must configure at least one of the following parameters: line and keys. For more information, see Example. | |
log_reduce | Boolean | true | Specifies whether to enable the LogReduce feature. After you enable the LogReduce feature, either the whitelist or blacklist takes effect. Valid values:
|
lastModifyTime | Long | 1622186280 | The time at which the index configuration was last modified. The value is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970. |
logging
The following table describes the data structure of the service log feature for a project.
Parameter | Type | Example | Description |
loggingProject | String | my-project | The name of the project to which service logs are stored. |
loggingDetails | Array | The configuration of service logs. | |
type | String | consumergroup_log | The type of service logs. Valid values:
|
logstore | String | my-logstore | The name of the Logstore. |
logstore
The following table describes the data structure of a Logstore.
Parameter | Type | Example | Description |
logstoreName | String | my-logstore | The name of the Logstore. |
ttl | Integer | 1 | The data retention period. Unit: days. Valid values: 1 to 3650. If you set the value to 3650, data is permanently stored. |
shardCount | Integer | 2 | The number of shards. |
enable_tracking | Boolean | false | Specifies whether to enable the web tracking feature. Default value: false. Valid values:
|
autoSplit | Boolean | true | Specifies whether to enable automatic sharding. Valid values:
|
maxSplitShard | Integer | 6 | The maximum number of readwrite shards that are obtained after shards are automatically split . Valid values: 1 to 64. |
appendMeta | Boolean | false | Specifies whether to record public IP addresses. Default value: false. Valid values:
|
telemetryType | String | Metrics | The type of the log data that you want to query. Valid values:
|
createTime | Integer | 1453949705 | The time at which the Logstore was created. The value is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970. |
lastModifyTime | Integer | 1524155379 | The time at which the Logstore was last modified. The value is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970. |
encrypt_conf | The configuration of data encryption. | ||
hot_ttl | Integer | 60 | The period of time during which you want to store logs in the hot storage tier of the Logstore. Minimum value: 30. Unit: days. |
mode | String | standard | The type of the Logstore. Log Service provides the following types of Logstores: Standard Logstores and Query Logstores. Valid values:
|
productType | String | aliyun | The type of the service to which the logs belong. |
machine
The following table describes the data structure of a machine.
Parameter | Type | Example | Description |
ip | String | 192.168.xx.x | The IP address of the machine. |
machine-uniqueid | String | 3B70F4F1-80F7-46C4-A6C1-100***CEE647 | The unique identifier of the machine. |
lastHeartbeatTime | Long | 1447182247 | The last heartbeat time. The value is a timestamp that follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 UTC, Thursday, January 1, 1970. |
userdefined-id | String | None | The custom identifier of the machine. This parameter takes effect when the machine belongs to a custom identifier-based machine group. |
machineGroup
The following table describes the data structure of a machine group.
Parameter | Type | Example | Description |
groupName | String | machineGroup-1 | The name of the machine group. |
groupType | String | "" | The type of the machine group. Set the value to an empty string. |
machineIdentifyType | String | ip | The type of the machine group identifier. Valid values:
|
groupAttribute | Object | The attribute of the machine group. | |
groupTopic | String | testtopic | The topic of the machine group. |
externalName | String | testgroup | The identifier of the external management system on which the machine group depends. |
machineList | Array of String | 47.100.XX.XX | The IP address or custom identifier of the machine group. |
project
The following table describes the data structure of a Log Service project.
Parameter | Type | Example | Description |
createTime | String | 2021-07-07 14:08:09 | The time at which the project was created. |
lastModifyTime | String | 2022-04-18 13:30:19 | The time at which the project was last modified. |
description | String | Description of my-project | The description of the project. |
owner | String | "" | The Alibaba Cloud account to which the project belongs. |
projectName | String | ali-test-project | The name of the project. The name is included in the value of the Host header. The name must be unique in a region. You cannot change the name after you create the project. |
region | String | cn-hangzhou | The ID of the region where the project resides. |
status | String | Normal | The status of the project. Valid values:
|
resourceGroupId | String | rg-acf******sq | The ID of the resource group to which the project belongs. |