AliyunLogConfig is a CustomResourceDefinition (CRD) that Simple Log Service defines. Create a custom resource (CR) from this CRD in a YAML file to create, view, update, and delete Logtail configurations declaratively in a Kubernetes cluster.
Prerequisites
-
Logtail components v0.5.1 or later are installed. For more information, see Install Logtail components in an ACK cluster and Install Logtail components in a self-managed Kubernetes cluster.
-
The target container is actively generating logs.
ImportantLogtail collects only incremental logs. If a log file is not updated after configuration delivery, Logtail skips it. Read logs.
Limits
The following limits apply when you manage a Logtail configuration by using AliyunLogConfig:
Console changes are not synchronized — If a Logtail configuration is created by an AliyunLogConfig CR, the changes that you make to the configuration in the Simple Log Service console are not synchronized to the CR. To modify such a configuration, modify only the CR. This prevents configuration inconsistencies.
Existing LogStores — If the LogStore that you specify in the CR already exists, all subsequent LogStore-related parameters do not take effect.
Immutable configuration name — After a Logtail configuration is created, you cannot change its name, which is the value of the
configNameparameter.alibaba-log-controller version — Several parameters require a minimum version of alibaba-log-controller. Before you write the CR, check the minimum version that is noted for each parameter in CR parameters.
Procedure
Create a Logtail configuration
Create a CR from AliyunLogConfig. After you create the CR, alibaba-log-controller creates a Logtail configuration based on the CR and applies the configuration automatically.
Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Run the following command to create a YAML file. In this example, the file name is
cube.yaml. Replace the file name as needed.vim cube.yamlEnter the following script in the YAML file and configure the parameters as needed. The script contains the required fields only. Set
inputTypeto specify how logs are collected, and setinputDetailto configure the settings of the log input. For more information about the fields of the CR, seeCR parameters.apiVersion: log.alibabacloud.com/v1alpha1 # Use the default value. You do not need to modify this parameter. kind: AliyunLogConfig # Use the default value. You do not need to modify this parameter. metadata: name: test-config # The name of the resource. The name must be unique in the current Kubernetes cluster. spec: logstore: k8s-stdout # The name of the Logstore. If the Logstore that you specify does not exist, Simple Log Service automatically creates the Logstore. logtailConfig: # The Logtail configuration. inputType: plugin # The type of the data source from which logs are collected. configName: test-config # The name of the Logtail configuration. inputDetail: # The details of the Logtail configuration. ...Run the following command to create the CR from
AliyunLogConfig. After the Logtail configuration takes effect, Logtail starts to collect data and sends the data to Simple Log Service.kubectl apply -f cube.yamlVerify that the value of the
statusCodefield of the CR is 200. For instructions, see View Logtail configurations.
View Logtail configurations
To view all Logtail configurations that are created by AliyunLogConfig in the current Kubernetes cluster, run the following command:
kubectl get aliyunlogconfigsTo view the details and status of a single Logtail configuration that is created by AliyunLogConfig, run the following command. <config_name> is the name of the AliyunLogConfig. Replace the name as needed.
kubectl get aliyunlogconfigs <config_name> -o yamlA result similar to the following one is returned. In this example, the Logtail configuration uses inputType: file.
apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
name: example-k8s-file
namespace: kube-system
spec:
project: k8s-log-test
logstore: k8s-file
logtailConfig:
inputType: file
configName: example-k8s-file
inputDetail:
logType: common_reg_log
logPath: /data/logs/app_1
filePattern: test.LOG
dockerFile: true
status:
status: OK
statusCode: 200The status field and the statusCode field in the result indicate the status of the Logtail configuration:
If the value of the
statusCodefield is 200, the Logtail configuration is applied.If the value of the
statusCodefield is not 200, the Logtail configuration fails to be applied. Check thestatusfield for the reported status.
Update a Logtail configuration
To update a Logtail configuration that is created by AliyunLogConfig, modify the existing YAML file of the CR and apply the file again.
Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Edit the existing
AliyunLogConfigYAML file, such ascube.yaml.vim cube.yamlModify the parameters in the YAML file as needed. For more information about the fields of the CR, seeCR parameters.
Run the following command to apply the file and make the Logtail configuration take effect.
kubectl apply -f cube.yamlVerify that the value of the
statusCodefield of the CR is 200. For instructions, see View Logtail configurations.
Delete a Logtail configuration
To delete a Logtail configuration that is created by AliyunLogConfig, delete the corresponding CR. In the following command, <config_name> is the name of the AliyunLogConfig. Replace the name as needed.
After the CR is deleted, the corresponding Logtail configuration is also deleted automatically.
kubectl delete aliyunlogconfigs <config_name>To confirm the deletion, run kubectl get aliyunlogconfigs and check that the CR is no longer listed.
CR parameters
The following sections describe the fields of an AliyunLogConfig CR.
apiVersion
The value is fixed to log.alibabacloud.com/v1alpha1.
kind
The value is fixed to AliyunLogConfig.
metadata
The metadata field identifies the CR in the Kubernetes cluster.
| Parameters | Type | Required | Description |
name | string | Yes | The unique identifier of the CR. As a best practice, use the same value as spec.logtailConfig.configName. |
namespace | string | No | The namespace to which the CR belongs. |
spec
The spec field specifies the destination project and LogStore, the attributes of the destination LogStore, and the details of the Logtail configuration.
| Parameters | Type | Required | Description |
project | string | No | The name of the destination project. The default value is the project that is configured for the cluster, which is the value of the ALICLOUD_LOG_PROJECT environment variable of alibaba-log-controller. If the project that you specify does not exist, Simple Log Service automatically creates the project. |
logstore | string | Yes | The name of the destination LogStore. If the LogStore that you specify does not exist, Simple Log Service automatically creates the LogStore. |
logtailConfig | AliyunLogConfigDetail | Yes | The details of the Logtail configuration. |
uid | string | No | The UID of the destination account. To ship logs across accounts, configure this parameter and configure ALICLOUD_LOG_ACCOUNT_INFOS={"<uid>":{"accessKeyID":"<your_access_key_id>","accessKeySecret":"<your_access_key_secret>"}} in the environment variables of alibaba-log-controller. This parameter controls only the account to which the project belongs. To ship Logtail data to that account, you must also configure a user identifier for Logtail. For instructions, seeConfigure a user identifier. This parameter is supported only by alibaba-log-controller 0.3.2 or later. |
endpoint | string | No | The endpoint of the destination project. By default, the endpoint of the region where the current cluster resides is used. If the destination project and the cluster reside in different regions, configure this parameter. This parameter controls only the endpoint that the Logtail configuration uses. To ship Logtail data to that region, you must also configure the config_server_address and data_server_list parameters for Logtail. For instructions, seeLogtail configuration files and record files. This parameter is supported only by alibaba-log-controller 0.4.1 or later. For more information, see Endpoint. |
logstoreMode | string | No | The type of the destination LogStore. Default value: standard. Valid values: query (Query LogStore) and standard (Standard LogStore). This parameter is supported only by alibaba-log-controller 0.3.3 or later. For details, seeLogStore types. |
shardCount | int | No | The number of shards in the destination LogStore. Default value: 2. Valid values: 1 to 10. For details, seeShards. |
logstoreMaxSplitShard | int | No | The maximum number of shards into which the shards in the destination LogStore can be automatically split. Default value: 32. Valid values: 1 to 256. For details, seeautomatic sharding. |
logstoreAutoSplit | bool | No | Specifies whether to enable automatic sharding for the destination LogStore. Default value: true. For details, seeautomatic sharding. |
lifeCycle | int | No | The data retention period of the destination LogStore. Unit: days. Default value: 180. Valid values: 1 to 3650. The value 3650 specifies permanent retention. For details, seeData Retention Period. |
logstoreHotTTL | int | No | The hot data retention period of the destination LogStore. Unit: days. Default value: 0. The value must be less than the value of lifeCycle and greater than or equal to 7. For details, seeIntelligent Tiered Storage. |
logstoreTelemetryType | string | No | The type of observability data in the destination LogStore. Default value: None. Valid values: None (log data) and Metrics (metric data). For details, seetelemetryType. |
logstoreAppendMeta | bool | No | Specifies whether to enable the feature that records public IP addresses for the destination LogStore. Default value: true. For details, seeLog Public IP. |
logstoreEnableTracking | bool | No | Specifies whether to enable Web Tracking for the destination LogStore. Default value: false. For details, seeWebTracking. |
logstoreEncryptConf | object | No | The encryption settings of the destination LogStore. This parameter is empty by default. For details, seeCommon data structures. |
logstoreMeteringMode | string | No | The billing mode of the destination LogStore. This parameter is empty by default. Valid values: ChargeByFunction (pay-by-feature billing mode) and ChargeByDataIngest (pay-by-ingested-data billing mode). If logstoreMode of the LogStore is query, only the pay-by-feature billing mode is supported. If the pay-by-ingested-data billing mode is not enabled for your account, you can set this parameter only to ChargeByFunction. This parameter is supported only by alibaba-log-controller 0.4.3 or later. For details, seeManage LogStores. |
machineGroups | array | No | The machine groups that are associated with the Logtail configuration. When you install the logtail-ds component, Simple Log Service automatically creates a machine group named k8s-group-${your_k8s_cluster_id}. This machine group is the default value. For details, seeIntroduction. |
configTags | map | No | The custom tags that are used to identify the Logtail configuration. This parameter is supported only by alibaba-log-controller 0.4.2 or later. |
AliyunLogConfigDetail
For more information about the parameters, seeLogtail configurations (legacy). The data format of AliyunLogConfigDetail is the same as the format that is described inLogtail configurations (legacy).
| Parameters | Type | Required | Description |
configName | string | Yes | The name of the Logtail configuration. The name must be unique in the project to which the Logtail configuration belongs. As a best practice, use the same value as the resource name (metadata.name). The name can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name must start and end with a lowercase letter or a digit. The name must be 2 to 128 characters in length. |
logSample | string | No | The sample log. The value must be less than 1,500 bytes in length. |
inputType | string | Yes | The method that is used to collect logs. Valid values: plugin (collects logs such as standard output logs and MySQL binary logs by using Logtail plug-ins) and file (collects logs from text files in fixed modes, such as the full regex mode and the delimiter mode). |
inputDetail | Object | Yes | The settings of the log input. For details, seeinputDetail. |