If you want to use a separate Logtail process to collect logs from all containers in a pod, you can install Logtail in a Kubernetes cluster in Sidecar mode. This topic describes the implementation, limits, prerequisites, and procedure of collecting container text logs in Sidecar mode.
Implementation
Sidecar mode
In Sidecar mode, each pod runs a Logtail container. You can use Logtail to collect logs from all containers in the pod. Log collection from each pod is isolated.
To ensure that Logtail can collect logs from other containers in a pod, make sure that the Logtail container and application containers share the same volume. For more information about how to collect container logs in Sidecar mode, see Sidecar container with a logging agent and Pods with multiple containers. For more information about volumes, see Storage basics.
Prerequisites
Ports 80 (HTTP) and 443 (HTTPS) for outbound traffic are enabled for the server on which Logtail is installed. If the server is an Elastic Computing Service (ECS) instance, you can reconfigure the related security group rules to enable the ports. For more information about how to configure a security group rule, see Add a security group rule.
The kubectl command-line tool is installed in your Kubernetes cluster. For more information, see kubectl.
Usage notes
Logs are continuously generated in the container from which you want to collect logs. Logtail collects only incremental logs. If a log file on your server is not updated after a Logtail configuration is delivered and applied to the server, Logtail does not collect logs from the file. For more information, see Read log files.
The files from which you want to collect logs are stored in the volume that is mounted to the required Logtail container.
Step 1: Inject a Logtail container into a business pod
Log on to your Kubernetes cluster.
Create a YAML file. In the following command,
sidecar.yaml
is a sample file name. You can specify a different file name based on your business requirements.vim sidecar.yaml
Enter the following script in the YAML file and configure the parameters based on your business requirements.
WarningIn the following YAML template, replace all placeholders in the
${}
format with actual values. Do not modify or delete other parameters.Run the following command to apply the configurations in the
sidecar.yaml
file.In the following command,
sidecar.yaml
is a sample file name. You can specify a different file name based on your business requirements.kubectl apply -f sidecar.yaml
Step 2: Create a custom identifier-based machine group
The value of the ALIYUN_LOGTAIL_USER_DEFINED_ID
parameter in the YAML file created in Step 1 is the custom identifier.
Log on to the Simple Log Service console. In the Projects section, click the project that you want to manage.
In the left-side navigation pane, choose
. In the Machine Groups list, choose .In the Create Machine Group panel, configure parameters and click OK. The following table describes the parameters.
Parameter
Description
Name
The name of the machine group. The name must meet the following requirements:
The name can contain only lowercase letters, digits, hyphens (-), and underscores (_).
The name must start and end with a lowercase letter or a digit.
The name must be 2 to 128 characters in length.
ImportantAfter you create a machine group, you cannot change the name of the machine group. Proceed with caution.
Machine Group Identifier
The identifier type of the machine group. Select Custom Identifier.
Machine Group Topic
Optional. The topic of the machine group. The topic is used to identify the logs that are generated by different servers. For more information, see Log topics.
Custom Identifier
The custom identifier. Enter the custom identifier that is configured in Step 1.
Step 3: Create a Logtail configuration
In the required project, find the Logstore that you want to manage and choose
.In the Quick Data Import dialog box, click the Kubernetes - File card.
Select a project and a Logstore. Then, click Next. In this example, select the project that you use to install the Logtail components and the Logstore that you create.
In the Machine Group Configurations step of the Import Data wizard, perform the following operations. For more information about machine groups, see Introduction to machine groups.
Use one of the following settings based on your business requirements:
Confirm that the required machine group is added to the Applied Server Groups section. Then, click Next. Select the machine group that you create in Step 2.
If you want to create a machine group, click Create Machine Group. In the panel that appears, configure the parameters to create a machine group. For more information, see Collect text logs from Kubernetes containers in DaemonSet mode.
If the heartbeat status of a machine group is FAIL, click Automatic Retry. If the issue persists, you can troubleshoot the error by following the instructions that are provided in How do I troubleshoot an error that is related to a Logtail machine group in a host environment?
Create a Logtail configuration and click Next. Then, Simple Log Service starts to collect logs.
NoteApproximately 3 minutes are required to create a Logtail configuration.
Create indexes and preview data. Then, click Next. By default, full-text indexing is enabled for Simple Log Service. You can also configure field indexes based on collected logs in manual mode or automatic mode. To configure field indexes in automatic mode, click Automatic Index Generation. This way, Simple Log Service automatically creates field indexes. For more information, see Create indexes.
If you want to query all fields in logs, we recommend that you use full-text indexes. If you want to query only specific fields, we recommend that you use field indexes. This helps reduce index traffic. If you want to analyze fields, you must create field indexes. You must include a SELECT statement in your query statement for analysis.
Click Query Log. Then, you are navigated to the query and analysis page of the created Logstore.
Step 4: Query and analyze logs
In the End step of the Import Data wizard, click Query Log. Then, you are navigated to the query and analysis page of the created Logstore. The following error message may appear because indexes are not created. After you close the error message page and wait for 1 minute, you can view the collected logs.
Enter a query statement in the search box, specify a query time range, and then click Search & Analyze. Then, you can obtain the logs that meet the specified conditions. For more information about the query and analysis syntax, see Query and analyze logs in index mode.
References
View Logtail configurations
Log on to the Simple Log Service console.
In the Projects section, click the target project.
Choose . Click the > icon of the target Logtail configurations. Choose .
Click the target Logtail configurations to view the details of the configurations.
Troubleshooting
You can check whether error messages appear in the Simple Log Service console. For more information, see How do I view Logtail collection errors?
If an error occurs when you use Logtail to collect logs but no error messages appear in the Simple Log Service console, you can check the heartbeat status of your machine group and your Logtail configuration. For more information, see What do I do if errors occur when I collect logs from containers?