This topic describes how to collect logs of an elastic container instance to Kafka.
Prerequisites
A connection is established between an elastic container instance and Kafka
If you deploy Kafka on Alibaba Cloud, we recommend that you deploy Kafka and the elastic container instance in the same VPC.
Log collection settings are configured. For more information, see Configure log collection for an elastic container instance.
Configure log collection by using the Log Service console
Logtail is a log collection agent that is provided by Alibaba Cloud Log Service. Logtail can be used to collect text file logs and standard output logs. You can configure the flushers plug-in in the Log Service console to import logs to Kafka. The following code provides the configuration file of flushers:
"flushers": [
{
"detail": {
"Brokers": [
"Kafka broker address"
],
"Topic": "log"
},
"type": "flusher_kafka"
}
]If you use Logtail to collect standard output logs, you can directly modify the configurations of Logtail and add the flushers plug-in to the Plug-in Config field. The following figure shows how to add the flushers plug-in.
If you use Logtail to collect text file logs, you need to unfold Advanced Options, turn on Enable Plug-in Processing, and then add the flushers plug-in to the Plug-in Config field. The following figure shows how to add the flushers plug-in.
Verify the result
After you complete the configurations, you can see that logs of the elastic container instance have been collected to Kafka. For example, if you are using Alibaba Cloud Message Queue for Apache Kafka, you can query the logs of the elastic container instance on the Message Query page. The following figure shows an example of the expected outcome.
