Log Service allows you to collect data from multiple sources, such as servers, applications, open source software, IoT devices, mobile devices, and Alibaba Cloud services. You can also collect data that is transferred over standard protocols. This topic describes the data sources that are supported by Log Service.
Data sources
The following table describes the sources from which Log Service can collect data.
| Category | Source | Collection method | References |
|---|---|---|---|
| Application | Program output | Logtail | None |
| Access log | Logtail | Collect and analyze NGINX access logs | |
| Programming language | Java | None | |
| Log4J Appender | None | ||
| LogBack Appender | LogBack | None | |
| C | Log Service C SDK | None | |
| Python | Log Service Python SDK | None | |
| Python Logging | None | None | |
| PHP | Log Service PHP SDK | None | |
| .NET | Log Service csharp SDK | None | |
| C++ | Log Service C++ SDK | None | |
| Go | None | ||
| Node.js | NodeJs | None | |
| JS | JS/Web Tracking | None | |
| OS | Linux | Logtail | None |
| Windows | Logtail | None | |
| MAC/Unix | Native C | None | |
| Docker file | Use Logtail to collect Docker files | None | |
| Docker output | Use Logtail to collect container logs | None | |
| Database | MySQL binary log | Collect MySQL binary logs | None |
| JDBC SELECT | Collect MySQL query results | None | |
| Mobile device | iOS and Android | None | |
| Web page | JS/Web Tracking | None | |
| Intelligent IoT | C Producer Library | None | |
| Standard protocol | HTTP polling data | Logtail HTTP | Collect and analyze NGINX monitoring logs |
| Syslog | Use Logtail to collect syslog logs | None | |
| Imported data | MaxCompute | Import data from MaxCompute to Log Service | None |
| Object Storage Service (OSS) | Import data from OSS to Log Service | None | |
| Flink | Use Flink to write data to Log Service | Register a Log Service project | |
| Third-party software | Logstash | Logstash and Use the Kafka protocol to upload logs | None |
| Flume | Use Flume to consume log data | None | |
| Beats | Use the Kafka protocol to upload logs | None | |
| Fluentd | Use the Kafka protocol to upload logs | None | |
| Telegraf | Use the Kafka protocol to upload logs | None | |
| Alibaba Cloud service | Services such as Elastic Compute Service (ECS) and OSS | Alibaba Cloud service logs | None |
Endpoints
Log Service provides the following types of endpoints:
- Internal endpoint: If you select an internal endpoint, Log Service can be accessed by other Alibaba Cloud services in the same region over the classic network or a virtual private cloud (VPC), which is reliable.
- Public endpoint: If you select a public endpoint, Log Service can be accessed over the Internet, and the data transmission speed varies based on network conditions. We recommend that you use HTTPS to ensure secure data transmission.
For more information, see Endpoints.
FAQ
- Which network type do I need to select if I use Express Connect together with Log
Service?
We recommend that you select the internal network type.
- Can I obtain public IP addresses when I collect data from the Internet?
Yes, you can turn on Log Public IP for your Logstore to obtain public IP addresses. For more information, see Create a Logstore.
- Which network type do I need to select if I want to collect ECS logs from Region A
to a Log Service project in Region B?
We recommend that you select the Internet type and specify the region name of Region B in the command used to install Logtail on your ECS instance in Region A. This way, the Logtail can access the project in Region B over the Internet. For more information about how to select a network type in other scenarios, see Select a network type.
- How do I determine whether an endpoint is accessible?
You can run the following command. If output is returned, the endpoint is accessible.
curl $myproject.cn-hangzhou.log.aliyuncs.com$myproject: the project namecn-hangzhou.log.aliyuncs.com: the endpoint