Log Service allows you to collect data from multiple sources, such as servers, applications, open source software, IoT devices, mobile devices, and Alibaba Cloud services. You can also collect data that is transferred over standard protocols. This topic describes the data sources that are supported by Log Service.
Data sources
The following table describes the sources from which Log Service can collect data.
| Category | Source | Collection method | References |
|---|---|---|---|
| Application | Program output | Logtail | None |
| Access log | Logtail | Collect and analyze NGINX access logs | |
| Trace data | None | ||
| Metric data | None | ||
| Programming language | Java | None | |
| Log4J Appender | None | ||
| LogBack Appender | LogBack | None | |
| C | Log Service SDK for C | None | |
| Python | Log Service SDK for Python | None | |
| Python Logging | None | None | |
| PHP | Log Service SDK for PHP | None | |
| .Net | Log Service SDK for csharp | None | |
| C++ | Log Service SDK for C++ | None | |
| Go | None | ||
| Node.js | SDK for Node.js | None | |
| JavaScript | JavaScript/Web tracking | None | |
| Operating system | Linux | Logtail | None |
| Windows | Logtail | None | |
| macOS/UNIX | Native C | None | |
| Docker file | Logtail | None | |
| Docker output | Logtail | None | |
| Database | MySQL binary log | Logtail | None |
| MySQL query result | Logtail | None | |
| SQL Server query result | Logtail | None | |
| PostgreSQL query result | Logtail | None | |
| Mobile device | iOS and Android | None | |
| Web page | JavaScript/Web tracking | None | |
| Intelligent IoT | Log Producer Library for C | None | |
| Standard protocol | HTTP | Logtail | Collect and analyze NGINX monitoring logs |
| Syslog | Logtail | None | |
| Kafka | Log Producer SDK for Kafka | None | |
| OpenTelemtry | OpenTelemetry | None | |
| Imported data | MaxCompute data | Data import | None |
| Object Storage Service (OSS) data | Data import | None | |
| MySQL data | Data import | None | |
| Elasticsearch data | Data import | None | |
| CloudMonitor data | Data import | None | |
| GCP Pub/Sub message | Data import | None | |
| Flink | Use Flink to write data to Log Service | Register a Log Service project | |
| Third-party software | Logstash | Logstash and Kafka protocol | None |
| Flume | Use Flume to consume log data | None | |
| Beats | Kafka protocol | None | |
| Fluentd | Kafka protocol | None | |
| Telegraf | Kafka protocol | None | |
| Alibaba Cloud service | Services such as Elastic Compute Service (ECS) and OSS | Alibaba Cloud service logs | None |
Endpoints
Log Service provides the following types of endpoints:
- Internal endpoint: If you select an internal endpoint, Log Service can be accessed by other Alibaba Cloud services in the same region over the classic network or a virtual private cloud (VPC), which is reliable.
- Public endpoint: If you select a public endpoint, Log Service can be accessed over the Internet, and the data transmission speed varies based on network conditions. To ensure secure data transmission, we recommend that you use HTTPS.
For more information, see Endpoints.
FAQ
- Which network type do I need to select if I use Express Connect together with Log Service?
We recommend that you select the internal network type.
- Can I obtain public IP addresses when I collect data from the Internet?
Yes, you can turn on Log Public IP for your Logstore to obtain public IP addresses. For more information, see Create a Logstore.
- Which network type do I need to select if I want to collect ECS logs from Region A to a Log Service project in Region B?
We recommend that you select the Internet type and specify the region name of Region B in the command that is used to install Logtail on your ECS instance in Region A. This way, Logtail can access the project in Region B over the Internet. For more information about how to select a network type in other scenarios, see Select a network type.
- How do I check whether an endpoint is accessible?
You can run the following command to check whether an endpoint is accessible. If output is returned, the endpoint is accessible.
curl $myproject.cn-hangzhou.log.aliyuncs.com$myproject: the name of the project.cn-hangzhou.log.aliyuncs.com: the endpoint.