LogHub supports multiple methods to collect logs, such as by using clients, Web pages, protocols, SDKs, and APIs. All collection methods are implemented based on Restful APIs. You can also implement new collection methods by using APIs and SDKs.

Data sources

The following table describes the data sources from which Log Service can collect logs.
Category Source Collection method Reference
Application Program output Logtail
Access log Logtail Collect and analyze Nginx access logs
Link tracking Jaeger Collector and Logtail -
Language Java SDK and Java Producer Library -
Log4j appender 1.x and 2.x -
Logback appender Logback -
C Native -
Python Python -
Python logging Python logging handler -
PHP PHP -
C# C# -
C++ C++ SDK -
Go Go and Go Producer Library -
Node.js Node.js -
JavaScript JavaScript/Web tracking -
Operating system (OS) Linux Logtail -
Windows Logtail -
Mac OS or Unix Native C -
Docker files Use Logtail to collect Docker files -
Docker output Use Logtail to collect container logs -
Mobile client iOS/Android iOS SDK, Android SDK -
Web page JavaScript/Web tracking -
Intelligent IoT C Producer Library
Alibaba Cloud services ECS, OSS, and other Alibaba Cloud services. For more information, see Cloud service logs. Activate Log Service in the Alibaba Cloud console Cloud service logs
MaxCompute import Use DataWorks to export MaxCompute data Use DataWorks to export MaxCompute data to Log Service
Third-party software Logstash Logstash -
Flume Use Flume to consume LogHub logs -
The following table lists the Alibaba Cloud services from which Log Service can collect logs.
Type Cloud service Activation method Remarks
Elastic computing ECS Install Logtail. Logtail introduction
Container Service/Container Service for Kubernetes Activate the service in the Container Service or Container Service for Kubernetes console. Text logs and output
Storage OSS Activate the service in the OSS console. Overview
Network SLB Activate the service in the SLB console. Access logs of Layer-7 SLB
VPC Activate the service in the VPC console. Create a flow log
API Gateway Activate the service in the API Gateway console. Access logs of API Gateway
Security ActionTrail Activate the service in the ActionTrail console. Overview
Anti-DDoS Pro/BGP-line Anti-DDoS Pro Activate the service in the Anti-DDoS Pro console. Anti-DDoS Pro overview and BGP-line Anti-DDoS Pro overview
Threat Detection Service Purchase Threat Detection Service Enterprise Edition and activate the service in the Threat Detection Service console. TDS logs
Anti-Bot Service Activate the service in the Anti-Bot Service console. Anti-Bot Service logs
Application Log Service Activate the service in the Log Service console. Service log overview

Select a network

Log Service provides service endpoints for different Alibaba Cloud regions. For more information, see Service endpoint. Each region allows access from the following networks:

  • Internal network (classic network) or private network (VPC): Log Service can access other Alibaba Cloud services in the same region, offering optimal link bandwidth. We recommend that you select this option.
  • Public network (classic network): accessible without any limits. The transmission speed depends on the link quality. We recommend that you use HTTPS to ensure secure transmission of data.

FAQ

  • Q: Which network do I select for private line access?

    A: Select the internal network or private network.

  • Q: Can I collect public IP addresses when collecting public network data?

    A: You need to enable Log Service to record public IP addresses. For more information, see Manage a Logstore.

  • Q: Which network do I select if I want to collect ECS logs from Region A and write these logs into the Log Service project in Region B?

    A: Select the public network. You can install Logtail on the ECS instance in Region A for Internet transmission and specify the service endpoint that is associated with Region B. For more information about how to select a network, see Select a network type.

  • Q: How can I determine whether a service endpoint is accessible?

    A: You can run the following command. The service endpoint is accessible if any information is returned.

     curl $myproject.cn-hangzhou.log.aliyuncs.com

    $myproject specifies the project name and cn-hangzhou.log.aliuncs.com specifies the service endpoint.