LogHub supports multiple methods to collect logs, such as by using clients, Web pages, protocols, SDKs, and APIs. All collection methods are implemented based on Restful APIs. You can also implement new collection methods by using APIs and SDKs.

Data sources

The following table describes the data sources from which Log Service can collect logs.
Category Source Collection method Reference
Application Program output Logtail
Access log Logtail Collect and analyze NGINX access logs
Link tracking Jaeger Collector and Logtail -
Language Java Log Service Java SDK and Java Producer Library -
Log4j appender 1.x and 2.x -
Logback appender Logback -
C Log Service C SDK -
Python Log Service Python SDK -
Python logging Python logging handler -
PHP Log Service PHP SDK -
C# Log Service C SDK -
C++ Log Service C++ SDK -
Go Log Service Go SDK and Go Producer Library -
Node.js Node.js -
JavaScript JavaScript/Web tracking -
Operating system (OS) Linux Logtail -
Windows Logtail -
Mac OS or Unix Native C -
Docker files Use Logtail to collect Docker files -
Docker output Use Logtail to collect container logs -
Mobile client iOS/Android Log Service Android SDKLog Service iOS SDK -
Web page JavaScript/Web tracking -
Intelligent IoT C Producer Library
Alibaba Cloud services ECS, OSS, and other Alibaba Cloud services. For more information, see Cloud service logs. Activate Log Service in the Alibaba Cloud console Cloud service logs
MaxCompute import Use DataWorks to export MaxCompute data -
Third-party software Logstash Logstash -
Flume Use Flume to consume log data -
The following table lists the Alibaba Cloud services from which Log Service can collect logs.
Type Cloud service Activation method Documentation
Elastic computing ECS Install Logtail. Overview
Container Service or Alibaba Cloud Container Service for Kubernetes Activate the service in the Container Service or Container Service for Kubernetes console. Container text logs and Container standard output
Storage OSS Activate the service in the OSS console. Overview
Networking SLB Activate the service in the SLB console. SLB Layer-7 Access logs
Virtual Private Cloud (VPC) Activate the service in the VPC console. Create a flow log
API Gateway Activate the service in the API Gateway console. API Gateway access logs
Security ActionTrail Activate the service in the ActionTrail console. Overview
Anti-DDoS Pro Activate the service in the Anti-DDoS Pro console. Anti-DDoS Pro overview and Anti-DDoS Pro overview
Security Center Purchase Security Center Enterprise Edition and activate the service in the Security Center console. TDS logs
Anti-Bot Service Activate the service in the Anti-Bot Service console. Anti-Bot Service logs
Application Log Service Activate the service in the Log Service console. Service log overview

Select a network

Log Service provides service endpoints for different Alibaba Cloud regions. For more information, see Service endpoint. Each region allows access from the following networks:

  • Internal network (classic network) or private network (VPC): Log Service can access other Alibaba Cloud services in the same region, offering optimal link bandwidth. We recommend that you select this option.
  • Public network (classic network): accessible without any limits. The transmission speed depends on the link quality. We recommend that you use HTTPS to ensure secure transmission of data.

FAQ

  • Q: Which network do I select for private line access?

    A: Select the internal network or private network.

  • Q: Can I collect public IP addresses when collecting public network data?

    A: You need to enable Log Service to record public IP addresses. For more information, see Manage a Logstore.

  • Q: Which network do I select if I want to collect ECS logs from Region A and write these logs into the Log Service project in Region B?

    A: Select the public network. You can install Logtail on the ECS instance in Region A for Internet transmission and specify the service endpoint that is associated with Region B. For more information about how to select a network, see Select a network type.

  • Q: How can I determine whether a service endpoint is accessible?

    A: You can run the following command. The service endpoint is accessible if any information is returned.

     curl $myproject.cn-hangzhou.log.aliyuncs.com

    $myproject specifies the project name and cn-hangzhou.log.aliuncs.com specifies the service endpoint.