Java applications currently have two mainstream log collection solutions:
By using Appender, you can use Config to complete real-time log collection easily without changing any code.
You can integrate Appender by performing the configuration steps of aliyun-log-log4j-appender.
The contents of the configuration file log4j.properties are as follows:
log4j.rootLogger=WARN,loghub log4j.appender.loghub=com.aliyun.openservices.log.log4j.LoghubAppender # Log Service project name (required parameter) log4j.appender.loghub.projectName=[your project] # Log Service LogStore name (required parameter) log4j.appender.loghub.logstore=[your logstore] #Log Service HTTP address (required parameter) log4j.appender.loghub.endpoint=[your project endpoint] # User identity (required parameter) log4j.appender.loghub.accessKeyId=[your accesskey id] log4j.appender.loghub.accessKey=[your accesskey]
After configuring the Appender as described in the previous step, the logs produced by Java applications are automatically sent to Log Service. You can use LogSearch/Analytics to query and analyze these logs in real time. For details, please go to 3 Ways to Migrate Java Logs to the Cloud: Log4J, LogBack, and Producer Lib.
Logs are ubiquitous. As a carrier that records changes in the world, logs are widely used in many fields, such as marketing, research and development, operation, security, BI, and auditing.
Alibaba Log Service is an all-in-one service platform for log data. Its core component LogHub has become an infrastructure of big data processing, especially real-time data processing, by virtue of outstanding features such as high throughput, low latency, and auto-scaling. Jobs running on big data computing engines such as Flink, Spark, and Storm write data processing results or intermediate results to LogHub in real time. With data from LogHub, downstream systems are able to provide many services such as query analysis, monitoring alarms, machine learning, and iterative calculation. The LogHub big data processing architecture is provided in the following figure.
This article will show you how to use Alibaba Cloud's LOG Java Producer, an easy-to-use, highly configurable Java library that helps you send data to Log Service.
The Alibaba Cloud LOG Java Producer is high performance write LogHub library that is designed for Java applications running in big data and high concurrency scenarios. In comparison with using APIs or SDKs, using Alibaba Cloud LOG Java Producer (Producer) has many advantages, such as high performance, isolated computing and I/O logic, and controllable resource usage. To understand the features and mechanisms of the Producer, see the article Alibaba Cloud LOG Java Producer - A powerful tool to migrate logs to the cloud. This article will focus on the use of the Producer.
The complete regular mode requires format consistency among all logs. However, some logs may contain content in multiple formats. In this case, you can use the Schema-On-Write or Schema-On-Read mode to process the logs.
For example, a Java log is a program log that contains both correct information and error information (such as information about abnormal stacks), including:
In this article, we will specify log line separation method, extract log fields and specify log time.
Alibaba Cloud Storage - June 19, 2019
Alibaba Clouder - May 17, 2019
Alibaba Clouder - November 19, 2019
XianYu Tech - September 24, 2019
Alibaba Clouder - November 20, 2019
Alibaba Clouder - August 31, 2018
More Posts by Alibaba Clouder