All Products
Search

This Product

    SchedulerX:Integrate Simple Log Service with SchedulerX 2.0

    Document Center

    SchedulerX:Integrate Simple Log Service with SchedulerX 2.0

    Last Updated:Mar 28, 2024

    To integrate Simple Log Service with SchedulerX 2.0, you only need to add the Log4j or Logback configuration. Then, you can view the logs of the jobs, including distributed jobs, that are triggered in the SchedulerX 2.0 console. This topic describes how to integrate Simple Log Service with SchedulerX 2.0 and view the logs of jobs.

    Prerequisites

    • A namespace is created. For more information, see Create resources.

    • Resources are created. For more information, see Create resources.

    • The SchedulerX edition used by your application is upgraded to SchedulerX 2.0 Professional Edition and Simple Log Service is activated.

      image.png

      Note

      After you activate Simple Log Service, you can retain logs for up to two weeks. After the logs expire, the system automatically deletes the logs.

    Procedure

    Update the SchedulerX agent

    Update the SchedulerX client to the latest version. For more information about the latest version, see Agent release notes. In the following sample code, Spring Boot Starter is used. Add the SchedulerxWorker dependency to the pom.xml file of the application. 

    <dependency>
    <groupId>com.aliyun.schedulerx</groupId>
    <artifactId>schedulerx2-spring-boot-starter</artifactId>
    <version>${Latest version of the SchedulerX client}</version>  
</dependency>

    Configure a log appender to collect logs

    Log4j2 Appender

    • Method 1: Collect only logs of the triggered jobs

      You can collect only the logs of the triggered jobs for troubleshooting based on your business requirements.

      1. Add an appender and a logger to Log4j 2. The following example shows the configurations:

        <?xml version="1.0" encoding="UTF-8"?>
<Configuration status="off">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout
                pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n" />
        </Console>
        <SchedulerxLog4j2Appender name="schedulerxLog"
            timeFormat="yyyy-MM-dd'T'HH:mmZ"
            timeZone="UTC"
            ignoreExceptions="true">
        <PatternLayout pattern="%d %-5level [%thread] %logger{0}: %msg"/>
    </SchedulerxLog4j2Appender>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="Console" />
        </Root>
        <Logger name="schedulerx" level="info" additivity="false">
            <AppenderRef ref="schedulerxLog" />
        </Logger>
    </Loggers>
</Configuration>

      2. Modify the code by referring to the following example. Collect the logs of the triggered jobs by using the logger named schedulerx. 

        package com.hxm.test.processor;

import com.alibaba.schedulerx.worker.domain.JobContext;
import com.alibaba.schedulerx.worker.processor.JavaProcessor;
import com.alibaba.schedulerx.worker.processor.ProcessResult;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;


@Component
public class HelloWorldJob3 extends JavaProcessor {
    private static final Logger logger = LoggerFactory.getLogger("schedulerx");

    @Override
    public ProcessResult process(JobContext context) throws Exception {
        LOGGER.info("hello HelloWorldJob3");
        return new ProcessResult(true);
    }

}

    • Method 2: Collect all service logs

      If you use this method, you need to only configure the Log4j file. If you configure the appender named schedulerxLog in the root logger, all logs are collected by using SchedulerX. This will mix service logs with logs of the triggered jobs, which makes troubleshooting difficult and compromises query efficiency. The following sample shows the configurations:

      <?xml version="1.0" encoding="UTF-8"?>
<Configuration status="off">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout
                pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n" />
        </Console>
        <SchedulerxLog4j2Appender name="schedulerxLog"
            timeFormat="yyyy-MM-dd'T'HH:mmZ"
            timeZone="UTC"
            ignoreExceptions="true">
        <PatternLayout pattern="%d %-5level [%thread] %logger{0}: %msg"/>
    </SchedulerxLog4j2Appender>
    </Appenders>
    <Loggers>
        <Root level="info">
          <AppenderRef ref="Console" />
          <AppenderRef ref="schedulerxLog" />
        </Root>
    </Loggers>
</Configuration>

    Log4j Appender

    • Method 1: Collect only logs of the triggered jobs

      1. Add an appender and a logger to the log4j.properties file. 

        log4j.rootLogger = INFO,console
log4j.logger.schedulerx=schedulerxLog
log4j.appender.schedulerxLog=com.alibaba.schedulerx.worker.log.appender.SchedulerxLog4jAppender

      2. Modify the code by referring to the following example. Collect the logs of the triggered jobs by using the logger named schedulerx. 

        package com.hxm.test.processor;

import com.alibaba.schedulerx.worker.domain.JobContext;
import com.alibaba.schedulerx.worker.processor.JavaProcessor;
import com.alibaba.schedulerx.worker.processor.ProcessResult;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;


@Component
public class HelloWorldJob3 extends JavaProcessor {
    private static final Logger logger = LoggerFactory.getLogger("schedulerx");

    @Override
    public ProcessResult process(JobContext context) throws Exception {
        logger.info("hello HelloWorldJob3");
        return new ProcessResult(true);
    }

}

    • Method 2: Collect all logs

      Add an appender to the log4j.properties file and configure this appender in the root logger.

      log4j.rootLogger = INFO,console,schedulerxLog
log4j.appender.schedulerxLog=com.alibaba.schedulerx.worker.log.appender.SchedulerxLog4jAppender

    Logback Appender

    Note

    The SchedulerxLogbackAppender class is in the schedulerx2-worker.jar package. You must update the SchedulerX client to the latest version. For more information about the latest version of the client, see Agent release notes.

    • Method 1: Collect only logs of the triggered jobs

      1. Add an appender and a logger to the logback.xml file. 

        <appender name="schedulerxLog" class="com.alibaba.schedulerx.worker.log.appender.SchedulerxLogbackAppender">
    <timeFormat>yyyy-MM-dd'T'HH:mmZ</timeFormat>
    <timeZone>UTC</timeZone>
</appender>

<root level="INFO">
    <appender-ref ref="CONSOLE" />
</root>
<logger name="schedulerx" level="INFO">
    <appender-ref ref="schedulerxLog"/>
</logger>

      2. Modify the code by referring to the following example. Collect the logs of the triggered jobs by using the logger named schedulerx. 

        package com.hxm.test.processor;

import com.alibaba.schedulerx.worker.domain.JobContext;
import com.alibaba.schedulerx.worker.processor.JavaProcessor;
import com.alibaba.schedulerx.worker.processor.ProcessResult;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;


@Component
public class HelloWorldJob3 extends JavaProcessor {
    private static final Logger logger = LoggerFactory.getLogger("schedulerx");

    @Override
    public ProcessResult process(JobContext context) throws Exception {
        LOGGER.info("hello HelloWorldJob3");
        return new ProcessResult(true);
    }

}

    • Method 2: Collect all logs

      Add an appender to the logback.xml file and configure the appender in the root logger.

      <appender name="schedulerxLog" class="com.alibaba.schedulerx.worker.log.appender.SchedulerxLogbackAppender">
    <timeFormat>yyyy-MM-dd'T'HH:mmZ</timeFormat>
    <timeZone>UTC</timeZone>
</appender>

<root level="INFO">
    <appender-ref ref="CONSOLE" />
    <appender-ref ref="schedulerxLog"/>
</root>

    View the logs

    1. Log on to the SchedulerX console.

    2. In the left-side navigation pane, click Task Management.

    3. Find the job that you want to manage in the list of jobs and choose more>Historical records in the Operation column of the job.

    4. On the Task instance records page, find the desired job and click Log in the Operation column of the job.

    The log contains the following fields:

    • ip: the IP address of the worker that printed the log.

    • executionId: the execution ID of the job instance. The value is in the ${jobId}_${jobInstanceId}_${taskId} format.

    • level: the log level.

    • log: the log content.

    • throwable: the error field. This field is printed only if the application encounters an error.

    • time: the time when the log was printed.

    Scenarios and demos

    Check the cause of a service failure

    After you integrate Simple Log Service with SchedulerX 2.0, SchedulerX 2.0 can collect job logs, collect service logs, and record exceptions.

    1. Create code.

      1. Create a job and use SchedulerX 2.0 to print logs. 

        package com.hxm.test.processor;

import com.alibaba.schedulerx.test.service.TestService;
import com.alibaba.schedulerx.worker.domain.JobContext;
import com.alibaba.schedulerx.worker.processor.JavaProcessor;
import com.alibaba.schedulerx.worker.processor.ProcessResult;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class HelloWorldJob extends JavaProcessor {
    // Use SchedulerX 2.0 to collect logs.
    private static final Logger LOGGER = LogManager.getLogger("schedulerx");

    @Autowired
    private TestService testService;

    @Override
    public ProcessResult process(JobContext context) throws Exception {
        String parameters = context.getJobParameters();
        String tokens[] = parameters.split(" ");
        int a = Integer.valueOf(tokens[0]);
        int b = Integer.valueOf(tokens[1]);
        int c = testService.doDivision(a, b);
        LOGGER.info("testService.doDivision finished, a={}, b={}, c={}", a, b, c);
        if (c < 0) {
            return new ProcessResult(false, "result=" + c);
        }
        return new ProcessResult(true);
    }

}

      2. Create code of a Service and use SchedulerX 2.0 to print logs. 

        package com.hxm.test.service;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.stereotype.Service;

@Service("testService")
public class TestServiceImpl implements TestService {
    // Use SchedulerX 2.0 to collect logs. 
    private static final Logger LOGGER = LogManager.getLogger("schedulerx");

    @Override
    public int doDivision(int a, int b) {
        try {
             LOGGER.info("start to do division c = " + a + "/" + b);
             int c = a/b;
             LOGGER.info("c=" + c);
             return c;
        } catch (Exception e) {
            LOGGER.error("", e);
        }
        return -1;
    }

}

    2. Configure the job in the SchdulerX console.

      If you divide 1 by 0, an error occurs due to the business logic of the job.

    3. Run the job once, go to the Task instance records page, and view the log of the job.

      The log content indicates that the job failed because TestServiceImpl threw a "by zero" error. 日志除0

    Check the cause of a distributed job error

    You can use distributed jobs in SchedulerX 2.0 to perform batch processing. If a distributed job fails, you can perform the following operations to identify the task that fails:

    1. Create code of a job and use SchedulerX 2.0 to print logs. 

      package com.hxm.test.processor;

import java.util.ArrayList;
import java.util.List;

import com.alibaba.schedulerx.worker.domain.JobContext;
import com.alibaba.schedulerx.worker.processor.MapJobProcessor;
import com.alibaba.schedulerx.worker.processor.ProcessResult;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class TestMapJobProcessor extends MapJobProcessor {
    private static final Logger LOGGER = LogManager.getLogger("schedulerx");

    @Override
    public ProcessResult process(JobContext context) throws Exception {
        String taskName = context.getTaskName();
        String parameter = context.getJobParameters();
        int dispatchNum = Integer.valueOf(parameter);
        if (isRootTask(context)) {
            LOGGER.info("start root task");
            List<String> msgList = new ArrayList<>();
            for (int i = 0; i <= dispatchNum; i++) {
                msgList.add("msg_" + i);
            }

            return map(msgList, "Level1Dispatch");
        } else if (taskName.equals("Level1Dispatch")) {
            String task = (String)context.getTask();
            if (task.equals("msg_23")) {
                LOGGER.error("msg={}, failed", task);
                return new ProcessResult(false);
            } else {
                LOGGER.info("msg={}, success", task);
            }
            return new ProcessResult(true);
        }
        return new ProcessResult(false);
    }
}

    2. Configure the job in the SchedulerX console and trigger the job once.

    3. View the log. For more information, see View the logs.

    4. Search for keywords in the log to quickly locate the failed task and the cause.

    Query log data by keyword

    SchedulerX 2.0 retains only the most recent 60 execution records of a job instance. If you want to troubleshoot errors based on historical execution records, click Log query in the left-side navigation pane of the SchedulerX console. On the Log query page, you can query log data by job ID, keyword, or time range.