SchedulerX:FAQ about job management

Spring application cannot find beans

Work through these checks in order:

1. Go to the Application Management page and click the number in the Total number of instances column for your application. Confirm the boot mode is Spring or Spring Boot.

2. Make sure JobProcessor is injected as a bean in your code -- for example, by adding the @Component annotation.

3. If your pom.xml includes the spring-boot-devtools dependency, remove it. This dependency causes class-loading conflicts with SchedulerX.

4. If your JobProcessor or its process method uses any aspect-oriented programming (AOP) annotation, upgrade the SchedulerX agent to the latest version. Earlier versions do not support AOP annotations.

5. Check whether beans are proxied by unintended aspects. Add breakpoints to the DefaultListableBeanFactory class and inspect beanDefinitionNames to see all registered beans. If any are proxied by aspects from indirectly imported libraries, remove those aspects.

If none of these steps resolve the issue, debug ThreadContainer.start. A class.forName error for a class that exists typically means the Spring framework and the SchedulerXWorker agent are using different class loaders. Fix this by calling SchedulerxWorker.setClassLoader with the correct class loader.

Error: Unable to make field private

MapReduce jobs use serialization and deserialization frameworks that require reflective access to private fields. Java 9 and later restrict this access by default. Add the following JVM parameter:

--add-opens java.base/java.lang=ALL-UNNAMED

Error: submit jobInstanceId to worker timeout

If this error occurs during application deployments or only occasionally, it is safe to ignore.

If the error persists and the workAddr in each occurrence is the same, the persistent connection between the server and the agent has dropped. Restart the affected worker node, or upgrade the SchedulerX agent to the latest version. The connection recovers automatically after either action.

Error: used space beyond 90.0%!

The disk on your Elastic Compute Service (ECS) instance or container is full. Free up disk space to resolve this.

Error: ClassNotFoundException

The worker running the job does not have the specified class. Verify that the Processor class name for the Java job is the fully qualified class path (for example, com.example.MyJobProcessor).

If the class name is correct but the class is still missing from the worker, the likely cause is an incorrect deployment package or a connection to the wrong worker. Log on to the worker and use Java decompilation to inspect the deployed classes.

Error: jobInstance=xxx don't update progress more than 60s

When a running worker fails to report progress for 60 seconds, the server forcibly terminates it. If the worker crashed or no longer exists, this is expected behavior and can be ignored.

Job fails with no error message

This usually means your business logic failed without throwing an exception.

To investigate:

1. Open the Execution List page and click the Task instance List tab. Find the job and click Details in the Operation column. If no task details are available, the job is a standalone job. The WorkAddr on the Basic information tab identifies which worker ran the job.

2. Log on to that worker and open ~/logs/schedulerx/worker.log. Run the following command to filter logs for the specific instance:

   grep <Instance ID> worker.log

3. If the error description is empty, the failure is in your business logic. Check your application code for unhandled exceptions or silent failures.

4. If the error points to a SchedulerX framework exception, join DingTalk group 23103656 for technical support.

Identify the cause of a job failure

The approach depends on your job type and SchedulerX edition:

Job stuck in running state

If the issue is caused by business logic (for example, a deadlock or infinite loop), use one of these approaches:

• Professional Edition: Use the ThreadDump feature in the console to capture the thread stack. This feature requires agent version 1.4.2 or later.

• Basic Edition: Log on to the worker node and run:

  jstack <pid> | grep <Instance ID> -A 20

If the issue is not related to business logic, join DingTalk group 23103656 for technical support.

Slow job execution

Enable the Tracing Analysis feature in Professional Edition for end-to-end performance analysis. For details, see Integrate tracing analysis.

Job stuck or slow during application deployment

For distributed jobs, when a worker goes offline during deployment, SchedulerX redistributes its tasks and polls for online workers. This can slow down the entire job.

Upgrade the agent to version 1.7.9 or later to resolve this issue.

Job not triggered at the expected time

If a standalone job is not triggered, check the following:

1. Open the Application Management page and verify that workers appear in the worker list.

2. Confirm that not all workers are in the Busy state. For details on handling busy workers, see What to do when all workers are busy.

For related connection issues, see What do I do if the system displays the message "no workers available"?.

Turn on the No machine alarm available switch to receive alerts when no workers are available. For configuration details, see Job management.

What happens when a job does not finish before the next scheduled run

The default Instance concurrency is 1, meaning only one instance of the job runs at a time. If a job is still running when the next trigger time arrives, the new run is discarded -- it is not queued.

To allow overlapping runs, increase Instance concurrency. For example, setting it to 2 allows up to two instances of the same job to run simultaneously.

Create a one-time job

Set the Time type parameter to one_time. The job runs once and is then automatically destroyed. No execution records are kept.

View history for a one-time job

One-time jobs are destroyed after execution, so no historical records are available by default. To retain execution logs for up to two weeks, enable Log Service. For details, see Application management.

Set up second-delay scheduling

Set the Time type parameter to second_delay. The job runs repeatedly, with each run starting after a specified delay from the end of the previous run.

Jobs with Time type set to cron or fix_rate do not support second-delay scheduling.

Set the timeout period

SchedulerX supports timeout settings for jobs, but not for individual tasks. Configure the timeout in the console. For details, see Job management.

Error: The number of concurrent job instances reaches the upper limit

This error appears when you click Run once and the job already has the maximum number of instances running (as defined by Instance concurrency).

To check which instances are running, go to the Task Management page, find the job, and choose more > Historical records in the Operation column.

To increase the limit, click Edit for the job and change the Instance concurrency value in Advanced Configuration.

Job keeps running after stopping an instance

After you stop a job instance, SchedulerX sends a Kill message to the agent. The agent then:

• Stops delivering new tasks.

• Destroys the instance context and all thread pools.

• Interrupts threads for tasks that are already executing.

However, tasks that are mid-execution continue running until they complete -- the interrupt signal only requests a stop, it does not force one.

To stop tasks immediately: Modify your task processing logic to check for the interrupt state on the current thread and exit gracefully when detected.

What to do when all workers are busy

1. Go to the Application Management page and click View instances for the application.

2. Identify workers in the Busy state. Hover over Busy to see which metrics exceed the threshold.

3. To adjust thresholds, click Edit for the application and update the settings in the Edit application grouping panel.

Kubernetes deployments: If the CPU utilization metric appears incorrect, it may be because the agent is reading host-level metrics instead of container-level metrics. Add the following configuration:

For setup instructions, see Connect a Spring Boot application to SchedulerX.

What is the instance parameters field in the Run once dialog

When you click Run once on the Task Management page, the Instance Parameters field is optional. It is intended for testing and lets you pass parameters to a single manual run without changing the job configuration.

Instance parameters vs. job parameters

Instance parameters and job parameters are separate values. Which one your code reads depends on your implementation. Access either through JobContext:

@Component
public class JavaDemoProcessor extends JavaProcessor {

    private static final Logger LOGGER = LoggerFactory.getLogger("schedulerxLog");

    @Override
    public ProcessResult process(JobContext jobContext) throws InterruptedException {

        // Get job parameters (configured in the job definition)
        String jobParameters = jobContext.getJobParameters();

        // Get instance parameters (passed via Run once dialog)
        String instanceParameters = jobContext.getInstanceParameters();

        LOGGER.info("Job parameters: " + jobParameters);
        LOGGER.info("Instance parameters: " + instanceParameters);
        return new ProcessResult(InstanceStatus.SUCCESS);
    }
}

Fall back to job parameters when no instance parameters are set

Read instance parameters when available, and fall back to job parameters otherwise:

@Component
public class JavaDemoProcessor extends JavaProcessor {

    private static final Logger LOGGER = LoggerFactory.getLogger("schedulerxLog");

    @Override
    public ProcessResult process(JobContext jobContext) throws InterruptedException {
        String params;
        if (StringUtils.isNotBlank(jobContext.getInstanceParameters())) {
            params = jobContext.getInstanceParameters();
        } else {
            params = jobContext.getJobParameters();
        }
        LOGGER.info("JavaDemoProcessor params: {}", params);
        return new ProcessResult(InstanceStatus.SUCCESS);
    }
}

Configure advanced job settings

For the full list of advanced parameters (retry policies, data expiration, custom alarm rules, and more), see Advanced parameters for job management.

Integrate tracing analysis

SchedulerX supports end-to-end tracing for job execution. For setup instructions, see Integrate tracing analysis.