Microservices Engine:Connect to an agent (script/big data/SQL tasks)

Prerequisites

• JDK 17 or later

Step 1: Download and extract the package

wget https://schedulerx3.oss-cn-hangzhou.aliyuncs.com/xxljob/schedulerx3-agent-1.0.0-bin.tar.gz
tar -zxvf schedulerx3-agent-1.0.0-bin.tar.gz
cd schedulerx3-agent-1.0.0-bin

Directory structure after extraction:

schedulerx3-worker-2.4.2-jdk17-bin/
├── bin/              # Startup and management scripts
├── conf/             # Configuration files
│   ├── application.yml      # Application configuration
│   └── logback-spring.xml   # Log configuration
├── lib/              # Dependency JARs
└── logs/             # Log output (created at runtime)

Step 2: Configure the agent

Edit conf/application.yml with your MSE-XXLJOB instance details:

xxl:
  job:
    admin-addresses: <service-registration-address>
    access-token: <application-access-token>
    executor:
      appname: <application-app-name>

Replace the placeholders with values from the MSE console:

Step 3: Start the agent

Linux/Mac

# Start in the background
./bin/start.sh

# Start in the foreground (for debugging)
./bin/start.sh -f

# Stop
./bin/stop.sh

# Restart
./bin/restart.sh

# Check status
./bin/status.sh

# View logs
tail -f logs/worker.log

Windows

REM Start in the background
.\bin\start.cmd

REM Start in the foreground (for debugging)
.\bin\start.cmd -f

REM Stop
.\bin\stop.cmd

REM Restart
.\bin\restart.cmd

REM Check status
.\bin\status.cmd

REM View logs
type logs\worker.log

Step 4: Verify the connection

After starting the agent, confirm that it connected successfully:

1. Check logs/worker.log for a registration success message.

2. In the MSE console, go to Application Management and verify that the executor appears under your application.

If the executor does not appear, check the following:

• The service registration address, AppName, and AccessToken are correct.

• The machine can reach the admin server URL over the network.

• Port 9999 is open for inbound connections from the admin server.

(Optional) Configure logging

Log files are stored in the logs/ directory. Task execution logs default to ${user.home}/applogs/xxl-job/jobhandler.

To adjust log levels, edit conf/logback-spring.xml:

<!-- Change the root log level -->
<root level="INFO">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="FILE" />
</root>
<!-- Set a package-specific log level -->
<logger name="com.aliyun.schedulerx" level="DEBUG" />
<logger name="com.xxl.job" level="DEBUG" />

(Optional) Tune JVM parameters

Adjust JVM memory based on your workload.

Linux/Mac

# Set temporarily for a single run
JAVA_OPTS="-Xms2g -Xmx4g" ./bin/start.sh

# Set permanently
vim bin/start.sh  # Edit the JAVA_OPTS variable

Windows

REM Set temporarily for a single run
set JAVA_OPTS=-Xms2g -Xmx4g
.\bin\start.cmd

REM Set permanently
notepad bin\start.cmd  REM Edit the JAVA_OPTS variable

Method 1: Use the public image

The public image includes pre-installed Python, Node.js, and Go runtimes for script tasks:

# Pull the image
docker pull schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx3/schedulerx3-agent:1.0.0

# Run the agent
docker run -d \
  --name schedulerx3-agent \
  -p 9999:9999 \
  -e JAVA_OPTS="-Xms1g -Xmx2g" \
  -e SCHEDULERX3_ADMIN_ADDRESSES="<service-registration-address>" \
  -e SCHEDULERX3_EXECUTOR_APPNAME="<application-app-name>" \
  -e SCHEDULERX3_ACCESS_TOKEN="<application-access-token>" \
  -v $(pwd)/logs:/opt/schedulerx3-agent/logs \
  --restart unless-stopped \
  schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx3/schedulerx3-agent:1.0.0

After the container starts, verify the connection in the MSE console under Application Management.

Method 2: Build a custom image from the tar package

If your tasks require additional dependencies or a custom base image, build your own Docker image:

# Download the installation package
wget https://schedulerx3.oss-cn-hangzhou.aliyuncs.com/xxljob/schedulerx3-agent-1.0.0-bin.tar.gz

# Build the image
docker build -t schedulerx3-agent:1.0.0 -f Dockerfile .

Sample Dockerfile:

################################################################################
### Customize this Dockerfile to install components your tasks require.
################################################################################

# Use your preferred base image with JDK 17+
FROM hub.docker.xxx.com/library/openjdk:17.0.1-jdk-bullseye

LABEL maintainer="SchedulerX Team"
LABEL description="SchedulerX3 Agent - XXL-Job Executor"
LABEL version="2.4.2"

# Configure Alibaba Cloud mirror sources
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list && \
    sed -i 's|security.debian.org/debian-security|mirrors.aliyun.com/debian-security|g' /etc/apt/sources.list

# Install Python 3, Node.js, and Go
RUN apt-get update && \
    apt-get install -y python3 python3-distutils curl wget ca-certificates nodejs npm golang-go && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Install pip
RUN curl https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py && \
    python3 /tmp/get-pip.py && \
    rm -f /tmp/get-pip.py && \
    ln -sf /usr/bin/python3 /usr/bin/python

# Set Go environment variables
ENV GOPATH=/root/go
ENV PATH=$GOPATH/bin:$PATH
ENV GO111MODULE=on

# Extract the agent package
COPY schedulerx3-agent-*-bin.tar.gz /tmp/schedulerx3-agent.tar.gz
RUN mkdir -p /opt/schedulerx3-agent && \
    tar -xzf /tmp/schedulerx3-agent.tar.gz --strip-components=1 -C /opt/schedulerx3-worker && \
    chmod +x /opt/schedulerx3-agent/bin/*.sh && \
    mkdir -p /opt/schedulerx3-agent/logs && \
    rm -f /tmp/schedulerx3-agent.tar.gz

WORKDIR /opt/schedulerx3-agent

EXPOSE 9999

CMD ["bin/start.sh", "-f"]