Connect non-Java applications to SchedulerX using the agent - Microservices Engine

If your applications are not Java-based, you cannot integrate SchedulerX through the SDK. The SchedulerX agent solves this by running as a lightweight Java process on your server. It connects to the SchedulerX server and runs script or HTTP jobs on the local machine. After deploying the agent, you can create scheduled jobs, rerun tasks, and view execution history and logs from the SchedulerX console.

Three deployment methods are available: manual installation from a tar package, Docker, and Kubernetes.

How it works

You deploy the agent on your machines and configure it with your SchedulerX namespace and application credentials.
The agent connects to the SchedulerX server and registers itself as a worker.
When a scheduled task triggers, SchedulerX dispatches the job to the agent. The agent runs the script or sends the HTTP request on the local machine.
The agent reports the execution result and logs back to the SchedulerX server, which you can view in the console.

Use cases

Periodic log cleanup: Run a shell script to clear logs across multiple servers or start a service on a schedule.
Database maintenance: Run a Python script to purge historical data from a database.
Order processing: Call an HTTP endpoint to scan orders at regular intervals.

Note

The agent also supports DataWorks jobs. For details, see DataWorks tasks.

Prerequisites

Before you begin, make sure you have:

Java Runtime Environment (JRE) 1.8 or later on the target machine
A resource is created. For details, see Create a resource
(Optional) A dedicated namespace. For details, see Create a namespace

Gather the following values from the SchedulerX console before you start:

Parameter	Where to find it	Example
Endpoint	Endpoints reference	`addr-hz-internal.edas.aliyun.com`
Namespace ID	Namespace page	`fd2965c4-xxxx-xxxx-af52-bb62aa4f19f2`
Application ID (groupId)	Application Management page	`hxm.test`
Application key (appKey)	Application Management page	`12******789`

Deploy the agent manually

Step 1: Download and extract the agent

Download the latest agent package to your server:

wget https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.13.5.tar.gz
tar -xzf schedulerxAgent-1.13.5.tar.gz

Important

If wget times out, your machine may not have public internet access. Either enable public network access or transfer the file manually. See Upload or download files (Linux).

Older agent versions

Version	Download URL	Changes
Agent 1.13.5 (latest)	schedulerxAgent-1.13.5.tar.gz	second_delay tasks now support graceful shutdown; script tasks now support upstream and downstream data transfer in workflows; supports cgroup v2 for collecting system performance metrics
Agent 1.13.2	schedulerxAgent-1.13.5.tar.gz	CPU metric collection under cgroup v2; custom thread pool sizes (`share.pool.size` parameter)
Agent 1.12.5	schedulerxAgent-1.12.5.tar	Task system variables via placeholders; custom thread pool sizes for HTTP tasks; script tasks return last line of stdout as the execution result
Agent 1.11.5	schedulerxAgent-1.11.5.tar	Security fixes for Netty and Logback; fix for "too old resource version" in K8s tasks
Agent 1.10.13	schedulerxAgent-1.10.13.tar	Multiple agents on a single machine; HTTP responses larger than 1,000 bytes; reduced log storage
Agent 1.10.5	schedulerxAgent-1.10.5.tar	Graceful shutdown; Simple Log Service (SLS) application group fencing
Agent 1.9.8	schedulerxAgent-1.9.8.tar	Agent-mode HTTP tasks; script failure returns last log line; K8s tasks
Agent 1.7.10	schedulerxAgent-1.7.10.tar.gz	Fix for slow second-level tasks; fix for SLS out-of-memory (OOM) error
Agent 1.4.2	schedulerxAgent-1.4.2.tar.gz	Simple Log Service support

Step 2: Configure the agent

Edit the configuration file at schedulerxAgent/conf/agent.properties:

# Internal endpoint for the SchedulerX server
endpoint=addr-hz-internal.edas.aliyun.com

# For public network access, use domainName instead of endpoint:
# domainName=<your-domain-name>

# Namespace ID from the SchedulerX console
namespace=<your-namespace-id>

# Application ID and key from the Application Management page
groupId=<your-group-id>
appKey=<your-app-key>

Replace the placeholders with your actual values:

Placeholder	Description	Example
`<your-namespace-id>`	Namespace ID from the Namespace page	`fd2965c4-xxxx-xxxx-af52-bb62aa4f19f2`
`<your-group-id>`	Application ID from the Application Management page	`hxm.test`
`<your-app-key>`	Application key from the Application Management page	`12******789`

Step 3: Start the agent

Run the startup script:

cd schedulerxAgent/bin
./start.sh

To allocate a specific amount of memory at startup, use one of these scripts instead:

Script	Memory
`start-200m.sh`	200 MB
`start-500m.sh`	500 MB
`start-1g.sh`	1 GB

Verify the connection

Check the agent log at ${user.home}/logs/schedulerx/worker.log. The log path depends on which user started the process:

User	Log path
admin	`/home/admin/logs/schedulerx/worker.log`
root	`/root/logs/schedulerx/worker.log`

To determine the user, run:

ps aux | grep java

A successful startup produces this message:

Schedulerx Worker started

Important

The agent requires Java Development Kit (JDK) 1.8 through 14. For JDK 15 or later, remove the -XX:+UseConcMarkSweepGC parameter from schedulerxAgent/bin/start.sh before starting the agent.

Stop the agent

cd schedulerxAgent/bin
./stop.sh

Deploy the agent with Docker

Step 1: Select an image

Choose an image based on your region and CPU architecture. The following table lists public network registry addresses:

Region	x86_64	arm64
China (Hangzhou)	`schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64`	`schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-arm64`
Singapore	`schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-amd64`	`schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-arm64`

Step 2: Run the container

docker run -d \
  --env SCHEDULERX_ENDPOINT="<your-endpoint>" \
  --env SCHEDULERX_NAMESPACE="<your-namespace-id>" \
  --env SCHEDULERX_GROUPID="<your-group-id>" \
  --env SCHEDULERX_APPKEY="<your-app-key>" \
  <image-address>

Replace the placeholders with your actual values:

Placeholder	Description
`<your-endpoint>`	Endpoint for your region. See Endpoints
`<your-namespace-id>`	Namespace ID from the Namespace page
`<your-group-id>`	Application ID from the Application Management page
`<your-app-key>`	Application key from the Application Management page
`<image-address>`	Image address from the table above

Verify the connection

Check the container logs:

docker logs <container-id>

A successful startup produces the Schedulerx Worker started message.

Deploy the agent on Kubernetes

Apply a Kubernetes Deployment to run the agent as a pod.

Sample schedulerx-agent.yaml file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: schedulerx-agent
  labels:
    app: schedulerx-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: schedulerx-agent
  template:
    metadata:
      labels:
        app: schedulerx-agent
    spec:
      containers:
      - name: schedulerx-agent
        image: schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64
        imagePullPolicy: Always
        resources:
          requests:
            cpu: 500m
        env:
          - name: "SCHEDULERX_ENDPOINT"
            value: "<your-endpoint>"
          - name: "SCHEDULERX_NAMESPACE"
            value: "<your-namespace-id>"
          - name: "SCHEDULERX_GROUPID"
            value: "<your-group-id>"
          - name: "SCHEDULERX_APPKEY"
            value: "<your-app-key>"
          - name: "SCHEDULERX_STARTER_MODE"
            value: "pod"
        livenessProbe:
          exec:
            command: ["/bin/bash","/root/health.sh"]
          timeoutSeconds: 30
          initialDelaySeconds: 30

Environment variables

Variable	Description
`SCHEDULERX_ENDPOINT`	Endpoint for your region. For example, `addr-sh-internal.edas.aliyun.com`. See Endpoints
`SCHEDULERX_NAMESPACE`	Namespace ID from the Namespace page
`SCHEDULERX_GROUPID`	Application ID from the Application Management page
`SCHEDULERX_APPKEY`	Application key from the Application Management page
`SCHEDULERX_STARTER_MODE`	Set to `pod` for Kubernetes deployments
`SCHEDULERX_SHARE_POOL_SIZE`	Optional. Custom thread pool size for task execution

Apply the deployment

kubectl apply -f schedulerx-agent.yaml

Verify the connection

kubectl logs deployment/schedulerx-agent

A successful startup produces the Schedulerx Worker started message.