Agent connection (script or HTTP jobs)

Connect an agent to SchedulerX to create and manage scheduled jobs, view execution history, and check run logs from the console. This guide explains how to connect your non-Java applications using the agent as a tar package or Docker image.

Prerequisites

(Optional) You have created a namespace. For more information, see Create a namespace.
You have created a resource. For more information, see Create a resource.

Note

The agent supports the scheduling and execution of DataWorks jobs. For more information, see DataWorks jobs.

Use cases

Common use cases for connecting to SchedulerX with an agent include:

Using a Shell script to periodically clean up logs on all machines or start a service.
Using a Python script to periodically clean up historical data in a database.
Using an HTTP API to periodically scan for orders.

Environment requirements

A Java Runtime Environment (JRE) of version 1.8 or later is required.

Manually deploy the agent

Download and decompress the agent package.

If your machine has a public IP address, you can run the following command to download the package:

Note

Replace X.X.X with the required version number, for example, 1.12.5.

wget https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-X.X.X.tar

Version	Download URL	Description
Agent 1.13.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.13.5.tar.gz	New features: second_delay tasks support graceful shutdown. Script tasks support upstream/downstream data passing in workflows. Support collecting system performance metrics via cgroup v2.
Agent 1.13.2	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.13.2.tar.gz	New features: Supports CPU metric collection under cgroup v2. Supports customizing the size of the job execution thread pool. The parameter key is `share.pool.size`.
Agent 1.12.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.12.5.tar	New features: Supports obtaining job system variables by using placeholders. Supports custom thread pool sizes for HTTP jobs. Script jobs can return the last line of standard output as the execution result.
Agent 1.11.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.11.5.tar	Fixed issues: Fixes security vulnerabilities in Netty and Logback. Fixes the "too old resource version" issue for Kubernetes jobs.
Agent 1.10.13	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.10.13.tar	New features: Supports deploying multiple agents on a single machine. HTTP agent jobs support responses larger than 1,000 bytes. Optimizes logging to reduce historical log storage.
Agent 1.10.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.10.5.tar	New features: Supports graceful shutdown. The log service supports isolation by application group.
Agent 1.9.8	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.9.8.tar	New features: Supports HTTP jobs in agent mode. Supports returning the last log entry as the result if a script job fails. Supports Kubernetes jobs.
Agent 1.7.10	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.7.10.tar.gz	Fixed issues: Fixes an issue where second-level jobs run slowly. Fixes an issue where the log service might cause out-of-memory (OOM) errors.
Agent 1.4.2	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.4.2.tar.gz	New feature: Supports the log service.

Important

If the connection times out when you use the wget command to download the agent, enable a public IP address for your machine or see Upload or download files on a Linux instance.

Go to the schedulerxAgent/conf directory, and configure the agent.properties file.
```
endpoint=addr-hz-internal.edas.aliyun.com
namespace=fd2965c4-****-****-af52-bb62aa4*****
groupId=hxm.test
appKey=1234*****
```
- Endpoint specifies the endpoint. For more information, see the Endpoint list.
- namespace is the namespace ID. You can obtain the ID on the Namespace page.
- The application ID and application key are the unique identifier and account of an application. You can obtain them on the Application Management page.
Go to the schedulerxAgent/bin directory and run the start.sh script to start the agent.
Note
If you want to set the startup memory for the agent, you can run start-200m.sh, start-500m.sh, or start-1g.sh.
To manage the agent, use the following commands:
- To stop the agent, go to the schedulerxAgent/bin directory and run the stop.sh script.
- You can view the log at ${user.home}/logs/schedulerx/worker.log.
  You can find the user.home directory by running the ps aux | grep java command. If the process is started by the admin user, the log path is /home/admin/logs/schedulerx/worker.log. If the process is started by the root user, the log path is /root/logs/schedulerx/worker.log.
If the returned message contains Schedulerx Worker started, the deployment is successful.
Note
The runtime environment requires JDK 1.8 to 14. If you use a JDK version higher than 14, remove -XX:+UseConcMarkSweepGC from the schedulerxAgent/bin/start.sh file in the Agent installation directory.

Deploy the agent using a Docker image

Select an image based on your network environment and CPU architecture.

Network	x86_64	arm64
(Public network) China (Hangzhou)	schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64	schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-arm64
(Public network) Singapore	schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-amd64	schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-arm64

Deploy the agent.
Environment variables:
```
docker run  --env SCHEDULERX_ENDPOINT="addr-hz-internal.edas.aliyun.com" \
   --env SCHEDULERX_NAMESPACE="fd2965c4-****-****-af52-bb62aa4*****" \
   --env SCHEDULERX_GROUPID="hxm.test" \
   --env SCHEDULERX_APPKEY="1234*****" \
   -d {image id}
```
- SCHEDULERX_ENDPOINT is the Endpoint.
- SCHEDULERX_NAMESPACE is the namespace ID, which you can obtain from the Namespace or page in the console.
- SCHEDULERX_GROUPID is the application ID, and SCHEDULERX_APPKEY is the application key. You can obtain them on the Application Management page in the console.
After you start the deployment task, you can use the docker logs command to view the logs.

Deploy the agent using a Kubernetes deployment

If you already have a k8s environment, the most convenient option is to deploy schedulerx-agent.yaml directly.

schedulerx-agent.yaml sample code:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: schedulerx-agent
  labels:
    app: schedulerx-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: schedulerx-agent
  template:
    metadata:
      labels:
        app: schedulerx-agent
    spec:
      containers:
      - name: schedulerx-agent
        image: schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64
        imagePullPolicy: Always
        resources:
          requests:
            cpu: 500m
        env:
          - name: "SCHEDULERX_ENDPOINT"
            value: "${SCHEDULERX_ENDPOINT}"
          - name: "SCHEDULERX_NAMESPACE"
            value: "${SCHEDULERX_NAMESPACE}"
          - name: "SCHEDULERX_GROUPID"
            value: "${SCHEDULERX_GROUPID}"
          - name: "SCHEDULERX_APPKEY"
            value: "${SCHEDULERX_APPKEY}"
          - name: "SCHEDULERX_STARTER_MODE"
            value: "pod"
        livenessProbe: 
          exec: 
            command: ["/bin/bash","/root/health.sh"]
          timeoutSeconds: 30
          initialDelaySeconds: 30

Variable	Description
${SCHEDULERX_ENDPOINT}	The region where you deploy your application and the corresponding Endpoint. For more information, see Endpoint list. For example, `addr-sh-internal.edas.aliyun.com`.
${SCHEDULERX_NAMESPACE}	The namespace ID. You can obtain it from the Namespace page in the left-side navigation pane of the console.
${SCHEDULERX_GROUPID}	The application ID. You can obtain it from the Application Management page in the left-side navigation pane of the console.
${SCHEDULERX_APPKEY}	The application key. You can obtain it from the Application Management page in the left-side navigation pane of the console.