SchedulerX:Connect via agent (script or HTTP jobs)

Use cases

The following are common use cases for connecting an agent to SchedulerX:

• Use a shell script to periodically clean logs on all worker nodes or start a service.

• Use a Python script to periodically clean historical data from a database.

• Use an HTTP API to periodically scan orders.

Environment requirements

A Java Runtime Environment (JRE) 1.8 or later is required.

Manually deploy the agent

1. Download and decompress the agent.

   If your worker node has a public IP address, you can run the following command to download the agent:

   Note

   Replace X.X.X with the required version number, for example, 1.12.5.

   wget https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-X.X.X.tar

   Version	Download url	Release notes
   Agent 1.14.0	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.14.0.tar.gz	New features: Script tasks now support forced termination. clientInfo.log prints the full script output, which you can collect through SLS.
   Agent 1.13.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.13.5.tar.gz	New features: Supports graceful shutdown for second-delay jobs. Supports data transfer between upstream and downstream jobs in a workflow for script jobs. Supports collecting system performance metrics in cgroup v2.
   Agent 1.13.2	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.13.2.tar.gz	New features: Supports collecting CPU metrics in cgroup v2. Supports customizing the thread pool size for job execution. Parameter: share.pool.size.
   Agent 1.12.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.12.5.tar	New features: Supports using placeholders to obtain system variables for jobs. Supports customizing the thread pool size for HTTP jobs. Enables script jobs to return the last line of standard output as the result.
   Agent 1.11.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.11.5.tar	Bug fixes: Fixes security vulnerabilities in Netty and Logback. Fixes the "too old resource version" issue for Kubernetes jobs.
   Agent 1.10.13	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.10.13.tar	New features: Supports deploying multiple agents on a single worker node. Supports responses larger than 1,000 bytes for HTTP agent jobs. Optimizes the logging feature to reduce historical log storage.
   Agent 1.10.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.10.5.tar	New features: Supports graceful shutdown. The logging feature supports application group isolation.
   Agent 1.9.8	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.9.8.tar	New features: Supports HTTP jobs in agent mode. Supports returning the last log line if a script job fails. Supports Kubernetes jobs.
   Agent 1.7.10	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.7.10.tar.gz	Bug fixes: Fixes an issue where second-delay jobs ran slowly. Fixes an issue where the logging feature could cause an Out of Memory (OOM) error.
   Agent 1.4.2	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.4.2.tar.gz	New feature: Supports the logging feature.

   Important

   If the wget command times out while downloading the agent, enable a public IP address for your worker node or see Upload or download files (Linux).

2. Go to the schedulerxAgent/conf directory and configure the agent.properties file.

   endpoint=addr-hz-internal.edas.aliyun.com
   namespace=fd2965c4-****-****-af52-bb62aa4*****
   groupId=hxm.test
   appKey=1234*****

   • endpoint: The service endpoint. For more information, see List of endpoints.

   • namespace: The namespace ID. You can find it on the Namespace or page.

   • groupId and appKey are your application's unique ID and key. You can find them on the Application Management page.

3. Go to the schedulerxAgent/bin directory and run the start.sh command.

   Note

   To specify the memory allocation for the agent at startup, you can run start-200m.sh, start-500m.sh, or start-1g.sh.

   After the agent starts, you can perform the following operations:

   • To stop the agent, go to the schedulerxAgent/bin directory and run stop.sh.

   • To view logs, check the file at ${user.home}/logs/schedulerx/worker.log.

     The ${user.home} variable is the home directory of the user who started the agent. You can identify this user by running the ps aux | grep java command. For example, if the process is started by the admin user, the log path is /home/admin/logs/schedulerx/worker.log. If it is started by the root user, the path is /root/logs/schedulerx/worker.log.

   The message Schedulerx Worker started in the output indicates a successful deployment.

   Note

   If you use a Java Development Kit (JDK) version 14 or later, you must remove the -XX:+UseConcMarkSweepGC option from the schedulerxAgent/bin/start.sh file in the agent installation directory.

   
   JAVA_OPTS="-Dproject.name=schedulerxAgent -XX:+UseConcMarkSweepGC -XX:+HeapDumpOxxx -XX:HeapDumpPath=${WORKER_LOG_DIR} -XX:+xxx lysis"
   if [ -n "${SCHEDULERX_STARTER_MODE}" ]; then
       JAVA_OPTS="${JAVA_OPTS} -Dschedulerx.stater.mode=${SCHEDULERX_STARTER_MODE}"

Deploy with a Docker image

1. Select an image based on your network environment and CPU architecture.

   Network	x86_64	arm64
   China (Hangzhou) (Public network)	schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64	schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-arm64
   Singapore (Public network)	schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-amd64	schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-arm64

2. Deploy the agent.

   Parameter description:

   docker run  --env SCHEDULERX_ENDPOINT="addr-hz-internal.edas.aliyun.com" \
      --env SCHEDULERX_NAMESPACE="fd2965c4-****-****-af52-bb62aa4*****" \
      --env SCHEDULERX_GROUPID="hxm.test" \
      --env SCHEDULERX_APPKEY="1234*****" \
      -d {image id}

   • SCHEDULERX_ENDPOINT: The endpoint.

   • SCHEDULERX_NAMESPACE: The namespace ID. You can find it on the Namespace or page in the console.

   • SCHEDULERX_GROUPID is the groupId and SCHEDULERX_APPKEY is the application key. You can obtain them from the Application Management page in the console.

   After the deployment starts, you can run the docker logs command to view the logs.

Deploy with a Kubernetes deployment

If you use Kubernetes, the easiest way to deploy the agent is to apply the schedulerx-agent.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: schedulerx-agent
  labels:
    app: schedulerx-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: schedulerx-agent
  template:
    metadata:
      labels:
        app: schedulerx-agent
    spec:
      containers:
      - name: schedulerx-agent
        image: schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64
        imagePullPolicy: Always
        resources:
          requests:
            cpu: 500m
        env:
          - name: "SCHEDULERX_ENDPOINT"
            value: "${SCHEDULERX_ENDPOINT}"
          - name: "SCHEDULERX_NAMESPACE"
            value: "${SCHEDULERX_NAMESPACE}"
          - name: "SCHEDULERX_GROUPID"
            value: "${SCHEDULERX_GROUPID}"
          - name: "SCHEDULERX_APPKEY"
            value: "${SCHEDULERX_APPKEY}"
          - name: "SCHEDULERX_STARTER_MODE"
            value: "pod"
        livenessProbe: 
          exec: 
            command: ["/bin/bash","/root/health.sh"]
          timeoutSeconds: 30
          initialDelaySeconds: 30