Microservices Engine:Connect to SchedulerX using the agent (script or HTTP jobs)

Prerequisites

• (Optional) A namespace is created. For more information, see Create a namespace.

• A resource is created. For more information, see Create a resource.

Scenarios

Common scenarios for connecting to SchedulerX using the agent include the following:

• Use a Shell script to periodically clear logs on all machines or start a service.

• Use a Python script to periodically clear historical data from a database.

• Use an HTTP interface to periodically scan orders.

Environment requirements

The runtime environment must be Java Runtime Environment (JRE) 1.8 or later.

Manually deploy the agent

1. Download and decompress the agent.

   If your machine has a public IP address, you can run the following command to download the agent:

   Note

   Replace X.X.X with the required version number, such as 1.12.5.

   wget https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-X.X.X.tar

   Version	Download URL	Version guide
   Agent 1.13.2	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.13.2.tar.gz	New features Supports CPU metric collection under CGroup V2. Supports custom thread pool sizes for task execution thread pools. The parameter key is share.pool.size.
   Agent 1.12.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.12.5.tar	New features: Supports getting task system variables using placeholders. For more information, see Obtain system variables of a task using placeholders. Supports custom thread pool sizes for HTTP tasks. Allows script tasks to return the last line of standard output as the execution result.
   Agent 1.11.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.11.5.tar	Fixed issues: Fixes security vulnerabilities in Netty and Logback. Fixes the "too old resource version" issue for Kubernetes (K8s) tasks.
   Agent 1.10.13	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.10.13.tar	New features: Supports deploying multiple agents on a single machine. Supports HTTP agent tasks with responses larger than 1000 bytes. Optimizes logs to reduce historical log storage.
   Agent 1.10.5	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.10.5.tar	New features: Supports graceful shutdown. Simple Log Service supports application group fencing.
   Agent 1.9.8	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.9.8.tar	New features: Supports agent-mode HTTP tasks. Supports returning the last line of the log if a script task fails. Supports K8s tasks.
   Agent 1.7.10	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.7.10.tar.gz	Fixed issues: Fixes the issue where second-level tasks run slowly. Fixes an issue where Simple Log Service could cause an out-of-memory (OOM) error.
   Agent 1.4.2	https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/agent/schedulerxAgent-1.4.2.tar.gz	New feature: Supports Simple Log Service.

   Important

   If the connection times out when you use the wget command to download the agent, enable public network access for your machine or see Upload or download files (Linux).

2. Go to the schedulerxAgent/conf directory and configure the agent.properties file.

   The configuration includes the following parameters:

   endpoint=addr-hz-internal.edas.aliyun.com
   # For public network access, use domainName instead of endpoint.
   # domainName={domainName}
   namespace=fd2965c4-xxxx-xxxx-af52-bb62aa4f19f2
   groupId=hxm.test
   appKey=12******789

   • endpoint: The endpoint. For more information, see Endpoints.

   • namespace: The namespace ID. You can obtain this ID on the Namespace or page.

   • groupId and appKey: The unique ID and key for the application. You can obtain them on the Application Management page.

3. Go to the schedulerxAgent/bin directory and run the start.sh command to start the agent.

   Note

   To set the memory for the agent at startup, you can run start-200m.sh, start-500m.sh, or start-1g.sh.

   After the agent starts, you can perform the following operations as needed:

   • To stop the agent, go to the schedulerxAgent/bin directory and run the stop.sh script.

   • To view logs, check the ${user.home}/logs/schedulerx/worker.log file.

     You can find the user.home path by running the ps aux | grep java command. For example, if the process is started by the admin user, the log path is /home/admin/logs/schedulerx/worker.log. If the process is started by the root user, the log path is /root/logs/schedulerx/worker.log.

   A response that contains Schedulerx Worker started indicates that the agent started successfully.

   Note

   The runtime environment must be Java Development Kit (JDK) 1.8 to 14. If you use a JDK version later than 14, remove the -XX:+UseConcMarkSweepGC parameter from the schedulerxAgent/bin/start.sh file in the agent installation directory.

   

Deploy the agent using a Docker image

1. Select an image based on your network environment and CPU architecture.

   Note

   The following image registry addresses are for public network access.

   Network	x86_64	arm64
   (Public Network) China (Hangzhou)	schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64	schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-arm64
   (Public Network) Singapore	schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-amd64	schedulerx-registry.ap-southeast-1.cr.aliyuncs.com/schedulerx2/agent:latest-arm64

2. Deploy the agent.

   Parameter descriptions:

   docker run  --env SCHEDULERX_ENDPOINT="addr-hz-internal.edas.aliyun.com" \
      --env SCHEDULERX_NAMESPACE="fd2965c4-xxxx-xxxx-af52-bb62aa4f19f2" \
      --env SCHEDULERX_GROUPID="hxm.test" \
      --env SCHEDULERX_APPKEY="12*****789" \
      -d {image id}

   • SCHEDULERX_ENDPOINT: The endpoint.

   • SCHEDULERX_NAMESPACE: The namespace ID. You can obtain this ID from the Namespace or page in the console.

   • SCHEDULERX_GROUPID: The application ID.

   • SCHEDULERX_APPKEY: The application key. You can obtain the application ID and key from the Application Management page or the page in the console.

   After the deployment starts, you can view logs by running the docker logs command.

Deploy a deployment using a K8s image

If you have a Kubernetes environment, you can deploy the agent directly using the schedulerx-agent.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: schedulerx-agent
  labels:
    app: schedulerx-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: schedulerx-agent
  template:
    metadata:
      labels:
        app: schedulerx-agent
    spec:
      containers:
      - name: schedulerx-agent
        image: schedulerx-registry.cn-hangzhou.cr.aliyuncs.com/schedulerx2/agent:latest-amd64
        imagePullPolicy: Always
        resources:
          requests:
            cpu: 500m
        env:
          - name: "SCHEDULERX_ENDPOINT"
            value: "${SCHEDULERX_ENDPOINT}"
          - name: "SCHEDULERX_NAMESPACE"
            value: "${SCHEDULERX_NAMESPACE}"
          - name: "SCHEDULERX_GROUPID"
            value: "${SCHEDULERX_GROUPID}"
          - name: "SCHEDULERX_APPKEY"
            value: "${SCHEDULERX_APPKEY}"
          - name: "SCHEDULERX_STARTER_MODE"
            value: "pod"
        livenessProbe: 
          exec: 
            command: ["/bin/bash","/root/health.sh"]
          timeoutSeconds: 30
          initialDelaySeconds: 30