Microservices Engine:Deploy SchedulerX in a Kubernetes cluster

Edit scripts without rebuilding images

Standard Kubernetes Jobs require you to package scripts into container images and configure commands in YAML. Every script change means a new image build and redeployment.

With SchedulerX, edit scripts directly in the console. SchedulerX runs them in Pods automatically, and changes take effect on the next scheduled run. No image builds or YAML updates are needed.

Supported languages: Shell, Python, PHP, and Node.js.

apiVersion: batch/v1
kind: Job
metadata:
  name: hello
spec:
  template:
    spec:
      containers:
      - name: hello
        image: registry.cn-hangzhou.aliyuncs.com/test/hello:1.0.0
        command: ["sh",  "/root/hello.sh"]
      restartPolicy: Never
  backoffLimit: 4

Drag-and-drop workflow orchestration

SchedulerX provides a visual editor for orchestrating Kubernetes Jobs with drag-and-drop operations. Running workflows are displayed as directed acyclic graphs (DAGs), making it straightforward to identify blocked Jobs and troubleshoot issues.

# Diamond workflow:
#   A
#  / \
# B   C
#  \ /
#   D
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-
spec:
  entrypoint: diamond
  templates:
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: [{name: message, value: A}]
      - name: B
        depends: "A"
        template: echo
        arguments:
          parameters: [{name: message, value: B}]
      - name: C
        depends: "A"
        template: echo
        arguments:
          parameters: [{name: message, value: C}]
      - name: D
        depends: "B && C"
        template: echo
        arguments:
          parameters: [{name: message, value: D}]

  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: alpine:3.7
      command: [echo, "{{inputs.parameters.message}}"]

Argo Workflows, a widely adopted solution, requires workflow definitions in YAML:

Built-in monitoring and alerting

SchedulerX monitors every Pod and Job it schedules:

• Alert channels: text messages, phone calls, emails, and webhook URLs for DingTalk, WeCom, and Lark bots.

• Alert triggers: Job execution failures and timeouts.

Automatic log collection

SchedulerX collects Pod and Job logs automatically. When a Pod fails, check the logs directly in the SchedulerX console instead of running kubectl logs.

Real-time monitoring dashboard

The Overview page in the SchedulerX console displays all Jobs and their status in real time.

Mixed deployment of online and offline Jobs

Some periodic Jobs consume significant resources but run infrequently (for example, once per hour or once per day). Instead of running them in the same process as your business application, create a separate Pod for the Job. This isolates resource-intensive batch work from your online services. SchedulerX supports scheduling both Java and Kubernetes Job types in a single environment.

Step 1: Configure a service account

The service account provides an identity for validation and authorization when SchedulerX schedules Kubernetes Jobs. By default, the service account of the namespace to which the Job belongs is used.

Run the service account YAML once per namespace. Choose the scope that fits your use case:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: schedulerx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: schedulerx-role
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["create","delete","get","list","patch","update"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["watch"]
  - apiGroups: ["batch"]
    resources: ["jobs","cronjobs"]
    verbs: ["create","delete","get","list","patch","update","watch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: schedulerx-binding
subjects:
  - kind: ServiceAccount
    name: schedulerx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: schedulerx-role

kubectl apply -f schedulerx-serviceaccount.yaml -n <your-namespace>

kubectl apply -f schedulerx-clusterrole.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: schedulerx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: schedulerx-cluster-role
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["create","delete","get","list","patch","update"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["watch"]
  - apiGroups: ["batch"]
    resources: ["jobs","cronjobs"]
    verbs: ["create","delete","get","list","patch","update","watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: schedulerx-cluster-binding
subjects:
  - kind: ServiceAccount
    name: schedulerx
    namespace: <NAMESPACE1>
  - kind: ServiceAccount
    name: schedulerx
    namespace: <NAMESPACE2>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: schedulerx-cluster-role

Step 2: Install the SchedulerX agent

Create a file named schedulerx-agent.yaml with the following content. Replace the environment variable values with your application's access parameters (see Agent environment variables).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: schedulerx-agent
  labels:
    app: schedulerx-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: schedulerx-agent
  template:
    metadata:
      labels:
        app: schedulerx-agent
    spec:
      serviceAccountName: schedulerx
      containers:
      - name: schedulerx-agent
        image: <agent-image>    # See the agent image reference table
        imagePullPolicy: Always
        resources:
          limits:
            cpu: 500m
          requests:
            cpu: 500m
        env:
          - name: "SCHEDULERX_ENDPOINT"
            value: "<your-endpoint>"
          - name: "SCHEDULERX_NAMESPACE"
            value: "<your-namespace-id>"
          - name: "SCHEDULERX_GROUPID"
            value: "<your-app-id>"
          - name: "SCHEDULERX_APPKEY"
            value: "<your-app-key>"
          - name: "SCHEDULERX_STARTER_MODE"
            value: "pod"
        livenessProbe:
          exec:
            command: ["/bin/bash","/root/health.sh"]
          timeoutSeconds: 30
          initialDelaySeconds: 30

Apply the Deployment:

kubectl apply -f schedulerx-agent.yaml -n <your-namespace>

Verify the deployment

1. Check that the agent Pod is running: Expected output:

      kubectl get pods -l app=schedulerx-agent -n <your-namespace>

      NAME                               READY   STATUS    RESTARTS   AGE
      schedulerx-agent-xxxxxxxxx-xxxxx   1/1     Running   0          1m

2. Confirm the agent is connected by checking the Application Management page in the SchedulerX console. An available instance indicates a successful connection.

Step 1: Download the Helm chart

wget https://schedulerx2.oss-cn-hangzhou.aliyuncs.com/helm/schedulerxchart-2.0.0.tgz

Step 2: Install the Helm chart

1. Get your application's access parameters from the SchedulerX console:

   1. Log on to the MSE SchedulerX console.

   2. In the top navigation bar, select a region.

   3. In the left-side navigation pane, click Application Management.

   4. On the Application Management page, click Access configuration in the Operation column. In the upper-left corner of the Access configuration panel, select k8s from the drop-down list.

2. Run the installation command. Replace the placeholder values with your access parameters:

   Note

   The default image in the access configuration is the x86_64 Internet image. Select an image that matches your cluster's architecture and region from theagent image referencetable.

      helm install schedulerxchart schedulerxchart-2.0.0.tgz \
        --set SCHEDULERX_ENDPOINT=<your-endpoint> \
        --set SCHEDULERX_NAMESPACE=<your-namespace-id> \
        --set SCHEDULERX_GROUPID=<your-app-id> \
        --set SCHEDULERX_APPKEY=<your-app-key> \
        --set SCHEDULERX_AGENT_IMAGE=<agent-image>

   

Verify the deployment

1. Check that the agent Pod is running: Expected output:

      kubectl get pods -l app=schedulerx-agent -n <your-namespace>

      NAME                               READY   STATUS    RESTARTS   AGE
      schedulerx-agent-xxxxxxxxx-xxxxx   1/1     Running   0          1m

2. Confirm the agent is connected by checking the Application Management page in the SchedulerX console. An available instance indicates a successful connection.

Connect your application to SchedulerX

1. Integrate the SchedulerX SDK for Java with your Spring Boot application. See Connect a Spring Boot application to SchedulerX.

2. Add the schedulerx-plugin-kubernetes dependency to enable Kubernetes Job scheduling:

   Note

   Replace schedulerx2.version with the latest version. See Release notes.

      <dependency>
        <groupId>com.aliyun.schedulerx</groupId>
        <artifactId>schedulerx2-spring-boot-starter</artifactId>
        <version>${schedulerx2.version}</version>
      </dependency>
      <dependency>
        <groupId>com.aliyun.schedulerx</groupId>
        <artifactId>schedulerx2-plugin-kubernetes</artifactId>
        <version>${schedulerx2-plugin-kubernetes.version}</version>
      </dependency>