HTTP tasks

Schedule and run tasks over HTTP or HTTPS without integrating a language-specific SDK. HTTP tasks let you connect legacy services, multi-language microservices, and AI agent workflows to the MSE scheduling center through standard HTTP endpoints.

Zero code changes: Expose an HTTP endpoint from any existing service and the scheduling center calls it on schedule. No SDK required.
Language-agnostic: Java, Python, Go, Node.js -- any framework that serves HTTP works the same way.
AI agent and workflow support: Trigger AI agent pipelines and workflow orchestrations through standard HTTP calls.

Two connection types are available:

Kubernetes service: Routes requests directly to pod instances discovered through Kubernetes service endpoints. All tasks in the application share the pod resources associated with the service.
Gateway (non-Kubernetes): Routes requests through a gateway to domain names configured at the application level.

Prerequisites

If the current engine version is earlier than 2.3.0, upgrade it to 2.3.0 or later.

Important

The instance restarts during the upgrade.

Choose a connection type

Dimension	Kubernetes service	Gateway (non-Kubernetes)
Connection pattern	Directly accesses backend pod instances	Routes through a unified gateway
Service discovery	Actively listens for Endpoint changes and dynamically maintains the pod list	Does not support service discovery and is unaware of the backend node status
Target granularity	Schedules to specific pod IP addresses and ports for point-to-point calls	Gateway handles internal load balancing; the scheduling center does not see backend nodes
Routing policy	Single-instance routing and broadcast sharding	Does not support routing policies; uses the gateway's own routing policy
Broadcast sharding	Natively supported for large-scale parallel batch processing	Not supported; all tasks run as single-point executions, and cross-node collaborative processing is not possible
Health management	Syncs Endpoint statuses in real time; automatically removes pods that are not ready or have been terminated	Relies on manual maintenance of domain name availability
Scalability	Adapts to cloud-native environments with automatic scaling and rolling deployments	Better suited for traditional architectures or heterogeneous system integration; easy to connect with existing gateway systems
Best for	Containerized microservices, sharded batch processing jobs, high-concurrency scheduled tasks	Non-container environments, hybrid deployment architectures, legacy system integration, enterprise API scheduling with unified gateway administration

How connection types work

Kubernetes service

Bind a Kubernetes service to an application, and the scheduling center discovers and tracks all pod instances behind that service. Each task only needs an HTTP request path -- the scheduling center selects a target pod based on the routing policy and sends the request directly.

A service discovery thread monitors pod scaling events and health status changes in real time. When pods scale up, scale down, or become unhealthy, the scheduling center automatically syncs the latest pod information with the pod manager.

The pod manager maintains network addresses and health statuses for all pods. Two routing modes are available:

Single-instance routing: Sends the request to one selected pod.
Broadcast sharding: Sends a request to every pod in the service. The scheduling center injects sharding context -- shard index and total shard count -- into the HTTP request headers, so each pod can process its assigned data partition.

Task execution results are recorded and pushed to Simple Log Service (SLS) for viewing and analysis.

Gateway (non-Kubernetes)

Bind one or more HTTP domain names to an application. All tasks within that application share these domain name configurations.

Each task only needs a request path. The scheduling center selects a domain name based on the routing policy and sends the request to the gateway, which handles authentication, traffic shaping, and load balancing before forwarding to the target service.

The domain name manager supports:

Adding, deleting, and modifying domain names
Tag-based routing to distribute traffic by business tags
Multiple routing policies for different scheduling and load balancing needs

Task execution results are synced to Simple Log Service (SLS).

Set up an HTTP task

Log on to the MSE XXL-JOB console and select a region from the top menu bar.
In the left-side navigation pane, choose Task Scheduling > XXL-JOB Version.
Click your target instance. In the left-side navigation pane, choose Application Management, and then click CreateApplication. Set app type to HTTPApplication, and then click OK.
Connect an HTTP executor to the application. For details, see Connect an HTTP application to an executor.
In the left-side navigation pane, choose Task Management, and then click Create Task. Set Associated Application to the application you created in step 3, and set Job Type to HTTP.

Configure the HTTP request settings.

Parameter	Description
Domain name	The destination domain name for the request.
Path	The HTTP request path on the target service.
Request method	The HTTP method for the request.
Timeout	The maximum time to wait for a response.

Configure request parameters as needed.
Parameter Description
Header Custom HTTP headers to include in the request.
Query Query string parameters appended to the URL.
Body The request body content.
Configure the response definition to determine how the scheduling center parses task results. Three parsing types are available: response code, response body, and response Body.

Parameter	Description
Header	Custom HTTP headers to include in the request.
Query	Query string parameters appended to the URL.
Body	The request body content.

Configure the retry policy.

Parameter	Description
Number of retries	How many times to retry after a failure. Set to `0` to disable retries.
Retry interval	The time to wait between retry attempts.
Retry domain name and interface path (optional)	If set, retry requests are sent to this alternate endpoint instead of the original.

Implement an HTTP endpoint

The scheduling center sends an HTTP request to your service endpoint on each trigger. Your endpoint receives system headers that identify the task and execution context, and returns a response that the scheduling center parses to determine success or failure.

The following examples show a minimal endpoint that reads the scheduling headers and returns a JSON response.

Python (Flask)

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/task/run", methods=["POST"])
def run_task():
    job_id = request.headers.get("schedulerx-jobId")
    job_name = request.headers.get("schedulerx-jobName")
    attempt = request.headers.get("schedulerx-attempt")

    # Add your task logic here
    result = {"status": "success", "jobId": job_id, "jobName": job_name}

    return jsonify(result), 200

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Java (Spring Boot)

import org.springframework.web.bind.annotation.*;
import org.springframework.http.ResponseEntity;
import java.util.Map;

@RestController
public class TaskController {

    @PostMapping("/task/run")
    public ResponseEntity<Map<String, Object>> runTask(
            @RequestHeader("schedulerx-jobId") String jobId,
            @RequestHeader("schedulerx-jobName") String jobName,
            @RequestHeader("schedulerx-attempt") String attempt) {

        // Add your task logic here
        Map<String, Object> result = Map.of(
            "status", "success",
            "jobId", jobId,
            "jobName", jobName
        );

        return ResponseEntity.ok(result);
    }
}

Go (net/http)

package main

import (
	"encoding/json"
	"net/http"
)

func runTask(w http.ResponseWriter, r *http.Request) {
	jobId := r.Header.Get("schedulerx-jobId")
	jobName := r.Header.Get("schedulerx-jobName")

	// Add your task logic here
	result := map[string]string{
		"status":  "success",
		"jobId":   jobId,
		"jobName": jobName,
	}

	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(result)
}

func main() {
	http.HandleFunc("/task/run", runTask)
	http.ListenAndServe(":8080", nil)
}

Node.js (Express)

const express = require("express");
const app = express();

app.post("/task/run", (req, res) => {
  const jobId = req.headers["schedulerx-jobid"];
  const jobName = req.headers["schedulerx-jobname"];
  const attempt = req.headers["schedulerx-attempt"];

  // Add your task logic here
  res.json({ status: "success", jobId, jobName });
});

app.listen(8080, () => {
  console.log("Task endpoint listening on port 8080");
});

Handle broadcast sharding

When broadcast sharding is enabled, the scheduling center sends a request to every pod and injects sharding context into the headers. Use these headers to partition your workload.

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/task/shard", methods=["POST"])
def shard_task():
    shard_index = int(request.headers.get("schedulerx-shardingIndex", 0))
    shard_total = int(request.headers.get("schedulerx-shardingTotal", 1))

    # Partition your data by shard index
    # For example, process records where record_id % shard_total == shard_index
    result = {
        "status": "success",
        "shardIndex": shard_index,
        "shardTotal": shard_total,
        "message": f"Processed shard {shard_index} of {shard_total}"
    }

    return jsonify(result), 200

HTTP request headers

The scheduling center injects the following system headers into each HTTP task request. Use these headers in your endpoint to identify the task, track execution, and implement sharding logic.

Header	Description	Example
`schedulerx-appId`	Application ID	`12`
`schedulerx-jobId`	Task ID	`35`
`schedulerx-jobName`	Task name	`http-test-job`
`schedulerx-scheduleTimestamp`	Scheduling timestamp (milliseconds)	`1760164985000`
`schedulerx-dataTimestamp`	Data timestamp (milliseconds)	`1760164980000`
`schedulerx-user`	User information	`1344371792`
`schedulerx-maxAttempt`	Maximum number of retries	`3`
`schedulerx-attempt`	Current retry attempt	`1`
`schedulerx-jobExecutionId`	Task execution ID	`1417474640397221891`
`schedulerx-logId`	Shard ID (for broadcast sharding)	`1417474640531439619`
`schedulerx-shardingIndex`	Shard index (for broadcast sharding)	`0`
`schedulerx-shardingTotal`	Total number of shards (for broadcast sharding)	`3`