HTTP tasks run over the HTTP or HTTPS protocol, so MSE XXL-JOB can schedule workloads written in any language on any platform without integrating a software development kit (SDK). Depending on where your workload runs, MSE XXL-JOB reaches the target service either through a Kubernetes Service or through a gateway.
Why use HTTP tasks
There are several reasons to schedule your workloads as HTTP tasks:
Existing workloads — Connect an existing service without modifying it.
Multiple languages — Use one connection method for tasks that are implemented in different languages.
AI agents and workflows — Schedule AI agent and workflow tasks over HTTP.
HTTP task connection methods
MSE XXL-JOB supports two connection methods for HTTP tasks. The method determines whether the MSE XXL-JOB scheduling center addresses individual backend instances directly or delegates routing to a gateway.
Kubernetes environment — Bind a Service in a Kubernetes cluster to the application. Choose this method for containerized services whose instances change as the workload scales.
Non-Kubernetes environment — Bind HTTP domain names to the application and reach the service through a gateway. Choose this method for services that run outside Kubernetes or behind an existing gateway system.
You select a connection method at the application level, so all tasks in the application share it. Review the following comparison before you create an application and a task.
Connection method comparison
The following table compares the two connection methods for HTTP tasks.
| Dimension | Kubernetes environment: connection through a Service | Non-Kubernetes environment: connection through a gateway |
| Connection mode | Directly accesses backend pods based on a Kubernetes Service. | Proxies requests through a unified gateway and forwards them to the target domain name service. |
| Service discovery mechanism | Actively listens for endpoint changes and dynamically maintains the pod list. | Does not support service discovery and is unaware of the status of backend nodes. |
| Execution target granularity | Can schedule tasks to a specific pod IP address and port to implement point-to-point direct calls. | Requests are sent to the gateway, which completes internal load balancing and instance routing. The scheduling center cannot be aware of the actual backend nodes. |
| Routing policy | Supports single-instance routing policies and broadcast sharding. | Does not support routing policies and uses the routing policies of the gateway. |
| Broadcast sharding | Natively supports the broadcast sharding mode, which is suitable for large-scale parallel batch processing scenarios. | Does not support broadcast sharding. All tasks run at a single point, and cross-node collaborative processing cannot be implemented. |
| Health status management | Synchronizes endpoint status in real time and automatically removes pods that are not ready or already terminated. | Relies on you to manually maintain domain name availability. |
| Extensibility and flexibility | Adapts to cloud-native environments and supports scenarios such as elastic scaling and rolling deployment. | More suitable for traditional architectures or heterogeneous system integration. Provides strong compatibility and is easy to connect to an existing gateway system. |
| Common scenarios | Containerized microservices, sharded batch job execution, and scheduled tasks with high concurrency. | Non-container environments, hybrid deployment architectures, legacy system connection, and enterprise-level API scheduling that requires unified gateway governance. |
Broadcast sharding requires the Kubernetes connection method. If you connect through a gateway, all tasks run at a single point, and you maintain the availability of the bound domain names yourself.
Kubernetes environment: connection through a Service
In a Kubernetes environment, you bind a Service in a Kubernetes cluster to an application so that HTTP tasks can be scheduled. After the binding is complete, all tasks in the application share the pod resources associated with the Service.
After you configure a Kubernetes Service in an application, the scheduling center automatically obtains information about the related pods. When you configure a task, you specify only the HTTP request path, not a specific pod address. When the task is triggered on schedule, the scheduling center sends an HTTP request to the target pod based on the preset routing policy. The task execution result is recorded and pushed to Simple Log Service (SLS) for later viewing and analysis.
The scheduling center uses a service discovery thread to periodically detect pod scale-out and scale-in events and abnormal status, and automatically synchronizes the latest pod information to the pod manager. This achieves dynamic monitoring and status maintenance of the pod cluster.
The pod manager maintains the network addresses and health status of pods and supports multiple task routing policies. In addition to single-instance routing, a broadcast sharding routing mode is provided. When a task is triggered once in this mode, the request is broadcast to all pods under the target service. The scheduling center injects sharding context, including the shard index and the total number of shards, into the HTTP request header. This lets you implement fine-grained logic control and data partitioning during task execution. For the names of the injected headers, see HTTP task request headers.
Non-Kubernetes environment: connection through a gateway
In a non-Kubernetes environment, you bind target HTTP domain names to an application to achieve unified domain name management and routing policy control at the application level. After multiple domain names are bound to an application, all tasks in the application share these domain name configurations.
The scheduling center maintains the domain name information through the domain name manager. When you create a task, you configure only the request path, not the domain name. When the task is triggered, the scheduling center automatically selects an appropriate domain name based on the preset routing policy and sends the request to the gateway. After the gateway completes authentication, traffic control, and load balancing, it forwards the request to the target service. The execution result is finally synchronized to Simple Log Service (SLS).
The domain name manager maintains domain name information. It supports adding, deleting, and modifying domain names and configuring tag-based routing. It can distribute traffic based on business tags and supports multiple routing policies to meet different scheduling and load balancing requirements.
Prerequisites
The engine version of your instance is 2.3.0 or later. If the current engine version is earlier than 2.3.0, upgrade it to 2.3.0 or later.
The instance restarts during the upgrade.
Connection procedure
Log on to the MSE XXL-JOB console and select a region in the top menu bar.
In the left-side navigation pane, choose Task Scheduling > XXL-JOB Version.
Click the target instance. In the left-side navigation pane, click Application Management and then click Create Application. Set the application type to HTTP application and click OK.
Connect an HTTP executor. For instructions, see Connect an HTTP application to an executor.
In the left-side navigation pane, click Task Management, and then click Create Task. For the associated application, select the application that you created in step 3. For the task type, select HTTP. You can also specify information such as Task description, Priority, Task weight, and Input parameters (JSON format is supported) based on your business requirements.
Configure the request domain name, path, request method, and timeout period for the HTTP task. For Request method, select an HTTP method such as GET or POST. The Timeout is measured in seconds.
Configure the request parameters for the HTTP task. Specify Header, Query, and Body based on your requirements. Body supports three data formats: form-data, x-www-form-urlencoded, and raw-json. To add multiple groups of parameters, click + Add.
Configure the response definition of the HTTP task, including the success response parsing mode and the success result parsing mode. The parsing modes provide three parsing types: response code, response content, and response body. For example, if you select HTTP response code as the parsing mode, enter the expected response code, such as 200, in the HTTP response code field as the success criterion.
Configure failure retries for the HTTP task, including the number of retries and the retry interval. A number of retries greater than 0 means that the task is retried after it fails. In this case, you can also configure a retry domain name and an interface path so that failed tasks are sent to that domain name and path. The retry interval is measured in seconds.
HTTP task request headers
When MSE XXL-JOB runs an HTTP task, it adds the following system parameters to the request header:
| Name | Description | Example value |
| schedulerx-appId | Application ID | 12 |
| schedulerx-jobId | Task ID | 35 |
| schedulerx-jobName | Task name | http-test-job |
| schedulerx-scheduleTimestamp | Scheduling timestamp (in milliseconds) | 1760164985000 |
| schedulerx-dataTimestamp | Data timestamp (in milliseconds) | 1760164980000 |
| schedulerx-user | User information | 1344371792 |
| schedulerx-maxAttempt | Maximum number of retries | 3 |
| schedulerx-attempt | Current number of retries | 1 |
| schedulerx-jobExecutionId | Task execution ID | 1417474640397221891 |
| schedulerx-logId | Shard ID (broadcast sharding) | 1417474640531439619 |
| schedulerx-shardingIndex | Shard index (broadcast sharding) | 0 |
| schedulerx-shardingTotal | Total number of shards (broadcast sharding) | 3 |