Schedule Dify workflows using XXL-JOB

Background

A Dify workflow can be used in a wide range of business scenarios that require scheduled execution. For example:

Risk monitoring: Scan risk data every minute, analyze potential risk events using a large language model (LLM), and send timely alerts.
Data analysis: Fetch financial data daily, analyze it using an LLM, and provide investment recommendations to investors.
Content generation: Automatically summarize daily work and generate reports.

Important

Dify does not natively support scheduled execution. You can use the XXL-JOB Version to schedule workflows and monitor their status.

Core features

Using XXL-JOB to schedule a Dify workflow provides the following key features and benefits:

Dify jobs: Supports scheduling of self-hosted public Dify workflows and Alibaba Cloud-hosted Dify workflows within a VPC.
Flexible scheduling: Supports cron, api, fixed_rate, fixed_delay, and one_time scheduling types, as well as custom time zones and calendars.
Enterprise-grade alerting and monitoring.
- Flexible alert policies: Supports job-level failure alerts, timeout alerts, and success notifications, as well as threshold-based alerts at the instance and application levels.
- Multiple notification channels: Supports SMS, phone calls, webhooks, and email.
Enterprise-grade observability.
- Scheduling dashboard: View instance-level and application-level scheduling metrics, including scheduling, success, and failure curves.
- Execution history: Records the history of each Dify workflow execution, including status, basic information, inputs and outputs, duration, and token consumption.
- Scheduling events: Records all events for each Dify workflow scheduling, including workflow-related and node-related events.
- Node tracking: Traces a single execution of a Dify workflow, showing the status of all nodes. Supports drilling down into loops, iterations, and conditional branches.

Prerequisites

You have deployed the Dify service. For more information, see Deploy ACK-Dify service.
You have created an XXL-JOB instance (engine version 2.2.0 or later) and an XXL-JOB application.

Procedure

1. (Optional) Configure an internal API server domain

Log on to the Container Service for Kubernetes (ACK) console.
In the left-side navigation pane, click Clusters, and then click the name of the cluster where the Dify service is deployed.
In the left-side navigation pane, choose Network > Service, and then click Create.
Set Service Type to LoadBalancer and Access Method to Internal Access. For Backend, add a label with the key component and the value proxy. Configure other parameters and click OK.

Set Load balancer type to Network Load Balancer (NLB). Select Create resource and enter ack-dify-apiserver as the name.

Set External traffic policy to Local. The service association labels also include app.kubernetes.io/instance=ack-dify and app.kubernetes.io/name=ack-dify. In the port mapping section, set Name to http-dify, Service Port to 80, Container Port to dify, and Protocol to TCP.
Creating the service generates the internal API server address.

On the Services page of the ACK console, find the service and view the External IP column to get the address (NLB domain name).

2. Create a Dify workflow job

Log on to the Dify console. In the upper-right corner, click Studio, and then select Create from Template to create a workflow.
Log on to the MSE XXL-JOB console, and select a region in the top navigation bar.
In the left navigation bar, select Task Scheduling > XXL-JOB Edition.
Click the target instance to open its details page. In the left-side navigation pane, click Task Management, and then click Add Task. For Task Type, select Dify Workflow. For more information about creating jobs, see Task management.
1. Endpoint: The API server for the Dify workflow. Log on to the Dify console and find the API Server address in the upper-right corner of the API Access page. If you are using a self-hosted Dify service on Alibaba Cloud, we recommend changing the domain to the internal endpoint configured in Step 1.
2. API key: The API key for the Dify workflow. Each workflow has a different key. On the API Access page, click API key in the upper-right corner to get it.
3. Input: The input for the workflow, which corresponds to the value of Body in the inputs. The value must be in JSON format.
```
curl -X POST 'http://8.xxx.247/v1/workflows/run' \
--header 'Authorization: Bearer {api_key}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": {},
    "response_mode": "streaming",
    "user": "abc-123"
}'
```
  The following is an example of the inputs value:
```
{"input_text": "what is your name"}
```

3. Test and verify

In the Actions column for the job you created, click Run once to test it manually.
Click Scheduling Records under Actions > More for the target job to view the job execution records.

The execution list displays information for each execution record, such as the Job execution ID, Job ID/name, Application, Start/end time, Duration, Tokens, and Status.
In the Actions column for the job execution, click Details to view the Basic Information, Input and Output, and node tracking details for the Dify workflow.
- Basic information.
  
  This example shows the execution details of a scheduled job (iterator test). The Job ID is 3, the Scheduling type is scheduled, and the Routing policy is round-robin. The Data time and Scheduling time are both 2025-05-16 14:25:00. The execution ran from Start time 2025-05-16 14:25:00 to End time 2025-05-16 14:25:25, for a Duration of 25.568 s. It consumed 2.403K Tokens and ran on the Scheduling node 10.3.1.2.
- Input and output.
  
  In the Scheduling Details pop-up window, the JSON content in the Input area is {"input_text": "what is your name"}, and the JSON content in the Output area is {"final": "What is your name?"}. This indicates that the scheduling job successfully translated the English input into a Chinese output.
- The Tracking view allows you to drill down into the nodes of workflows that include iterators or loop branches.
  
  After you expand the Iteration node, you can view the input and output details of each iteration item. For example, the input is {"iterator_selector": ["what is your name"]} and the output is {"output": ["What is your name?"]}. Each node displays the execution status (succeeded) and the time consumed.
  
  Drilling down reveals the execution details of the internal sub-nodes. In this example, this includes four LLM nodes: Identify Terms, 1st Translation, Problems, and 2nd Translation. All executed successfully, showing the token consumption and duration for each node.