Sync DingTalk data to a DTS RAGFlow knowledge base - Data Transmission Service

This topic describes how to sync data from DingTalk to a DTS RAGFlow knowledge base. DTS supports two integration methods: the DingTalk Drive method (using the open platform API) and the MCP method (using the MCP protocol).

Prerequisites

You have created a RAGFlow knowledge base in DTS and configured an IP whitelist.

Supported data types

MCP method

DTS RAGFlow supports connecting to documents in a DingTalk knowledge base using the MCP protocol. Different document types are processed as follows:

Document type	File extension	Processing method
Online document	`adoc`	Fetches Markdown content using DingTalk Docs MCP and uploads it to RAGFlow in `.md` format.
Spreadsheet	`axls`	Fetches data through DingTalk Sheets MCP, exports it to `.csv` format, and uploads it to RAGFlow. If the `dingDingSheetMcpEndpoint` parameter is not configured, files of this type are skipped.
Regular attachment	Others	Downloads the original file through the Docs MCP and uploads it to RAGFlow.
Presentation, mind map, whiteboard, multi-dimensional table	`appt`, `amind`, `adraw`, `able`	Unsupported. Marked as `unsupported-source-type` and written to the checkpoint.

DingTalk Drive method

DTS RAGFlow supports connecting to files in DingTalk Drive. The supported file types include:

Word, Excel, and PowerPoint files created in DingTalk Drive.
Files uploaded to DingTalk Drive, such as TXT, PDF, and Markdown files.

Before you begin

DTS supports two methods for connecting to a DingTalk data source. Choose one based on your requirements:

MCP method (Recommended): Connects using the MCP protocol. This method requires no application creation, has a simpler configuration, and offers better performance.
DingTalk Drive method: Connects using the DingTalk open platform API. This method supports DingTalk Drive files and requires creating a DingTalk application and configuring its permissions.

MCP method

Step 1: Obtain the MCP server endpoint

DingTalk online documents and spreadsheets each correspond to a separate MCP server. Enable each service and obtain its endpoint.

Go to the DingTalk MCP Marketplace and enable the following MCP services:
- DingTalk Docs MCP (Required): Lets you read document content and download files. Enable this service at: DingTalk Docs MCP.
- DingTalk Sheets MCP (Optional): Lets you read spreadsheet data. If you do not enable this service, spreadsheets in the knowledge base are skipped. Enable this service at: DingTalk Sheets MCP.
On the service details page in the MCP Marketplace, copy the Streamable-HTTP Endpoint.
```
https://mcp-gw.dingtalk.com/server/<server_id>?key=<key>
```

Important

The key parameter in the endpoint is an access credential. Keep it secure and do not commit it to Git repositories or expose it in public channels.

Step 2: Obtain the WorkSpaceId

The WorkSpaceId is the unique identifier of a DingTalk knowledge base. Obtain it from the URL of the knowledge base homepage.

In your browser, open the homepage of the target DingTalk knowledge base.
```
https://alidocs.dingtalk.com/i/space/{WorkSpaceId}
```
Copy the {WorkSpaceId} part from the URL. For example, if the URL is https://alidocs.dingtalk.com/i/space/nb9XJj73Oxxxx, the WorkSpaceId is nb9XJj73Oxxxx.

Step 3: Obtain RootFolderIds (Optional)

If you only need to sync specific subdirectories instead of the entire knowledge base, you must obtain the node IDs of the target subdirectories. If not configured, KBSync recursively traverses all nodes from the root directory of the knowledge base.

In the DingTalk knowledge base web interface, open the target subdirectory.
```
https://alidocs.dingtalk.com/i/nodes/{nodeId}
```
Copy the {nodeId} part from the URL. To sync multiple subdirectories, separate the node IDs with commas.

DingTalk Drive method

Step 1: Create a DingTalk application

Log on to the DingTalk Open Platform, and go to the developer backend.
Create an application.
1. In the upper-right corner of the page, click My Console.
2. Click the App Development tab. In the left navigation pane, choose Internal Enterprise Applications > DingTalk Applications.
3. Click Create App.
4. Enter an App Name and App Description, and then click Save.
5. To the right of the application, click > App Details.
6. In the left navigation pane, choose Basic Information> Credentials & Basic Info.
7. In the App Credentials section, record the Client ID and Client Secret.
Configure permissions. In the left navigation pane, choose Development Configuration > Permission Management and select the following permissions:
- Read permission for application disk space
- Read permission for member information
- Read permission for enterprise storage files
- Read permission for enterprise storage file download information
Click Bulk Application.

Step 2: Obtain an employee UserID

Log on to the DingTalk Admin Console using a DingTalk account with enterprise or team administrator permissions.
In the left navigation pane, choose Contacts > Member Management.
Select a department and record the employee UserID of the target DingTalk account.
Important
The DingTalk account must have management permissions for the target files in DingTalk Drive, allowing it to view and download them.

Procedure

Obtain the KBSync file.

Note
You can join the DingTalk group (ID: 79690034672) and contact technical support to obtain the KBSync file.
Set up the KBSync runtime environment.
Note
The KBSync program requires a Linux environment with network access to both the DingTalk MCP service and the RAGFlow service.

Prepare the config file.

Create a Linux system file named config.

Copy the following code to the config file.

MCP method

whiteList=
blackList=
sinkType=RagFlowV2
sourceType=DingDingV3
ragflowUrl=http://XX.XX.XX.XX
ragflowApiKey=Bearer ragflow-Rh******BiYWY3MDAxNj
ragflowDatasetId=b2af7c4c******dbae
dingDingMcpEndpoint=https://mcp-gw.dingtalk.com/server/<server_id>?key=<key>
dingDingSheetMcpEndpoint=https://mcp-gw.dingtalk.com/server/<server_id>?key=<key>
dingDingWorkSpaceId=nb9XJj73Oxxxx
dingDingRootFolderIds=

DingTalk Drive method

whiteList=
blackList=
sinkType=RagFlowV2
sourceType=DingDingV2
ragflowUrl=http://XX.XX.XX.XX
ragflowApiKey=Bearer ragflow-Rh******BiYWY3MDAxNj
ragflowDatasetId=b2af7c4c******dbae
dingDingClientId=dingz******k66u
dingDingClientSecret=sa4vI******xrJT-OmEf_fgF0GpbKi4L******tzugFn-_
dingDingUserId=012******815010

Replace the parameter values in the config file.

Important

For parameters that do not require configuration, leave their values empty.
The blackList parameter takes precedence over the whiteList parameter.

Parameter	Required	Description	How to obtain
`whiteList`	No	The paths of documents in the knowledge base to include (whitelist) or exclude (blacklist). Note Regular expressions are supported. Separate multiple paths with spaces.	Obtain the paths from your DingTalk knowledge base.
`blackList`	No		Obtain the paths from your DingTalk knowledge base.
`sinkType`	Yes	The type of the sink.	Set to `RagFlowV2`.
`sourceType`	Yes	The type of the source.	For the DingTalk Drive method, set to `DingDingV2`. For the MCP method, set to `DingDingV3`.
`ragflowUrl`	Yes	The URL of your RAGFlow instance (the API server).	Obtain the API server URL of the RAGFlow knowledge base.
`ragflowApiKey`	Yes	The API key for the RAGFlow knowledge base.	Obtain the API key for the RAGFlow knowledge base.
`ragflowDatasetId`	Yes	The ID of the RAGFlow knowledge base.	Obtain the ID of the RAGFlow knowledge base.
`dingDingClientId`	Yes (DingTalk Drive method)	The Client ID of the DingTalk application. Required only for the DingTalk Drive method.	Obtained in Before you begin - DingTalk Drive method.
`dingDingClientSecret`	Yes (DingTalk Drive method)	The Client Secret of the DingTalk application. Required only for the DingTalk Drive method.	Obtained in Before you begin - DingTalk Drive method.
`dingDingUserId`	Yes (DingTalk Drive method)	The employee UserID of the DingTalk account used to export DingTalk Drive files. Required only for the DingTalk Drive method.	Obtained in Before you begin - DingTalk Drive method.
`dingDingMcpEndpoint`	Yes (MCP method)	The endpoint URL for DingTalk Docs MCP, used to read online document content and download attachments. Required only for the MCP method.	Obtain the MCP server endpoint.
`dingDingSheetMcpEndpoint`	No (MCP method)	The endpoint URL for DingTalk Sheets MCP, used to read spreadsheet data. Required only for the MCP method. If not configured, files of the `axls` type are skipped and written to the checkpoint.	Obtain the MCP server endpoint.
`dingDingWorkSpaceId`	Yes (MCP method)	The ID of the target DingTalk knowledge base. Required only for the MCP method.	Obtain the WorkSpaceId.
`dingDingRootFolderIds`	No (MCP method)	The IDs of the root directories to sync. Separate multiple IDs with commas. If not configured, KBSync syncs the entire knowledge base.	Obtain RootFolderIds.

Place the KBSync file and the config configuration file in the same directory in a Linux environment.
In your Linux environment, run the ./KBSync --config config command to start the KBSync program.

MCP method

Wait for the KBSync program to sync data. If the output resembles the following, the connection is successful and the sync process has started.

./KBSync --config config
 
INFO config SourceType=DingDingV3, SinkType=RagFlowV2
INFO config whiteList=, blackList=
INFO config ragflowUrl=http://XX.XX.XX.XX, ragflowApiKey=Bearer ragflow-RhMjc0NjFhNTZmNTExZjBiYWY****
INFO config ragflowDatasetId=b2af7c4c56f511f0a3d700163e****
INFO config dingDingMcpEndpoint=https://mcp-gw.dingtalk.com/server/<server_id>?key=****
INFO config dingDingSheetMcpEndpoint=https://mcp-gw.dingtalk.com/server/<server_id>?key=****
INFO config dingDingWorkSpaceId=nb9XJj73Ox37lXyA
INFO Verifying RAGFlow connection...
INFO Attempting to list datasets to validate the connection...
INFO Successfully found matching dataset: Name='test', ID='b2af7c4c56f511f0a3d700163e****'
INFO RAGFlow connection verified successfully.
INFO Starting DingTalk MCP connection...
INFO MCP connection established successfully.
INFO Listing workspace nodes...
INFO Found 42 nodes in workspace.
INFO Syncing documents...

Note

The KBSync program runs continuously and syncs data periodically. You can manually stop or restart the program as needed.

DingTalk Drive method

Run the KBSync program.
Note
The KBSync program periodically syncs data from the target documents. You can configure the sync interval, or manually stop and restart the KBSync program as needed.

Enter 2 and press Enter.

Wait for the KBSync program to sync the data. If the output resembles the following, the program is running correctly.

./KBSync --config config

INFO config SourceType=DingDingV2, SinkType=RagFlowV2
INFO config whiteList=, blackList=
INFO config ragflowUrl=http:///XX.XX.XX.XX, ragflowApiKey=Bearer ragflow-RhMjc0NjFhNTZmNTExZjBiYWY****
INFO config ragflowDatasetId=b2af7c4c56f511f0a3d700163e****
INFO config dingDingClientId=ding9nyefxviedv****, dingDingClientSecret=Kh3qQZafV9vZZIwvP****-V7f3tkmYmcYibwT7zWNXrvOm4UbETOhqEt****
INFO config dingDingUserId=012713339****
NFO Verifying RAGFlow connection...
INFO Attempting to list datasets to validate the connection...
INFO Successfully found matching dataset: Name='test', ID='b2af7c4c56f511f0a3d700163e****'
INFO RAGFlow connection verified successfully.
INFO Starting DingTalk authentication process...
INFO Waiting for the initial app access token...
INFO App access token refresh loop started.
INFO Attempting to refresh the app access token...
INFO {'headers': {'server': 'DingTalk/1.0.0', 'date': 'Tue, 29 Jul 2025 09:14:19 GMT', 'content-type': 'application/json;charset=utf-8', 'transfer-encoding': 'chunked', 'connection': 'keep-alive', 'access-control-allow-origin': '*', 'access-control-expose-headers': '*', 'x-acs-request-id': 'AD15D9B5-C47D-7C06-BCE7-03AA76A****', 'x-acs-trace-id': 'ad2c67867d7153507b9a74e399****', 'access-control-allow-headers': 'X-Requested-With, X-Sequence, _aop_secret, _aop_signature, x-acs-dingtalk-access-token', 'content-encoding': 'gzip'}, 'statusCode': 200, 'body': {'accessToken': '38a7d120903c35a7aa2e866dca****', 'expireIn': 7200}}
INFO set dingding access token: 38a7d120903c35a7aa2e866dca****
INFO set dingding access token expires in: 7200
INFO App access token refreshed successfully. Next refresh scheduled in 6600 seconds.
INFO Waiting 6600 seconds until the next token refresh.
INFO Initial app access token acquired.
INFO Fetching user UnionID...
INFO set dingding union id: XYesTh9u****
INFO Successfully fetched UnionID.

Appendix

Obtain the API server URL

Log on to the RAGFlow page.
In the navigation pane on the left, click API.
Obtain the API server.

Obtain the API key

Log on to the RAGFlow page.
In the navigation pane on the left, click API.
To the right of RAGFlow API, click API key.
In the API key dialog box, click Create New Key.
Click and copy the token.

Obtain the knowledge base ID

Log on to the RAGFlow page.
On the Knowledge Base page, click the target knowledge base.
In the URL, find and copy the knowledge base ID.

Note
The ID is the value that follows id=.