Transfer data from DingTalk to a DTS RAGFlow knowledge base - Data Transmission Service

Prerequisites

Before you begin, make sure you have:

A RAGFlow knowledge base created in DTS
An IP whitelist configured for the RAGFlow instance

Supported file types

KBSync can sync files from DingTalk Drive and knowledge bases. Supported file types include:

Word, Excel, and PowerPoint files created in DingTalk Drive
Files uploaded to DingTalk Drive, such as TXT, PDF, and Markdown files
All file types in knowledge bases

Set up the DingTalk application

Before running KBSync, create a DingTalk application to get the credentials (Client ID and Client Secret) that KBSync uses to authenticate with DingTalk.

Log on to the DingTalk Open Platform and go to the DingTalk Open PlatformDeveloper Console.
Create an application.
1. In the upper-right corner of the page, click My Background.
2. Click the Application Development tab. In the left-side navigation pane, choose Enterprise Internal Applications > DingTalk Applications.
3. Click Create Application.
4. In the Create Application panel, fill in the Application Name and Application Description fields, then click Save.
To the right of the application, choose > Application Details.
In the left-side navigation pane, choose Basic Information > Credentials and Basic Information.
In the Application Credentials section, record the Client ID and Client Secret.

Procedure

KBSync supports two data sources: DingTalk knowledge bases and DingTalk Drive. The setup steps are identical until you start the program, at which point the flow diverges depending on your source.

Step 1: Configure access permissions

Grant the application the permissions it needs to read files from DingTalk Drive and knowledge bases.

Log on to the DingTalk Open Platform and go to the Developer Console.
Go to the application details page.
1. In the upper-right corner of the page, click My Background.
2. Click the Application Development tab. In the left-side navigation pane, choose Enterprise Internal Applications > DingTalk Applications.
3. Find the application you created in Set up the DingTalk application.
4. To the right of the application, choose > Application Details.
In the left-side navigation pane, choose Development Configuration > Permission Management.
Configure the Permission Scope as needed, then select all of the following permissions:
- Read permission for application space in DingTalk Drive
- Read permission for member information
- Read permission for enterprise storage files
- Read permission for enterprise storage file download information
- Read permission for knowledge bases
- Read permission for knowledge base nodes
Click Batch Request.

Step 2: Get the UserID of a DingTalk account

KBSync uses the UserID to identify which DingTalk account to export files from.

Important

The account must have permissions to manage, view, and download the target files in DingTalk Drive or the knowledge base.

Log on to the DingTalk Admin Console using an account with enterprise or team administrator permissions.DingTalk Admin Console
In the left-side navigation pane, choose Address Book > Member Management.
Select a department.
Record the UserID of the target account.

Step 3: Run KBSync

3.1 Get the KBSync file

Join the DingTalk group (ID: 79690034672) and contact the helpdesk to get the KBSync file.

3.2 Prepare the runtime environment

The KBSync program requires:

A Linux environment with network access to both the DingTalk Open Platform and RAGFlow
Google Chrome installed (required only when syncing from a knowledge base)

3.3 Create the configuration file

Create a file named config in your Linux environment.

Copy the following template into the file:

whiteList=
blackList=
sinkType=RagFlow
sourceType=DingDing
ragflowUrl=http://XX.XX.XX.XX
ragflowApiKey=Bearer ragflow-Rh******BiYWY3MDAxNj
ragflowDatasetId=b2af7c4c******dbae
dingDingClientId=dingz******k66u
dingDingClientSecret=sa4vI******xrJT-OmEf_fgF0GpbKi4L******tzugFn-_
dingDingUserId=012******815010

Replace the placeholder values with your actual values using the table below.

Important

Leave the value blank for any parameter you do not need to configure.
blackList takes higher priority than whiteList.

Parameter	Required	Description	How to get the value
`whiteList`	No	Paths of files to sync (whitelist). Supports regular expressions. Separate multiple paths with spaces.	Get the paths from the DingTalk client.
`blackList`	No	Paths of files to exclude (blacklist). Has higher priority than `whiteList`. Supports regular expressions. Separate multiple paths with spaces.	Get the paths from the DingTalk client.
`sinkType`	Yes	Type of the destination.	Keep as `RagFlow`.
`sourceType`	Yes	Type of the source.	Keep as `DingDing`.
`ragflowUrl`	Yes	The RAGFlow API Server endpoint.	See Get the API Server endpoint.
`ragflowApiKey`	Yes	The API key for the RAGFlow knowledge base. Must start with `Bearer` followed by a space. Example: `Bearer ragflow-RhMjc0NjFhNTZmNTExZjBiYWY****`.	See Get the API key.
`ragflowDatasetId`	Yes	The ID of the RAGFlow knowledge base. The value is the string after `id=` in the knowledge base URL.	See Get the knowledge base ID.
`dingDingClientId`	Yes	The Client ID of the DingTalk application.	Recorded in Set up the DingTalk application.
`dingDingClientSecret`	Yes	The Client Secret of the DingTalk application.	Recorded in Set up the DingTalk application.
`dingDingUserId`	Yes	The UserID of the DingTalk account used to export files.	Recorded in Step 2.

3.4 Start KBSync

Place the KBSync binary and the config file in the same folder.
Run the following command:
```
./KBSync --config config
```
When prompted, select your data source and follow the steps for that source:
- Sync from a DingTalk knowledge base — enter 1
- Sync from DingTalk Drive — enter 2

Sync from a DingTalk knowledge base

Enter 1 and press Enter.
Enter the number that corresponds to the knowledge base containing your target documents, then press Enter.
KBSync opens a page in Google Chrome. Log on to your DingTalk account on that page.
Return to the Linux terminal and press Enter.
Wait for KBSync to complete the sync.

If the sync is running correctly, the output looks similar to:

./KBSync --config config

INFO config SourceType=DingDing, SinkType=RagFlow
INFO config whiteList=, blackList=
INFO config ragflowUrl=http:///XX.XX.XX.XX, ragflowApiKey=Bearer ragflow-RhMjc0NjFhNTZmNTExZjBiYWY****
INFO config ragflowDatasetId=b2af7c4c56f511f0a3d700163e0****
INFO config dingDingClientId=ding9nyefxviedv****, dingDingClientSecret=Kh3qQZafV9vZZIwvP****-V7f3tkmYmcYibwT7zWNXrvOm4UbETOhqEt****
INFO config dingDingUserId=012713339****
NFO Verifying RAGFlow connection...
INFO Attempting to list datasets to validate the connection...
INFO Successfully found matching dataset: Name='test', ID='b2af7c4c56f511f0a3d700163e****'
INFO RAGFlow connection verified successfully.
INFO Starting DingTalk authentication process...
INFO Waiting for the initial app access token...
INFO App access token refresh loop started.
INFO Attempting to refresh the app access token...
INFO {'headers': {'server': 'DingTalk/1.0.0', 'date': 'Tue, 29 Jul 2025 09:14:19 GMT', 'content-type': 'application/json;charset=utf-8', 'transfer-encoding': 'chunked', 'connection': 'keep-alive', 'access-control-allow-origin': '*', 'access-control-expose-headers': '*', 'x-acs-request-id': 'AD15D9B5-C47D-7C06-BCE7-03AA76A****', 'x-acs-trace-id': 'ad2c67867d7153507b9a74e399****', 'access-control-allow-headers': 'X-Requested-With, X-Sequence, _aop_secret, _aop_signature, x-acs-dingtalk-access-token', 'content-encoding': 'gzip'}, 'statusCode': 200, 'body': {'accessToken': '38a7d120903c35a7aa2e866dca****', 'expireIn': 7200}}
INFO set dingding access token: 38a7d120903c35a7aa2e866dca****
INFO set dingding access token expires in: 7200
INFO App access token refreshed successfully. Next refresh scheduled in 6600 seconds.
INFO Waiting 6600 seconds until the next token refresh.
INFO Initial app access token acquired.
INFO Fetching user UnionID...
INFO set dingding union id: XYesTh9u****
INFO Successfully fetched UnionID.

Key success indicators:

INFO RAGFlow connection verified successfully. — RAGFlow connected
INFO Successfully fetched UnionID. — DingTalk authenticated

Sync from DingTalk Drive

Enter 2 and press Enter.
Enter the number that corresponds to the folder containing your target documents, then press Enter.
Wait for KBSync to complete the sync.

If the sync is running correctly, the output looks similar to:

./KBSync --config config

INFO config SourceType=DingDing, SinkType=RagFlow
INFO config whiteList=, blackList=
INFO config ragflowUrl=http:///XX.XX.XX.XX, ragflowApiKey=Bearer ragflow-RhMjc0NjFhNTZmNTExZjBiYWY****
INFO config ragflowDatasetId=b2af7c4c56f511f0a3d700163e0****
INFO config dingDingClientId=ding9nyefxviedv****, dingDingClientSecret=Kh3qQZafV9vZZIwvP****-V7f3tkmYmcYibwT7zWNXrvOm4UbETOhqEt****
INFO config dingDingUserId=012713339****
NFO Verifying RAGFlow connection...
INFO Attempting to list datasets to validate the connection...
INFO Successfully found matching dataset: Name='test', ID='b2af7c4c56f511f0a3d700163e****'
INFO RAGFlow connection verified successfully.
INFO Starting DingTalk authentication process...
INFO Waiting for the initial app access token...
INFO App access token refresh loop started.
INFO Attempting to refresh the app access token...
INFO {'headers': {'server': 'DingTalk/1.0.0', 'date': 'Tue, 29 Jul 2025 09:14:19 GMT', 'content-type': 'application/json;charset=utf-8', 'transfer-encoding': 'chunked', 'connection': 'keep-alive', 'access-control-allow-origin': '*', 'access-control-expose-headers': '*', 'x-acs-request-id': 'AD15D9B5-C47D-7C06-BCE7-03AA76A****', 'x-acs-trace-id': 'ad2c67867d7153507b9a74e399****', 'access-control-allow-headers': 'X-Requested-With, X-Sequence, _aop_secret, _aop_signature, x-acs-dingtalk-access-token', 'content-encoding': 'gzip'}, 'statusCode': 200, 'body': {'accessToken': '38a7d120903c35a7aa2e866dca****', 'expireIn': 7200}}
INFO set dingding access token: 38a7d120903c35a7aa2e866dca****
INFO set dingding access token expires in: 7200
INFO App access token refreshed successfully. Next refresh scheduled in 6600 seconds.
INFO Waiting 6600 seconds until the next token refresh.
INFO Initial app access token acquired.
INFO Fetching user UnionID...
INFO set dingding union id: XYesTh9u****
INFO Successfully fetched UnionID.

Key success indicators:

INFO RAGFlow connection verified successfully. — RAGFlow connected
INFO Successfully fetched UnionID. — DingTalk authenticated

Manage ongoing syncs

After the initial run, KBSync periodically syncs data from the source. You can:

Set the sync interval
Manually pause or restart the program as needed

For knowledge base sources, you may need to log on to your DingTalk account again when restarting KBSync.

Appendix

Get the API Server endpoint

Log on to the RAGFlow page.
In the left-side navigation pane, click API.
Copy the API Server value.

Get the API key

Log on to the RAGFlow page.
In the left-side navigation pane, click API.
To the right of RAGFlow API, click API KEY.
In the API KEY dialog box, click Create New Key.
Click to copy and record the token.

Get the knowledge base ID

Log on to the RAGFlow page.
On the Knowledge Base page, click the target knowledge base.
In the URL of the page that opens, copy the value after id=. That value is the knowledge base ID.