Lingma Enterprise Edition administrators and organization-wide global administrators (Dedicated Edition) can configure security filtering policies for developers when using intelligent Q&A, code completion, and knowledge base uploads.
Supported edition | Enterprise Dedicated Edition |
Entry point
Use Lingma administrator or organization global administrator (only applicable to enterprise exclusive version) account to log on to the Lingma console, and select in the left navigation bar.
Based on the scenario where the filter is applied, select Intelligent Q&A, Code Completion, or Knowledge Base Upload (only applicable to Enterprise Dedicated Edition) from the top tabs.
Enable the switch for the appropriate filter according to your requirements, and then configure the related parameters. The filter types supported by each scenario are as follows:
Scenario
Filter type
Description
Intelligent Q&A
Model input filter
When using the intelligent Q&A feature of the Lingma IDE plug-in, user input to the LLM will pass through the model input filter, and the content output by the LLM will pass through the model output filter.
Model output filter
Code completion
Model input filter
When using the code completion feature of the Lingma IDE plug-in, user input to the LLM will pass through the model input filter, and the content output by the LLM will pass through the model output filter.
Model output filter
Knowledge base upload
Model input filter
Files uploaded to the Lingma knowledge base must first pass the review of this filter before they can be successfully uploaded.
ImportantMake sure that you upgrade the developer's Lingma IDE plug-in to V1.4.0 or later for the configured filter to take effect.
After the Intelligent Q&A Filter and Code Completion Filter are enabled or modified, it will take approximately 5 to 10 minutes for them to take effect on the Lingma IDE plug-in used by developers.
After the Knowledge Base Upload Filter is enabled or modified, it takes effect immediately, and filtering will be performed when uploading enterprise knowledge base files.
Configure the model input filter used in the chat or code completion scenario
Method 1: Configure using regular expressions
Make sure that administrators have fully validated regular expressions during expression configuration to avoid performance degradation or other exceptions before developers use the IDE plug-in.
Processing method: You can configure filters using regular expressions, with support for 3 modes.
Bypass
After a regular expression is matched, data is not processed.
Block
After a regular expression is matched, model requests are directly blocked.
Replace
After a regular expression is matched, replace the current content with the specified configurations.
Message notification: You can enable message notifications to push to the required message receiving platform through webhooks.
Execution order: Executed according to the configured order.
Regular expression limit: A maximum of 10 expressions can be added.
Regular expression standard: Regular expression configuration follows the ECMAScript standard, supporting common flags such as
i
(case-insensitive),g
(global match), ands
(DOTALL mode).Regular expression configuration examples:
Rule name
Regular expression
Replacement content
Original text
After replacement
ID card number
(?<pre>.*)(\d{15})((\d{2})([0-9Xx]))(?<post>.*)
$<pre>***$<post>
ID card number: 330204197709022312.
ID card number: ***.
Email
\w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*
***
My email address is lin***@aliyunmail.com
My email address is ***
Password
(.*password=)([\w\d]+)(.*)
$1***$3
{password=1213213}
{password=***}
Method 2: Configure the filter by customizing a script (only available to Enterprise Dedicated Edition)
Enterprise Dedicated Edition allows you to configure the filter by customizing a script to meet your requirements for model input filtering in complex scenarios. You must perform the following steps:
Step 1: Develop a script
You can use the TypeScript language for script development. You can refer to the following example for code development. Procedure:
Download the template code repository: Click the repository address: lingma-extension-template. This template repository integrates the scaffolding needed for script development. Please carefully read the
README.md
file and code examples.Implement the "pre-processing" interface: Implement the
RequestPreHandler
interface. You can refer to the Custom script API for the API. The following is an example implementation snippet ofSensitiveContentFilter.ts
./** * Define the sensitive content filter. The filter implements the sensitive information preprocessing of the data input to the model */ export const sensitiveContentFilter: RequestPreHandler = { handle: async (request: RawRequest, SDKTool: LingmaSDKTool) => { const dataMap = PayloadUtil.getPayloadData(request.payload); for (const [key, value] of dataMap.entries()) { if (value.includes('password')) { return ResultUtil.buildBlockResult('Content contains password'); } } // If you want to perform differentiated processing for different actions, refer to the following code. switch (request.action) { case ActionEnum.COMPLETION: // do something break; case ActionEnum.CODE_PROBLEM_SOLVE: // do something break; default: return ResultUtil.buildNoOpsResult(); } return ResultUtil.buildNoOpsResult(); }, };
Run and debug the code by running the
main
method to test whether the script meets your expectations. Procedure:Step 1
Edit the
src/index.ts
file, modify themain
function, and adjust the code that needs to be debugged, as shown in the following example:async function main() { const value1 = ['password=123', 'abc']; const value2 = 'hello world'; const dataMap = new Map(); dataMap.set(PayloadDataKeyEnum.SELECTED_CODE, value1); dataMap.set(PayloadDataKeyEnum.USER_INPUT, value2); const mockRequest: RawRequest = { action: ActionEnum.CODE_GENERATE_COMMENT, payload: { associatedContexts: [], data: dataMap, }, requestId: '123', }; const response = await sensitiveContentFilter.handle(mockRequest, SDKTool); console.log(response); }
Step 2
Open the code file that you want to debug in VS Code and set a breakpoint, select the program startup icon from the debug view, and then click the Run button.
Step 2: Compile and build the script
Compile the debugged ts file into a js file, such as compiling the SensitiveContentFilter.ts
file into the SensitiveContentFilter.js
file. Perform the following steps to compile and build:
Open the configuration file
src/build.js
, modify theentryPoints
andoutfile
configuration parameters, specify the path of the ts file to be compiled and built in theentryPoints
parameter, and specify the output path of the built product inoutfile
.Execute the command
node build.js
in the root directory of the code repository. After successful execution, the corresponding js file will be output to the product output path specified byoutfile
.
Step 3: Locally test the script
Before uploading the script to the enterprise configuration backend, you can complete debugging locally to ensure that the script can be integrated with the Lingma IDE plug-in and correctly perform security filtering for completion or Q&A scenario behaviors. Procedure:
Copy the built js file to the
/extension/local/script/
directory in the Lingma local storage path.Modify the
config.json
file: This file is located in the/extension/local/
directory of the Lingma local storage path. Open theconfig.json
file, findcontentHandlerScripts
, and add the configuration information of the script to the corresponding content. If there is nocontentHandlerScripts
, you can add a new array-type configuration. Refer to the following example:{ "contentHandlerScripts": [ { "identifier": "SensitiveContentFilter", "name": "Filter sensitive content", "version": "1.0.0", "scriptPath": "~/.lingma/extension/local/script/SensitiveContentFilter.js", "state": "enabled", "bizType": "completion" } ] }
The following table describes the required parameters.
Parameter
Description
identifier
The unique script ID.
name
The name of the script.
version
The version number of the script. If you want to change the script content, you must upgrade the version number. Otherwise, the script cannot take effect.
scriptPath
The path where the script is stored. Take note of the following items:
The script must be stored in the
/extension/local/script/
directory of the local storage path.The js file name of the script (such as
SensitiveContentFilter.js
) must be consistent with the value ofidentifier
.
state
The script status.
enabled
indicates that the script is enabled, anddisabled
indicates that the script is disabled.bizType
The business scenario where the script is applied.
completion
indicates code completion, andchat
indicates intelligent Q&A.
Step 4: Upload the script
After local debugging and verification, you can upload the script. Procedure:
Go to the Lingma console - Policy Management, and select the scenario for which you need to enable the security filter.
Select custom scripts as the required filter type.
Upload the built JavaScript file.
After you upload the file, click Save Configurations. The configuration takes effect on the plug-in side in about 5 minutes.
Custom script API
The following modes are available to process custom scripts:
Block: Subsequent process is blocked. Once blocked, the LLM is not requested for inference and this request is interrupted.
Filter: Sent data is changed, such as obfuscation, deletion, and replacement, and then the subsequent process continues.
Bypass: Sent data is not processed and is returned as it is, and then the subsequent process continues.
Interface definition
/**
* AI Coding Assistant pre-processing interface
*/
export interface RequestPreHandler {
// Process the current request.
handle: (request: RawRequest, SDKTool: LingmaSDKTool) => Promise;
}
Input parameters
/**
* Define the request object, including the action that triggers the current request and the raw data to be input to the LLM.
*/
export interface RawRequest {
// The unique identifier of the current request, which can be used to track request execution.
action: ActionEnum;
// The action that trigger the current request.
payload: ContentPayload;
// The payload that encapsulates the raw data.
requestId: string;
}
// The value types in ContentPayload.data.
export type PayloadDataValueType = string | number | string[];
/**
* Encapsulate the raw data that is input to the LLM.
*/
export class ContentPayload {
// The data set to be processed. You can view the corresponding key by querying the definitions of ContextValueKeyEnum.
data: Map;
// The context associated with data processing.
associatedContexts: ContextItem[];
constructor() {
this.data = new Map();
this.associatedContexts = [];
}
}
/**
* Define keys in ContentPayload.data.
*/
export enum PayloadDataKeyEnum {
// The code snippet selected by the user.
SELECTED_CODE ='lingma:code',
// The text entered by the user.
USER_INPUT = 'lingma:text',
// The error message.
ERROR_MESSAGES = 'lingma:error_messages',
// The log information on the terminal.
TERMINAL_CONTENT = 'lingma:terminal_content',
// The preceding code snippet of the line where the current cursor is located when the code is completed.
PREFIX_CODE = 'lingma:code_prefix',
// The following code snippet of the line where the current cursor is located when the code is completed.
SUFFIX_CODE = 'lingma:code_suffix',
// The similar code snippet.
SIMILAR_CODE = 'lingma:similar_code',
}
/**
* Define actions that trigger requests.
*/
export enum ActionEnum {
// Unit testing
GENERATE_TESTCASE = 'GENERATE_TESTCASE',
// Generate a comment.
CODE_GENERATE_COMMENT = 'CODE_GENERATE_COMMENT',
// Interpret the code.
EXPLAIN_CODE = 'EXPLAIN_CODE',
// Optimize the code.
OPTIMIZE_CODE = 'OPTIMIZE_CODE',
// Start a chat. You can enter text in the chat input box.
FREE_INPUT = 'FREE_INPUT',
// Quickly fix a code issue.
CODE_PROBLEM_SOLVE = 'CODE_PROBLEM_SOLVE',
// Generate an shell command.
TERMINAL_COMMAND_GENERATION = 'TERMINAL_COMMAND_GENERATION',
// Fix a terminal error.
TERMINAL_EXPLAIN_FIX = 'TERMINAL_EXPLAIN_FIX',
// Complete the code.
COMPLETION = 'COMPLETION',
}
Output parameters
/**
* Return model input filter results.
*/
export class HandlerResponse {
// The processing policy that is used to control the subsequent processing logic.
handlePolicy: HandlePolicy;
// The reason description.
reason?: string;
// When handlePolicy is set to FILTER, you must set this filtering property. The property value is the filtered data and must be consistent with the content of ContentRequest.payload.
payload?: ContentPayload;
constructor() {
// Default value
// eslint-disable-next-line @typescript-eslint/no-use-before-define
this.handlePolicy = HandlePolicy.NO_OPS;
this.reason = '';
this.payload = new ContentPayload();
}
}
/**
* Return the available processing policies.
*/
export enum HandlePolicy {
// Block. This policy indicates that requests are directly blocked.
BLOCK = 'BLOCK',
// Filter. This policy indicates that requests are intercepted and the payload content is modified.
FILTER = 'FILTER',
// Bypass. This policy indicates that requests are not processed.
NO_OPS = 'NO_OPS',
}
Configure the model output filter in the chat or code completion scenario
Method 1: Configure using regular expressions
Make sure that administrators have fully validated regular expressions during expression configuration to avoid performance degradation or other exceptions before developers use the IDE plug-in.
Processing method: You can configure filters using regular expressions, with support for only one mode, as follows:
Bypass
After a regular expression is matched, data is not processed.
Message notification: You can enable message notifications to push to the required message receiving platform through webhooks.
Execution order: Executed according to the configured order.
Regular expression limit: A maximum of 10 expressions can be added.
Regular expression standard: Regular expression configuration follows the ECMAScript standard, supporting common flags such as
i
(case-insensitive),g
(global match), ands
(DOTALL mode).Regular expression configuration examples:
Rule name
Regular expression
Original text
ID card number
(?<pre>.*)(\d{15})((\d{2})([0-9Xx]))(?<post>.*)
ID card number: 330204197709022312.
Email
\w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*
My email address is lin***@aliyunmail.com
Password
(.*password=)([\w\d]+)(.*)
{password=1213213}
Method 2: Configure the filter by customizing a script (only available to Enterprise Dedicated Edition)
Enterprise Dedicated Edition allows you to configure the filter by customizing a script to meet your requirements for model output filtering in complex scenarios. You must perform the following steps:
Step 1: Develop a script
You can use the TypeScript language for script development. You can refer to the following example for code development. Procedure:
Download the template code repository: Click the repository address: lingma-extension-template. This template repository integrates the scaffolding needed for script development. Please carefully read the
README.md
file and code examples.Implement the "post-processing" interface: Implement the
RequestPostHandler
interface. You can refer to the Custom script API for the API. The following is an example implementation snippet ofLLMChatAuditHandler.ts
, which implements the function of auditing AIChat operations of Lingma and reporting the audit content to Alibaba Cloud SLS.import {ResultUtil} from '../common/HandlerRespUtil'; import { JsonUtil } from '../common/JsonUtil'; import {PayloadUtil} from '../common/PayloadUtil'; import {Config} from '../sdk/ConfigManager'; import {LingmaSDKTool} from '../sdk/LingmaSDKTool'; import axios from "axios"; import moment from "moment"; import os from "os"; import { ActionEnum, AIResponse, HandlePolicy, RawRequest, RequestPostHandler, RequestPreHandler } from '../sdk/RequestHandleSDK'; /** * Configure the custom script output filter. The filter implements the processing of the request content on a remote server, such as code scanning and content audit. */ export const llmResultAuditHandler: RequestPostHandler = { handle: async (request: RawRequest, response: AIResponse,SDKTool: LingmaSDKTool) => { // The name of the operator. let userName = SDKTool.user.name; // The ID of the operator. let userId = SDKTool.user.uid; // ide let ide = SDKTool.idePlatform; // ide let ideVersion = SDKTool.ideVersion; // The operation time. let operationTime = moment().format("YYYY-MM-DD HH:mm:ss"); // The IP address of the operation. let opeartionIp = getIpAddress(); // The business scenario of the operation. The code completion or chat scenario is available. let bizType = "chat"; // The request ID of the operation. let requestId = request.requestId; // The operation that is performed. let action = request.action; // The operation content. To avoid failed reporting caused by the excessive size of reported operation content, we recommend that you select appropriate fields based on your audit requirements. The operation content cannot exceed 16 KB in size. let inferredResult = response.inferredResult.text; // Report the operation content to Simple Log Service (SLS). // The name of the project in SLS. let slsProject = "xxx"; // The name of the Logstore in SLS. let slsLogStore = "xxx"; // The endpoint of the region where the SLS instance resides. let endPoint = "cn-hangzhou.log.aliyuncs.com"; let slsWebTrackingUrl = `http://${slsProject}.${endPoint}/logstores/${slsLogStore}/track?APIVersion=0.6.0&request_id=${requestId}&action=${action}&biz_type=${bizType}&user_name=${userName}&user_id=${userId}&ide=${ide}&ide_version=${ideVersion}&operation_time=${operationTime}&opeartion_ip=${opeartionIp}&inferredResult=${inferredResult}`; axios.get(slsWebTrackingUrl).catch((error) => { console.error(error); }); // Return the filtering results. return ResultUtil.buildPostHandlerResponse(HandlePolicy.NO_OPS, response.inferredResult,'No operations are required.'); }, }; /** * Add the custom script filter to the configurations. * @param config. It is used to manage configurations in a unified manner and is provided by LingmaExtensionSDK. */ export function modifyConfig(config: Config) { config.postContentHandlers.push(llmResultAuditHandler); return config; } function getIpAddress() { const interfaces = os.networkInterfaces(); for (let devName in interfaces) { let iface = interfaces[devName]; for (let i = 0; i < iface.length; i++) { let alias = iface[i]; if ( alias.family === "IPv4" && alias.address !== "127.0.0.1" && !alias.internal ) return alias.address; } } return "No IP address found"; }
Run and debug the code by running the
main
method to test whether the script meets your expectations. Procedure:Step 1
Edit the
src/index.ts
file, modify themain
function, and adjust the code that needs to be debugged, as shown in the following example:async function main() { const value2 = 'hello world'; const dataMap = new Map(); dataMap.set(PayloadDataKeyEnum.USER_INPUT, value2); const request: RawRequest = { action: ActionEnum.CODE_GENERATE_COMMENT, payload: { associatedContexts: [], data: dataMap, }, requestId: 'test-request-id', }; const aiResponse: AIResponse = { inferredResult: { text: 'reply hello world', }, }; const response = await llmResultAuditHandler.handle(request, aiResponse, SDKTool); console.log(response); }
Step 2
Open the code file that you want to debug in VS Code and set a breakpoint, select the program startup icon from the debug view, and then click the Run button.
Step 2: Compile and build the script
Compile the debugged ts file into a js file, such as compiling the LLMChatAuditHandler.ts
file into the LLMChatAuditHandler.js
file. Perform the following steps to compile and build:
Open the configuration file
src/build.js
, modify theentryPoints
andoutfile
configuration parameters, specify the path of the ts file to be compiled and built in theentryPoints
parameter, and specify the output path of the built product inoutfile
.Execute the command
node build.js
in the root directory of the code repository. After successful execution, the corresponding js file will be output to the product output path specified byoutfile
.
Step 3: Locally test the script
Before uploading the script to the enterprise configuration backend, you can complete debugging locally to ensure that the script can be integrated with the Lingma IDE plug-in and correctly perform security filtering for completion or Q&A scenario behaviors. Procedure:
Copy the built js file to the
/extension/local/script/
directory in the Lingma local storage path.Modify the
config.json
file: This file is located in the/extension/local/
directory of the Lingma local storage path. Open theconfig.json
file, findcontentHandlerScripts
, and add the configuration information of the script to the corresponding content. If there is nocontentHandlerScripts
, you can add a new array-type configuration. Refer to the following example:{ "contentHandlerScripts": [ { "identifier": "LLMChatAuditHandler", "name": "AIChat audit", "version": "1.0.0", "scriptPath": "~/.lingma/extension/local/script/LLMChatAuditHandler.js", "state": "enabled", "stage":"post", "bizType": "completion" } ] }
The following table describes the required parameters.
Parameter
Description
identifier
The unique script ID.
name
The name of the script.
version
The version number of the script. If you want to change the script content, you must upgrade the version number. Otherwise, the script cannot take effect.
scriptPath
The path where the script is stored. Take note of the following items:
The script must be stored in the
/extension/local/script/
directory of the local storage path.The js file name of the script (such as
LLMChatAuditHandler.js
) must be consistent with the value ofidentifier
.
state
The script status.
enabled
indicates that the script is enabled, anddisabled
indicates that the script is disabled.stage
The stage where the script takes effect.
post
indicates model output filtering, andpre
indicates model input filtering. The default value ispre
(model input filtering).bizType
The business scenario where the script is applied.
completion
indicates code completion, andchat
indicates intelligent Q&A.
Step 4: Upload the script
After local debugging and verification, you can upload the script. Procedure:
Go to the Lingma console - Policy Management, and select the scenario for which you need to enable the security filter.
Select custom scripts as the required filter type.
Upload the built JavaScript file.
After you upload the file, click Save Configurations. The configuration takes effect on the plug-in side in about 5 minutes.
Custom script API
Only the following mode is available to process custom scripts:
Bypass: Sent data is not processed and is returned as it is, and then the subsequent process continues.
Interface definition
/**
* AI Coding Assistant post-processing interface
* @param request The current request sent by the user.
* @param response Inference content returned by the LLM.
* @param SDKTool SDK tool class. You can use this tool class to obtain information about IDE and plug-ins.
* @returns The returned results that are processed by the model output filter.
*/
export interface RequestPostHandler {
// The model output filtering method.
handle: (request: RawRequest, response: AIResponse, SDKTool: LingmaSDKTool) => Promise;
}
Input parameters
/**
* Define the request object, including the action that triggers the current request and the raw data to be input to the LLM.
*/
export interface RawRequest {
// The unique identifier of the current request, which can be used to track request execution.
action: ActionEnum;
// The action that trigger the current request.
payload: ContentPayload;
// The payload that encapsulates the raw data.
requestId: string;
}
/**
* Define the results that are generated by model inference.
*/
export class InferredResult {
// The text content that is generated by the LLM.
text: string;
constructor() {
this.text = '';
}
}
// The value types in ContentPayload.data.
export type PayloadDataValueType = string | number | string[];
/**
* Encapsulate the raw data that is input to the LLM.
*/
export class ContentPayload {
// The data set to be processed. You can view the corresponding key by querying the definitions of ContextValueKeyEnum.
data: Map;
// The context associated with data processing.
associatedContexts: ContextItem[];
constructor() {
this.data = new Map();
this.associatedContexts = [];
}
}
/**
* Define keys in ContentPayload.data.
*/
export enum PayloadDataKeyEnum {
// The code snippet selected by the user.
SELECTED_CODE ='lingma:code',
// The text entered by the user.
USER_INPUT = 'lingma:text',
// The error message.
ERROR_MESSAGES = 'lingma:error_messages',
// The log information on the terminal.
TERMINAL_CONTENT = 'lingma:terminal_content',
// The preceding code snippet of the line where the current cursor is located when the code is completed.
PREFIX_CODE = 'lingma:code_prefix',
// The following code snippet of the line where the current cursor is located when the code is completed.
SUFFIX_CODE = 'lingma:code_suffix',
// The similar code snippet.
SIMILAR_CODE = 'lingma:similar_code',
// The file path of the completed code.
FILE_PATH = 'lingma:file_path',
}
/**
* Define actions that trigger requests.
*/
export enum ActionEnum {
// Unit testing
GENERATE_TESTCASE = 'GENERATE_TESTCASE',
// Generate a comment.
CODE_GENERATE_COMMENT = 'CODE_GENERATE_COMMENT',
// Interpret the code.
EXPLAIN_CODE = 'EXPLAIN_CODE',
// Optimize the code.
OPTIMIZE_CODE = 'OPTIMIZE_CODE',
// Start a chat. You can enter text in the chat input box.
FREE_INPUT = 'FREE_INPUT',
// Quickly fix a code issue.
CODE_PROBLEM_SOLVE = 'CODE_PROBLEM_SOLVE',
// Generate an shell command.
TERMINAL_COMMAND_GENERATION = 'TERMINAL_COMMAND_GENERATION',
// Fix a terminal error.
TERMINAL_EXPLAIN_FIX = 'TERMINAL_EXPLAIN_FIX',
// Complete the code.
COMPLETION = 'COMPLETION',
}
Output parameters
/**
* Return the model output filtering results.
*/
export class PostHandlerResponse {
// The processing policy that is used to control the subsequent processing logic.
handlePolicy: HandlePolicy;
// The reason description.
reason?: string;
// The returned results that are processed by the model output filter.
processedResult: InferredResult;
constructor() {
// Default value
this.handlePolicy = HandlePolicy.NO_OPS;
this.reason = '';
this.processedResult = new InferredResult();
}
}
/**
* Encapsulate the returned LLM results.
*/
export class AIResponse {
// The model inference results.
inferredResult: InferredResult;
constructor() {
this.inferredResult = new InferredResult();
}
}
/**
* Define the results that are generated by model inference.
*/
export class InferredResult {
// The text content that is generated by the LLM.
text: string;
constructor() {
this.text = '';
}
}
/**
* The processing policy that is used to control the subsequent processing logic (only NO_OPS is currently supported for model output filters).
*/
export enum HandlePolicy {
// Block. This policy indicates that requests are directly blocked.
BLOCK = 'BLOCK',
// Filter. This policy indicates that requests are intercepted and the payload content is modified.
FILTER = 'FILTER',
// Bypass. This policy indicates that requests are not processed.
NO_OPS = 'NO_OPS',
}
Configure the knowledge base filter (applicable only to Enterprise Dedicated Edition)
In Enterprise Dedicated Edition, Lingma administrators and global administrators can configure knowledge base filters in policy configuration. After configuration, knowledge base files are reviewed before upload, meeting the requirements for pre-filtering knowledge base content in specific scenarios.
Procedure
Step 1: Enable and edit the knowledge base filter
In the left navigation bar, click Policy Configuration, and on the right page, click the Knowledge Base Filter tab.
On the knowledge base filter configuration page, you can turn on the switch Enable/Disable Knowledge Base Upload Pre-filter to edit parameter configurations.
URL Address
This parameter is required.
The address of the interface to access the third-party scanning service provided by the enterprise. POST requests are required for the interface.
Token Field Name
This parameter is required.
The field name for the token in request header.
Secret Key
This parameter is required.
The key required to access the token. The token is added to the specified request header field to verify the validity of the request. For more information, see the "Security token" section of this topic.
Step 2: Test the filter connectivity
After filling in the correct information, test the connectivity by clicking the Test Connection button. The test is successful when the third-party filter interface returns a
2xx
status code.If another status code is returned, the test will fail, and you need to check the information you filled in and test the connection again.
Step 3: Save the configurations of the knowledge base filter
Click the Save Configurations button to save your filter configuration. After the configurations are saved, the filter immediately takes effect.
Third-party scanning service interface specifications
An enterprise needs to provide a third-party scanning service so that the knowledge base filter can use the service to scan the uploaded knowledge base content and then upload passed content. To ensure that the filter with specified configurations works properly, the scanning service interface must meet the following design requirements:
Request header
Parameter name | Required | Description | Example |
X-Auth-Raw | Yes | The authentication parameter of the interface. The header can be used as the token field name that you specify on the filter configuration page. The header value must be the final secret key generated after the encryption algorithm is used. For more information, see the "Security token" section of this topic. | 6c3baa76c62550eab864e6f75c4bb |
Content-Type | Yes | The media type information in the request and response. | multipart/form-data |
Security Token: A security signature designed by Alibaba Cloud to prevent malicious attackers from stealing your cloud service permissions. To generate a token, the following items are required: secret key, current time, additional information, and encryption algorithm.
Token Generation: When Lingma calls the third-party scanning service interface, it carries a security token in the request header for identity authentication. The generated token value is calculated based on the following parameters:
token = sha256Hex(method + url + timestamp + tokenSecret) + timestampHex
method
The request method. The Post request method is used.
url
The scanning service interface URL address filled in when configuring the knowledge base filter.
timestamp
The current time.
tokenSecret
The Secret Key filled in when configuring the knowledge base filter.
timestampHex
The timestamp in the hexadecimal format.
Token Verification: Your third-party scanning service can refer to the following code to verify the legitimacy of the token authentication in the request.
ImportantTimestamp: Make sure that the time between the client and the server is synchronized to avoid failed token verification due to time offsets.
Key management: Properly manage the
tokenSecret
and do not disclose it to unauthorized users.Expiration time: Adjust the validity period of the token based on your business requirements. In this example, the validity period of the token is set to 60 seconds.
/* * Parameters: * receivedHash: the received hash value that contains the timestamp information. * tokenSecret: the key that is used to generate the hash. * url: the URL of the current request. */ public boolean validateAuthRaw(String receivedHash, String tokenSecret, String url) { final String method = "POST"; // Extract the timestamp from the receivedHash value. String tsHex = receivedHash.substring(receivedHash.length() - 8); long tsSec = Long.parseLong(tsHex, 16); // Calculate the difference between the current time and the receiving time. Assume that the allowed maximum time difference is 60 seconds. long now = System.currentTimeMillis() / 1000L; if (Math.abs(now - tsSec) > 60) { return false; // The timeout period. } // Construct the string to be signed. String plain = method + url + tsSec + tokenSecret; // Generate the expected hash value. String expectedHash = org.apache.commons.codec.digest.DigestUtils.sha256Hex(plain); // Compare the received hash value with the expected hash value. return expectedHash.equals(receivedHash.substring(0, receivedHash.length() - 8)); }
Request parameters
Parameter name | Parameter type | Required | Description | Example |
metadata | string | Yes | The business metadata that contains the user and queryID field (Content-Type: application/json). | {"user": "user0000001", "queryId": "cd2fd109-c4d4-489f-9b27-53752f7827d6"} |
file | file | Yes | The inspected file |
Request example.
Content-Type: multipart/form-data; boundary=${bound}
--${bound}
Content-Disposition: form-data; name="metadata"
Content-Type: application/json
{
"user":"user0000001",
"queryID":"cd2fd109-c4d4-489f-9b27-53752f7827d6"
}
--${bound}
Content-Disposition: form-data; name="file"; filename="test-file.pdf"
Content-Type: application/pdf
%binary-file-content-here%
Response parameters
The interface must return the HTTP status code 200 and contain the response parameters described in the following table.
Parameter name | Parameter type | Required | Description | Example |
forbidden | boolean | Yes | The security inspect result. The value "true" indicates that the security inspect failed. | false |
errorMsg | string | No | The error message that indicates the reason of the failed security inspect. | "The file contains malicious content. Please modify and upload it again." |
queryId | string | No | The request ID, also shown as the value of the queryID field in the request metadata. | "cd2fd109-c4d4-489f-9b27-53752f7827d6" |
user | string | No | The user ID, also shown as the value of the user field in the request metadata. | "user0001" |