AI Coding Assistant Lingma:Enterprise knowledge base storage service connection switch

Background information

Lingma administrators and organization global administrators (Dedicated Edition) can switch the knowledge base storage service from Lingma built-in storage service to enterprise self-managed storage service in the knowledge base configuration of the Lingma console.

Prerequisites

• Network requirements:

  • If your self-managed storage service supports public network access, Lingma Enterprise Dedicated Edition can directly establish a connection with your storage service. To improve data security, you can configure an IP address whitelist to manage access permissions.

  • If your self-managed storage service does not support public network access, a virtual private cloud (VPC) needs to be configured between Lingma Enterprise Dedicated Edition and your self-managed storage service for access, and reverse access needs to be enabled to support Lingma Enterprise Dedicated Edition to access resources in the self-managed storage service.

• Self-managed storage service:

  • Object Storage Service: See Alibaba Cloud OSS guide.

    Compatible products	Alibaba Cloud Object Storage Service (OSS), Amazon S3, Tencent Cloud Object Storage (COS), Huawei Cloud Object Storage Service (OBS), and open source MinIO
    Permission requirements	An authorized account must have the following permissions: List objects in a bucket (List): Allows users to list all objects in a bucket. Read object content (Read): Allows users to read the content of objects in a bucket. Upload new objects (Put): Allows users to upload new objects to a bucket. Delete objects (Delete): Allows users to delete objects in a bucket.
    Storage class	Normal (non-cold archive)
    Version	Compatible with all versions
    Storage capacity	More than 10 GB of storage capacity (You can expand the storage capacity based on your business requirements.)

  • Vector database: See Alibaba Cloud Elasticsearch configuration guide.

    Compatible products	Elasticsearch Enterprise Edition and Elasticsearch Open-source Edition
    Version	8.9 and later
    Permission requirements	An authorized account must have the following permissions: Create index (create_index): Allows users to create new indexes. If the create index request includes aliases to be added to the index, the manage permission is also required. Write documents (write,create_doc): Allows users to perform all write operations on documents, including indexing, updating, deleting, batch operations, and updating mappings. Read documents (read): Allows users to perform read-only operations, including count, explain, get, mget, scripts, search, scroll, etc. Delete documents (delete): Allows users to delete documents in an index. Read access to create_index, read, delete, index: Ensures users can perform these specific operations.
    CPU specification requirements	More than 4 cores and 8 GB memory (We recommend that you use 8 cores and 16 GB memory.)
    Storage capacity	More than 300 GB of storage capacity (You can expand the storage capacity based on your business requirements.)
    Configure instance access whitelist (optional)	When accessing Alibaba Cloud Elasticsearch (ES) instances, we recommend that you add the IP address of the device to be accessed to the access whitelist of the ES instance. Please submit a ticket to inquire about the egress IP address of the Lingma instance.

Scenario

An enterprise knowledge base includes two storage types: OSS and vector database. OSS stores uploaded files, such as documents or code packages, and the vector database stores file slices and vectorized data.

Lingma supports knowledge base storage service migration, including OSS and vector database, to maximize the security and privacy of knowledge base data. The following figure shows an overview of the entire process of knowledge base storage service migration and post-migration usage:

1. In the preliminary process, Lingma administrators or enterprise administrators complete the public network or VPC access configuration on the enterprise management page in the Alibaba Cloud Management Console, and configure storage service migration in the Lingma console.

2. In the usage process, when Lingma administrators upload knowledge base data files, because the storage service migration configuration has taken effect, the files will be saved to the configured self-managed OSS, and the vector data will be saved to the configured self-managed vector database.

Procedure

1. Log on to the Alibaba Cloud Management Console and select the network access configuration for Lingma Enterprise Dedicated Edition as needed.

   Public network access

   If your self-managed storage service supports public network access, you can to complete the public network access configuration for Lingma Enterprise Dedicated Edition. To improve data security, you can also configure an IP address whitelist to manage access permissions.

   Vpc access

   If your self-managed storage service does not support public network access, a virtual private cloud (VPC) needs to be configured between Lingma Enterprise Dedicated Edition and your self-managed storage service for access, and reverse access needs to be enabled to support Lingma Enterprise Dedicated Edition to access resources in the self-managed storage service. For VPC access and reverse access configuration.

2. Log on to the Lingma enterprise console, and click Knowledge Base Configuration on the Knowledge Management page.

3. Configure the settings for switching storage service connections on the Knowledge base configuration page, including settings for the OSS and vector database connections. After entering the corresponding data, you can click Check Connection to test the connection. If you confirm that the connection is correct, you can click Save Configuration. When the migration completion prompt is displayed, the configuration is successful.

   Important

   The configuration takes effect immediately after it is saved, and the knowledge base content provided by Lingma will be cleared. At this time, the knowledge base in the Lingma enterprise management console will return to its initial state, which is empty.

   After the application storage service switchover takes effect, this operation is irreversible in the enterprise management console. If you want to switch back to the Lingma built-in storage service, you need to submit a ticket for processing.

4. After the service switchover configuration takes effect, when you upload data to the knowledge base, Lingma will upload the knowledge data to the OSS configured by the administrator.

FAQ

If a knowledge base file fails to be uploaded after you use a self-managed storage service, the connection to the self-managed storage service may be abnormal. Check whether the connection to the self-managed storage service is normal on the Knowledge base configuration page.