AI Coding Assistant Lingma:Switch connections for the enterprise knowledge base storage service

Background information

Administrators of Lingma and global administrators of the organization (Dedicated Edition) can switch the knowledge base storage service from the built-in Lingma storage service to an enterprise self-managed storage service. This switch is configured in the knowledge base configuration of the Lingma console.

Prerequisites

• Network requirements:

  • If your self-managed storage service supports public network access, Lingma Enterprise Dedicated Edition can connect to it directly. To improve data security, you can configure an IP address whitelist to control access permissions.

  • If your self-managed storage service does not support public network access, you must configure a virtual private cloud (VPC) to connect Lingma Enterprise Dedicated Edition and your storage service. You must also enable reverse access to allow Lingma Enterprise Dedicated Edition to retrieve resources from your self-managed storage service.

• Self-managed storage service preparation:

  • Object Storage Service: For more information, see Alibaba Cloud OSS guide.

    Compatible products	Alibaba Cloud Object Storage Service (OSS), Amazon S3, Tencent Cloud COS, Huawei Cloud OBS, and open source MinIO
    Permission requirements	An authorized account must have the following permissions: List objects in a bucket (List): Lets users list all objects in a bucket. Read object content (Read): Lets users read the content of objects in a bucket. Upload new objects (Put): Lets users upload new objects to a bucket. Delete objects (Delete): Lets users delete objects in a bucket.
    Storage class	Standard (non-Cold Archive)
    Version	All versions are compatible
    Storage capacity	More than 10 GB of storage capacity (Can be expanded later as needed.)

  • Vector database: For more information, see Alibaba Cloud Elasticsearch configuration guide.

    Compatible products	Elasticsearch Enterprise Edition and Elasticsearch Open Source Edition
    Version	Version 8.9 or later
    Permission requirements	An authorized account must have the following permissions: Create index (create_index): Lets users create new indexes. If the create index request includes an alias to be added to the index, the manage permission is also required. Write documents (write,create_doc): Lets users perform all write operations on documents, including indexing, updating, deleting, batch operations, and updating mappings. Read documents (read): Lets users perform read-only operations, such as count, explain, get, mget, scripts, search, and scroll. Delete documents (delete): Lets users delete documents in an index. Read access to create_index, read, delete, and index: Ensures the user can perform these specific operations.
    CPU specification requirements	Minimum: 4 cores and 8 GB of memory. Recommended: 8 cores and 16 GB of memory. Optimal: 16 cores and 32 GB of memory.
    Storage capacity	More than 300 GB of storage capacity (Can be expanded later as needed.)
    Configure an instance access whitelist (Optional)	When you access an Alibaba Cloud Elasticsearch (ES) instance, it is recommended that you add the IP address of the device that requires access to the instance's whitelist. To get the egress IP address of the Lingma instance, submit a ticket.

Scenario

An enterprise knowledge base uses two types of storage: Object Storage Service (OSS) and a vector database. OSS stores uploaded files, such as documents or code packages. The vector database stores vectorized data from file segments.

Lingma supports migrating knowledge base storage services, including OSS and the vector database, to maximize the security and privacy of your knowledge base data. The following figure shows an overview of the migration process and post-migration usage:

1. First, a Lingma administrator or enterprise administrator configures public network or VPC access on the enterprise management page in the Alibaba Cloud Management Console, and then configures the storage service migration in the Lingma console.

2. After the configuration is complete, when a Lingma administrator uploads knowledge base data files, the files are saved to the configured self-managed Object Storage Service, and the vector data is saved to the configured self-managed vector database.

Procedure

1. Log on to the Alibaba Cloud Management Console and configure network access for Lingma Enterprise Dedicated Edition based on your requirements.

   Public network access

   If your self-managed storage service supports public network access, complete the public network access configuration for Lingma Enterprise Dedicated Edition. You can also configure an IP address whitelist to control access permissions and improve data security.

   VPC access

   If your self-managed storage service does not support public network access, you must configure a VPC to connect Lingma Enterprise Dedicated Edition and your storage service. You must also enable reverse access to allow Lingma Enterprise Dedicated Edition to retrieve resources from your self-managed storage service.

2. Log on to the Lingma enterprise console. On the Knowledge Base, click the Storage Migration button.

3. On the Knowledge base configuration page, configure the connections for your storage services, including Object Storage Service and the vector database. After you enter the required information, click the Test Connection button to test the connections. If the tests are successful, click the Save button. A message appears to confirm that the configuration is complete.

   Important

   The new configuration takes effect immediately. This action deletes all existing knowledge base content provided by Lingma. The knowledge base in the Lingma enterprise management console returns to an empty, initial state.

   Once the storage service switch is applied, this operation cannot be reversed in the enterprise management console. To switch back to the built-in Lingma storage service, you must submit a ticket.

4. After the new service configuration takes effect, you can upload data to the knowledge base. Lingma saves the knowledge data to the Object Storage Service that you configured.

FAQ

If you cannot upload files to the knowledge base after you switch to your self-managed storage service, there might be a connection issue. On the Knowledge base configuration page, check the storage configuration to verify that the connection is still active.