Dataphin:Create and manage smart analysis collections

Background information

A smart analysis collection is a set of resources that provides intelligent and reusable analysis capabilities. It allows different users in an enterprise to analyze problems and retrieve data using natural language. The collection combines data assets, SQL analysis skills, and data retrieval. This helps your enterprise quickly implement intelligent analysis in a standard and configurable manner.

• Purpose of an analysis collection: An analysis collection organizes large and scattered enterprise data. This helps the system locate the required data, which improves accuracy and simplifies data management. Searching for data without a collection is like looking for a book in a library without a catalog. The process is slow and prone to errors. Organizing data into collections is similar to arranging books on specific shelves, which makes locating them faster and more accurate.

• Advantages of an analysis collection:

  • Provide common questions to guide users.

  • Curate data tables by scenario. This avoids the need to search through all your data.

  • Use listed data assets. This helps the large language model (LLM) better understand the asset content, such as display names, descriptions, properties, tags, and relationships.

  • Set model prompts. You can provide the LLM with specific instructions or guidance to control the content and style of the output. These prompts improve the model's reasoning to meet your business needs more accurately.

Prerequisites

X-Analysis is turned on to use the Smart Analysis album.

Limits

• Only the MaxCompute compute engine supports the configuration of smart analysis collections.

• You can create a maximum of 200 collection groups in a single tenant.

• You can create a maximum of 1,000 analysis collections in a single collection group.

Permission description

• Super administrators, system administrators, operations administrators, and custom global roles with the Smart Analysis Collection - Manage permission can create and manage all collection groups and analysis collections.

• Custom global roles with the Smart Analysis Collection - Create permission can create and manage their own collections within existing collection groups.

• Users with asset maintenance permissions can add data assets.

Create a collection group

1. From the top menu bar on the Dataphin home page, choose Super X > Smart Application Management.

2. In the navigation pane on the left, choose Smart Application Management > Smart Analysis Collection. In the Collection Group list, click the icon to open the Create Collection Group dialog box.

3. Configure the parameters in the Create Collection Group dialog box.

   Parameter	Description
   Name	Enter a name for the collection group. The name must be unique and cannot exceed 128 characters.
   Description	Enter background information for the collection group. This helps others quickly understand the collection. The description cannot exceed 512 characters.

4. Click OK to complete the collection group configuration.

Create an analysis collection

1. On the Analysis Collection List page, click the Create Collection button.

2. On the Create Collection page, you can configure the parameters.

   Parameter		Description
   Basic Information	Collection Name	Enter a name for the collection. The name must be unique within its collection group and cannot exceed 128 characters.
   	Description	Enter a description for the collection. This helps others understand its scenarios and other related information. The description cannot exceed 1024 characters.
   	Collection Group	The current collection group is selected by default. You can select another existing collection group. You can also search for groups by name.
   	Owner	Select a user with a Normal status in the current tenant. This user is responsible for the routine management of the collection, such as creating and editing it.
   	Sample Questions	Pre-set common questions that can be used directly in AI chat. Click the Add Question button to add a question. Each question can be up to 128 characters long. You can add up to six questions.
   Model Prompt		Enter skill instructions and thought guidance specific to this collection. The prompt cannot exceed 5,000 characters. You can add extra information or SQL code standards needed for AI chat based on this collection, such as sample content.
   Data Asset	Data Source	Select the data source to add to the current collection. You can select Dataphin tables or tables from a specified data source. Cross-source analysis is not supported. For more information about supported data source types, see Operations supported by different types of data sources.
   	Data Table	Select data assets from the data source. This provides available data assets for AI chat within this collection. You can select listed assets for which you have maintenance permissions. You can click the Filter icon to filter assets by business segment/project or table type. You can add up to 20 tables.
   	Data Table List	Displays the name, description, and last sampling time of the selected data assets. Use Sample Data: Using sample data helps the LLM better understand the data assets and improves analysis accuracy. To enable this feature, see Smart Application Management.
   Term	Associate Knowledge Base	Select terms related to the current collection from a knowledge base. This provides consistent semantic explanations for AI chat within this collection. You can select knowledge bases that you have joined and that support the term type. The system automatically associates all terms from the selected knowledge base. Future updates to the terms will be synced to the associated collection. Note If you associate multiple knowledge bases that contain terms with the same name but different meanings, it may affect the model's reasoning accuracy. Add knowledge bases that are appropriate for your business.
   Business Document	Business Document	Business knowledge can supplement the analysis collection. It helps the large model better understand the analysis scenarios served by the collection. You can directly associate business documents uploaded to the enterprise knowledge base. You can select documents from knowledge bases that you have joined and that support the business document type. You can search for documents by name or description. Alternatively, you can upload a local document. Click the File button to select a document from your local machine. We recommend associating only business documents relevant to the current collection. A single collection can be associated with up to 20 documents.
   	Document List	Displays the name, description, and knowledge base folder information for associated documents. You can delete or download the documents.
   Sample Code		Pre-set common questions and their corresponding SQL logic. This can guide the model in generating SQL, improving its standardization and accuracy. Click the Add Sample Code button to add a new configuration set (question + sample code). The question can be up to 128 characters long, and the sample code can be up to 1,000 characters long. You can add up to five configuration sets. Click the Delete icon to remove a set.
   Execute Data Retrieval		Configure computing resources based on the data source. This is enabled by default but can be disabled. Computing Resource: This can be configured only when the data source is Dataphin or another data source. If the data source is Dataphin, select the computing resource for running data asset tasks. You can choose the built-in analysis platform project or a specific project. Ensure that the account configured for the selected compute source has read permissions on the relevant data tables. If the data source is another data source, its resources are used by default and cannot be modified. Analysis Platform: You must initialize the analysis and consumption platform before you can select this option. SQL tasks will run on the compute source configured in the analysis platform. Specific Project: Tasks for the selected data assets will run in the specified production project.
   Usage Permission	Available Personnel	Select the users who can use this collection. You can select all users, or specific users and user groups. We recommend that users have view permissions on the tables added to the collection. This prevents analysis from being limited due to an inability to view some table details, which could affect the accuracy of SQL generation. All Users: All users in the tenant can use the collection. Specific Users And User Groups: You can specify users and user groups. The total number of members from the selected users and user groups cannot exceed 500. User: Select users with a Normal status in the current tenant. User Group: You can select user groups that are in a Normal and enabled state. The members of the collection are automatically updated when the user group members change. For more information, see Create and manage user groups.

3. Click OK to create the collection.

Manage collection groups and collections

1. On the Smart Analysis Collection page, you can view information about collection groups and collections. When you select a collection group on the left, the list on the right automatically displays the collections in that group. The list displays the name, description, last updater, last update time, and status for each collection. You can also click All Collections to view all collections.

2. You can search for collections by name or description.

3. You can perform the following operations on the target collection groups and collections.

   Operation		Description
   Collection Group	Edit	You can modify the name and description of the collection group, similar to the creation process.
   	Delete	If a collection group contains collections, you must delete the collections before you can delete the group. This operation cannot be undone. Proceed with caution.
   Collection List	Status	Only enabled analysis collections can be used for X-Analysis.
   	Edit	The user performing the operation can delete all added data tables. They can only add new assets for which they have maintenance permissions.
   	Go to Analysis	Click to go to the X-Analysis page, where you can perform AI chat based on the current collection.
   	Delete	Deleting a collection cannot be undone. Proceed with caution.