OpenSearch:Segmentación de documentos

# Extract the chunking results. Note that only ["chunks"] and ["rich_texts"] (excluding images) are used.
doc_list = []
for chunk in document_split_result.body.result.chunks:
    doc_list.append({"id": chunk.meta.get("id"), "content": chunk.content})
for rich_text in document_split_result.body.result.rich_texts:
    if rich_text.meta.get("type") != "image":
        doc_list.append({"id": rich_text.meta.get("id"), "content": rich_text.content})

After you successfully enable the AI Search Open Platform service for the first time, the system automatically creates a default workspace: Default.
Click Create Workspace. Enter a custom workspace name, and then click Confirm. After you click Create new API key, the system generates an API key. You can then click the Copy button to copy and save the API key.

root (6b15)
  |
  +-- paragraph_node (557b)
       |
       +-- newline_node (ef4d)[After you successfully enable the AI Search Open Platform...Default.]
       |
       +-- newline_node (c618)
            |
            +-- sentence_node (98ce)[Click Create Workspace...and then click Confirm.]
            |
            +-- sentence_node (922a)[After you click Create new API key...to copy and save the API key.]

{host}/v3/openapi/workspaces/{workspace_name}/document-split/{service_id} 

curl -XPOST -H"Content-Type: application/json"  
"http://***-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/document-split/ops-document-split-001"  
-H "Authorization: Bearer YOUR_API_KEY"  
-d "{
    \"document\":{
          \"content\":\"Product Advantages\\nIndustry Algorithm Edition\\nIntelligent\\nFeatures a rich set of customizable algorithm models and industry-specific recall and ranking algorithms to ensure superior search results.\\n\\nFlexible and Customizable\\nDevelopers can customize algorithm models, application structures, data processing, query analysis, and ranking configurations based on their business characteristics and data. This meets personalized search needs, increases click-through rates, enables rapid business iteration, and significantly shortens the time-to-market for new features.\\n\\nSecure and Stable\\nProvides 24/7 operational maintenance and technical support through online tickets and phone support. A comprehensive incident response mechanism includes fault monitoring, automatic alerts, and rapid issue identification. Access control and isolation are enforced at the API level by using Alibaba Cloud AccessKeyId and AccessKeySecret security pairs, ensuring user-level data segregation and security. Data is redundantly backed up to prevent loss.\\n\\nElastic Scaling\\nResources can be elastically scaled up or down as needed.\\n\\nRich Peripheral Features\\nSupports a range of peripheral search features such as hot searches, search suggestions, and statistical reports for convenient display and analysis.\\n\\nOut-of-the-Box\\nNo need to deploy or maintain clusters. Quickly access a one-stop search service.\\n\\nHigh-Performance Retrieval Edition\\nHigh Throughput\\nSupports tens of thousands of write TPS for a single table, with updates in seconds.\\n\\nSecure and Stable\\nProvides 24/7 operational maintenance and technical support through online tickets and phone support. A comprehensive incident response mechanism includes fault monitoring, automatic alerts, and rapid issue identification. Access control and isolation are enforced at the API level by using Alibaba Cloud AccessKeyId and AccessKeySecret security pairs, ensuring user-level data segregation and security. Data is redundantly backed up to prevent loss.\\n\\nElastic Scaling\\nResources can be elastically scaled up or down as needed.\\n\\nOut-of-the-Box\\nNo need to deploy or maintain clusters. Quickly access a one-stop search service.\\n\\nVector Retrieval Edition\\nStable\\nThe underlying implementation in C++ has been developed for over a decade and supports multiple core businesses, making it highly stable and suitable for mission-critical search scenarios.\\n\\nEfficient\\nA distributed search engine that efficiently supports massive data retrieval and real-time data updates (effective in seconds), making it ideal for search scenarios that are sensitive to query latency and timeliness.\\n\\nCost-Effective\\nSupports multiple index compression strategies and multi-value index loading tests to meet user query needs at a lower cost.\\n\\nVector Algorithms\\nSupports vector retrieval for various unstructured data such as voice, images, videos, text, and behaviors.\\n\\nSQL Queries\\nSupports SQL syntax and online multi-table joins. Provides a rich set of built-in UDFs and a UDF customization mechanism to meet diverse retrieval needs. SQL Studio is integrated into the operations system for easy SQL development and testing.\\n\\nRecall Engine Edition\\nStable\\nThe underlying implementation in C++ has been developed for over a decade and supports multiple core businesses, making it highly stable and suitable for mission-critical search scenarios.\\n\\nEfficient\\nThe Wentian Engine is a distributed search engine that efficiently supports massive data retrieval and real-time data updates (effective in seconds), making it ideal for search scenarios that are sensitive to query latency and timeliness.\\n\\nCost-Effective\\nThe Wentian Engine supports multiple index compression strategies and multi-value index loading tests to meet user query needs at a lower cost.\\n\\nFeature-Rich\\nThe Wentian Engine supports various analyzer types, index types, and powerful query syntax to meet user retrieval needs. A plugin mechanism is also provided for customizing business logic.\\n\\nSQL Queries\\nThe Wentian Engine supports SQL syntax and online multi-table joins. Provides a rich set of built-in UDFs and a UDF customization mechanism to meet diverse retrieval needs. SQL Studio will soon be integrated into the operations system for easy SQL development and testing.\",
          \"content_encoding\":\"utf8\",\"content_type\":\"text\"
    },
    \"strategy\":{
          \"type\":\"default\",
          \"max_chunk_size\":300,
          \"compute_type\":\"token\",
          \"need_sentence\":false
    }
}"

{
	"request_id": "47EA146B-****-448C-A1D5-50B89D7EA434",
	"latency": 161,
	"usage": {
		"token_count": 800
	},
	"result": {
		"chunks": [
			{
				"content": "Product Advantages\\nIndustry Algorithm Edition\\nIntelligent\\nFeatures a rich set of customizable algorithm models and industry-specific recall and ranking algorithms to ensure superior search results.\\n\\nFlexible and Customizable\\nDevelopers can customize algorithm models, application structures, data processing, query analysis, and ranking configurations based on their business characteristics and data. This meets personalized search needs, increases click-through rates, enables rapid business iteration, and significantly shortens the time-to-market for new features.\\n\\nSecure and Stable\\nProvides 24/7 operational maintenance and technical support through online tickets and phone support. A comprehensive incident response mechanism includes fault monitoring, automatic alerts, and rapid issue identification. Access control and isolation are enforced at the API level by using Alibaba Cloud AccessKeyId and AccessKeySecret security pairs, ensuring user-level data segregation and security. Data is redundantly backed up to prevent loss.\\n\\nElastic Scaling\\nResources can be elastically scaled up or down as needed.\\n\\nRich Peripheral Features\\nSupports a range of peripheral search features such as hot searches, search suggestions, and statistical reports for convenient display and analysis.\\n\\nOut-of-the-Box\\nNo need to deploy or maintain clusters. Quickly access a one-stop search service.\\n\\nHigh-Performance Retrieval Edition\\nHigh Throughput\\nSupports tens of thousands of write TPS for a single table, with updates in seconds",
				"meta": {
					"parent_id": "dee776dda3ff4b078bccf989a6bd****",
					"id": "27eea7c6b2874cb7a5bf6c71afbf****",
					"type": "text"
				}
			},
			{
				"content": ".\\n\\nSecure and Stable\\nProvides 24/7 operational maintenance and technical support through online tickets and phone support. A comprehensive incident response mechanism includes fault monitoring, automatic alerts, and rapid issue identification. Access control and isolation are enforced at the API level by using Alibaba Cloud AccessKeyId and AccessKeySecret security pairs, ensuring user-level data segregation and security. Data is redundantly backed up to prevent loss.\\n\\nElastic Scaling\\nResources can be elastically scaled up or down as needed.\\n\\nOut-of-the-Box\\nNo need to deploy or maintain clusters. Quickly access a one-stop search service.\\n\\nVector Retrieval Edition\\nStable\\nThe underlying implementation in C++ has been developed for over a decade and supports multiple core businesses, making it highly stable and suitable for mission-critical search scenarios.\\n\\nEfficient\\nA distributed search engine that efficiently supports massive data retrieval and real-time data updates (effective in seconds), making it ideal for search scenarios that are sensitive to query latency and timeliness.\\n\\nCost-Effective\\nSupports multiple index compression strategies and multi-value index loading tests to meet user query needs at a lower cost.\\n\\nVector Algorithms\\nSupports vector retrieval for various unstructured data such as voice, images, videos, text, and behaviors.\\n\\nSQL Queries\\nSupports SQL syntax and online multi-table joins. Provides a rich set of built-in UDFs and a UDF customization mechanism to meet diverse retrieval needs",
				"meta": {
					"parent_id": "dee776dda3ff4b078bccf989a6bd****",
					"id": "bf9fcfb47fcf410aa05216e268df****",
					"type": "text"
				}
			},
			{
				"content": ". SQL Studio is integrated into the operations system for easy SQL development and testing.\\n\\nRecall Engine Edition\\nStable\\nThe underlying implementation in C++ has been developed for over a decade and supports multiple core businesses, making it highly stable and suitable for mission-critical search scenarios.\\n\\nEfficient\\nThe Wentian Engine is a distributed search engine that efficiently supports massive data retrieval and real-time data updates (effective in seconds), making it ideal for search scenarios that are sensitive to query latency and timeliness.\\n\\nCost-Effective\\nThe Wentian Engine supports multiple index compression strategies and multi-value index loading tests to meet user query needs at a lower cost.\\n\\nFeature-Rich\\nThe Wentian Engine supports various analyzer types, index types, and powerful query syntax to meet user retrieval needs. A plugin mechanism is also provided for customizing business logic.\\n\\nSQL Queries\\nThe Wentian Engine supports SQL syntax and online multi-table joins. Provides a rich set of built-in UDFs and a UDF customization mechanism to meet diverse retrieval needs. SQL Studio will soon be integrated into the operations system for easy SQL development and testing.",
				"meta": {
					"parent_id": "dee776dda3ff4b078bccf989a6bd****",
					"id": "26ab0e4f7665487bb0a82c5a226a****",
					"type": "text"
				}
			}
		],
		"nodes": [
			{
				"id": "dee776dda3ff4b078bccf989a6bd****",
				"type": "root",
				"parent_id": "dee776dda3ff4b078bccf989a6bd****"
			},
			{
				"id": "27eea7c6b2874cb7a5bf6c71afbf****",
				"type": "sentence",
				"parent_id": "dee776dda3ff4b078bccf989a6bd****"
			},
			{
				"id": "bf9fcfb47fcf410aa05216e268df****",
				"type": "sentence",
				"parent_id": "dee776dda3ff4b078bccf989a6bd****"
			},
			{
				"id": "26ab0e4f7665487bb0a82c5a226a****",
				"type": "sentence",
				"parent_id": "dee776dda3ff4b078bccf989a6bd****"
			}
		],
		"rich_texts": []
	}
}

{
    "request_id": "817964CD-1B84-4AE1-9B63-4FB99734****",
    "latency": 0,
    "code": "InvalidParameter",
    "message": "JSON parse error: Invalid UTF-8 start byte 0xbc; nested exception is com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0xbc\n at line: 2, column: 19]"
}