Platform for AI (PAI) - EAS supports LLM Intelligent Router to improve LLM inference efficiency
Aug 16 2024
Platform for AI (PAI)Content
Intended customers: Customers who use EAS to build LLM-driven applications and services, such as intelligent customer service, content generation, and translation. LLM Intelligent Router can improve throughput and reduce latency, helping customers process user requests efficiently and stably. LLM Intelligent Router can improve throughput and reduce latency, helping customers process user requests efficiently and stably. New features: When customers deploy LLM services on EAS, they can enable the LLM Intelligent Router feature. LLM Intelligent Router can evenly allocate the computing power and video memory of backend inference instances and improve the resource usage of clusters.
Help Document
https://www.alibabacloud.com/help/pai/user-guide/use-llm-intelligent-router-to-improve-inference-efficiency
