API Gateway - The AI Gateway feature is available
Nov 29 2024
API GatewayContent
1. Target customers: enterprises that need to call or manage LLM APIs Features: a. Support for AI Proxy in which multiple LLM servies can be proxied based on one set of API standard that is compatible with OpenAI b. Support for automatic routing to different model services based on model name, request ratio, and request characteristics (such as by header) c. Support for automatic fallback to other LLM services when one LLM service is unavailable d. Support for AI API debugging e. Support for unified authorization by API consumer and for consumer call monitoring and quota management (token-based throttling) f. Integration with Alibaba Cloud observability services for unified monitoring of AI request links g. Integration with Alibaba Cloud content moderation services to meet enterprise demands for regulatory compliance, security, and sensitive information protection h. Support for caching AI requests and responses to reduce LLM costs, including simple caching and semantic-based caching
Help Document
https://www.alibabacloud.com/help/api-gateway/cloud-native-api-gateway/product-overview/what-is-an-ai-gateway