New Features

API Gateway - The AI Gateway feature is available

Nov 29 2024

API Gateway
AI Gateway supports policies such as Large Language Model (LLM) proxy, observability, service fallback, content moderation, and token-based throttling.
Content

1. Target customers: enterprises that need to call or manage LLM APIs Features: a. Support for AI Proxy in which multiple LLM servies can be proxied based on one set of API standard that is compatible with OpenAI b. Support for automatic routing to different model services based on model name, request ratio, and request characteristics (such as by header) c. Support for automatic fallback to other LLM services when one LLM service is unavailable d. Support for AI API debugging e. Support for unified authorization by API consumer and for consumer call monitoring and quota management (token-based throttling) f. Integration with Alibaba Cloud observability services for unified monitoring of AI request links g. Integration with Alibaba Cloud content moderation services to meet enterprise demands for regulatory compliance, security, and sensitive information protection h. Support for caching AI requests and responses to reduce LLM costs, including simple caching and semantic-based caching

Help Document

https://www.alibabacloud.com/help/api-gateway/cloud-native-api-gateway/product-overview/what-is-an-ai-gateway

7th Gen ECS Is Now Available

Increase instance computing power by up to 40% and Fully equipped with TPM chips.
Powered by Third-generation Intel® Xeon® Scalable processors (Ice Lake).

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.