"Serverless GPU" is an emerging cloud-based GPU service. Serverless GPUs provide on-demand GPU computing resources for you and you do not have to worry about the underlying infrastructure such as servers. Compared with resident GPU computing resources, serverless GPUs improve the resource utilization and elasticity and reduce costs. This topic describes the features and benefits of serverless GPUs.
With the traditional resident GPUs, you must plan resources in advance. The deployed computing resources are constantly running. This may lead to idle and wasted GPU resources in off-peak hours. Serverless GPUs provide a more flexible way to use GPU computing resources. You need to only select the GPU type and configure the specifications of computing resources based on your business requirements. You can start and stop GPU applications at any time without the need to plan resource usage.
Serverless GPUs adopt measures to improve the utilization and elasticity of computing resources. For example, GPU computing resources can be quickly allocated and prepared by using the optimized end-to-end GPU start and stop feature. This allows you to start and stop a large number of GPU computing tasks in a short period of time. In addition, serverless GPUs can be used on a pay-as-you-go basis. You only pay for the GPU computing resources that you use. No extra costs are generated when at rest.
Serverless GPUs are highly flexible and efficient and allow you to use GPU computing resources on demand. Serverless GPUs can help you solve the issues of resource waste, high cost, and low elasticity caused by the long-term use of GPUs. Serverless GPUs provide you with GPU computing services in a more convenient and efficient way. Workloads in scenarios such as AI model inference, AI model training, audio and video acceleration and production, and graphics and image acceleration can be processed in an efficient manner.