Batch Compute

Resource management and task scheduling for large-scale batch processing

Batch Compute is a cloud service for massive simultaneous batch processing. The system automatically manages resources, schedules tasks, loads data, and bills by usage. Batch Compute is already widely used in scenarios such as scientific applications, genetic data analysis, CGI rendering, media transcoding, and financial data analysis.

Benefits

Batch Processing at Scale
Accelerate your task processing with tens of thousands of cores.
Easy to Use
Submit tasks with simple commands or API calls, with resource management and job scheduling automatically handle tasks.
Distributed Caching Technology
Expedite large-scale shared data access with unique distributed caching technology.
Rich Application Scenarios
Offer end-to-end solutions for application scenarios including media rendering and genetic data analysis.

Features

Flexible Resource Management and Task Scheduling

Batch Compute dynamically creates and releases VM instances according to task requirements, enabling you to launch thousands of instances with a single command or API call.For reserved clusters, Batch Compute enables you to meet flexible task management goals with priority-based task scheduling.

Compose Workflows with Directed Acyclic Graph

Batch Compute allows you to construct complex workflows by specifying dependencies between different tasks. It also offers a CLI to help you compose workflows with easy-to-use configuration files.

I/O Acceleration with Distributed Data Caching

Batch Compute can accelerate shared data access with its unique distributed caching technology. It dramatically improves the overall performance of tasks with a large number of simultaneous instances that access common data on OSS or NAS.

Access Object Storage with File System Interface

Batch Compute enables users to mount OSS paths to the local file system within a VM instance. With this feature, programs can operate data on OSS as standard files without the need for extra programming.

Reduce Compute Cost with Spot Instances

Batch Compute allows you to leverage spot instances to reduce computing cost. If a spot instance is released before a task is finished, the task will automatically rerun to save you management overhead.

Highly Customizable Compute Environments

Batch Compute allows you to deploy programs through Docker or VM images and you can customize computing environments anyway you want.

Application Scenarios

  • Genetic Data Analysis
CGI Rendering

CGI Rendering

Improve Rendering Performance

Batch Compute offers highly efficient resource management and task scheduling services, which enables users to manage tens of thousands of cores efficiently. With unique distributed caching technology, Batch Compute can accelerate concurrent data loading performance from hundreds or thousands of nodes.

Features

  • Distributed Caching

    You can simply enable distributed caching for OSS or NAS. The caching capability scales linearly with the size of the cluster, removing I/O bottleneck for large-scale distributed rendering.

  • Easy to Use

    Batch Compute offers cluster-level and job-level APIs. You can choose to deploy your task scheduler or take advantage of Batch's default scheduling. Also, Batch Compute provides a Web-based rendering solution, which enables you to perform easy cloud-based rendering.

  • Economic

    You only pay for the computing and storage resources used by the rendering tasks. There is no additional charge for task scheduling and distributed caching, saving you from license fees of traditional rendering management software.

Related Products & Services

Genetic Data Analysis

Genetic Data Analysis

Population-scale Sample Analysis

Batch Compute supports complex workflow construction through the directed acyclic graph mode. Combined with distributed caching and OSS mount features, Batch Compute enables you to easily build population-scale sample analysis platform on the cloud.

Features

  • Workflow Management

    Batch Compute allows you to construct workflows by specifying dependencies between tasks. The system guarantees the execution order. You can also speed up your workflow with the systems parallel processing capability.

  • Supports Multiple Data Sources

    Batch Compute support both OSS and NAS. With the OSS mount feature, Batch Compute enables you to access object storage with a file system interface without modifying your existing software.

  • Easy to Use

    Batch Compute offers an easy-to-use, powerful CLI. You can build workflows by editing configuration files, and submit and manage tasks with minimal programming.

Related Products & Services