Batch Compute

Resource management and task scheduling for large-scale batch processing

Batch Compute is a cloud service for massive simultaneous batch processing. The system automatically manages resources, schedules tasks, loads data, and bills by usage. Batch Compute is already widely used in scenarios such as scientific applications, genetic data analysis, CGI rendering, media transcoding, and financial data analysis.

Benefits

Batch Processing at Scale
Accelerate your task processing with tens of thousands of cores.
Easy to Use
Submit tasks with simple commands or API calls, with resource management and job scheduling automatically handle tasks.
Distributed Caching Technology
Expedite large-scale shared data access with unique distributed caching technology.
Rich Application Scenarios
Offer end-to-end solutions for application scenarios including media rendering and genetic data analysis.

Features

  • Flexible Resource Management and Task Scheduling

    Automatic Cluster Management

    Batch Compute dynamically creates and releases VM instances according to task requirements, enabling you to launch thousands of instances with a single command or API call.


    Priority-based Task Scheduling

    For reserved clusters, Batch Compute enables you to meet flexible task management goals with priority-based task scheduling.

  • Compose Workflows with Directed Acyclic Graph

    Workflow Construction

    Batch Compute allows you to construct complex workflows by specifying dependencies between different tasks. It also offers a CLI to help you compose workflows with easy-to-use configuration files.

  • I/O Acceleration with Distributed Data Caching

    Distributed Caching

    Batch Compute can accelerate shared data access with its unique distributed caching technology. It dramatically improves the overall performance of tasks with a large number of simultaneous instances that access common data on OSS or NAS.

  • Access Object Storage with File System Interface

    OSS Mount Feature

    Batch Compute enables users to mount OSS paths to the local file system within a VM instance. With this feature, programs can operate data on OSS as standard files without the need for extra programming.

  • Reduce Compute Cost with Spot Instances

    Spot Instance

    Batch Compute allows you to leverage spot instances to reduce computing cost. If a spot instance is released before a task is finished, the task will automatically rerun to save you management overhead.

  • Highly Customizable Compute Environments

    Supports Both Windows and Linux

    Batch Compute allows you to deploy programs through Docker or VM images and you can customize computing environments anyway you want.

Application Scenarios

  • CGI Rendering
  • Genetic Data Analysis
CGI Rendering

Render at Any Scale

Improve Rendering Performance

Batch Compute offers highly efficient resource management and task scheduling services, which enables users to manage tens of thousands of cores efficiently. With unique distributed caching technology, Batch Compute can accelerate concurrent data loading performance from hundreds or thousands of nodes.

Features

  • Distributed Caching

    You can simply enable distributed caching for OSS or NAS. The caching capability scales linearly with the size of the cluster, removing I/O bottleneck for large-scale distributed rendering.

  • Easy to Use

    Batch Compute offers cluster-level and job-level APIs. You can choose to deploy your task scheduler or take advantage of Batch's default scheduling. Also, Batch Compute provides a Web-based rendering solution, which enables you to perform easy cloud-based rendering.

  • Economic

    You only pay for the computing and storage resources used by the rendering tasks. There is no additional charge for task scheduling and distributed caching, saving you from license fees of traditional rendering management software.

Related Products & Services

Genetic Data Analysis

Genetic Data Analysis

Population-scale Sample Analysis

Batch Compute supports complex workflow construction through the directed acyclic graph mode. Combined with distributed caching and OSS mount features, Batch Compute enables you to easily build population-scale sample analysis platform on the cloud.

Features

  • Workflow Management

    Batch Compute allows you to construct workflows by specifying dependencies between tasks. The system guarantees the execution order. You can also speed up your workflow with the systems parallel processing capability.

  • Supports Multiple Data Sources

    Batch Compute support both OSS and NAS. With the OSS mount feature, Batch Compute enables you to access object storage with a file system interface without modifying your existing software.

  • Easy to Use

    Batch Compute offers an easy-to-use, powerful CLI. You can build workflows by editing configuration files, and submit and manage tasks with minimal programming.

Related Products & Services

Pricing Overview

With Batch Compute, you pay for the compute and storage resources consumed by your jobs or clusters. There is no additional charge on resource management and job scheduling services. For auto-cluster jobs, VM instances are automatically created and released as needed by the jobs. For reserved clusters, VM instances are created and released as explicitly instructed by users. In both cases, the compute cost is calculated based on the uptime of VMs.

Please refer to Elastic Compute Service for pricing of compute resources, and OSS or NAS for pricing of storage resources.