E-MapReduce:StarRocks

What is StarRocks

• StarRocks is a next-generation Massively Parallel Processing (MPP) database designed for fast, unified analytics across diverse scenarios.

• StarRocks is MySQL-compatible. You can connect with MySQL clients and BI tools to analyze data.

• StarRocks uses a distributed architecture:

  • It horizontally partitions data tables and stores them as multiple replicas.

  • The cluster scales to handle up to 10 PB of data.

  • It uses MPP parallel computing to accelerate queries.

  • It supports multiple replicas to ensure fault tolerance.

Key features

StarRocks combines MPP and distributed system design. Key features include:

Simplified architecture

• StarRocks executes SQL queries through its MPP framework, leveraging all nodes in parallel for interactive analytics.

• StarRocks clusters have no external dependencies, simplifying deployment and maintenance while improving reliability.

Fully vectorized engine

The computation layer is fully vectorized. It uses columnar memory layout and SIMD instructions to maximize CPU parallelism, enabling sub-second multidimensional analytics.

Intelligent query optimization

StarRocks uses a cost-based optimizer (CBO) that estimates execution costs and generates optimal query plans without manual intervention, improving efficiency for ad hoc and ETL workloads.

Federated query

StarRocks supports federated queries on external tables. You can query Hive, MySQL, Elasticsearch, Iceberg, and Hudi data directly without importing it.

Efficient updates

StarRocks supports detail, aggregation, primary key, and update data models. The primary key model supports Upsert and Delete operations. Optimized storage and indexes enable efficient queries during concurrent updates, ideal for real-time data warehouses.

Intelligent materialized views

• You can create materialized views to pre-calculate aggregation tables and accelerate queries.

• Materialized views automatically aggregate data during import, staying consistent with the source table.

• StarRocks automatically selects the best materialized view for each query.

Standard SQL

• StarRocks supports standard SQL, including aggregation, JOINs, sorting, window functions, and user-defined functions.

• StarRocks fully supports all 22 TPC-H queries and all 99 TPC-DS queries.

• StarRocks is MySQL-compatible. You can use existing client tools and BI software for drag-and-drop analysis.

Unified stream and batch processing

• StarRocks supports both real-time and batch data import.

• Supported sources: Kafka, HDFS, and local files.

• Supported formats: ORC, Parquet, and CSV.

• StarRocks consumes Kafka data in real time with exactly-once semantics.

• Batch import is supported from local files and HDFS.

High availability and easy scaling

• StarRocks stores metadata and data as multiple replicas with hot standby services to prevent single points of failure.

• The cluster self-heals automatically. Node failures do not affect overall service stability.

• Storage and compute scale near-linearly. A single cluster supports hundreds of nodes and up to 10 PB of data.

• You can scale up or down without service interruption. Queries continue running during scaling.

• StarRocks supports online schema changes. You can add or drop columns and create materialized views while imports and queries continue.

Use cases