Evaluate HGraph Index QPS & Recall with VectorDBBench - Hologres

VectorDBBench is an open-source benchmarking tool for vector databases. This topic shows how to run a performance benchmark against Hologres using VectorDBBench, so you can measure queries per second (QPS) and latency across different concurrency levels.

Prerequisites

Before you begin, ensure that you have:

A Hologres V4.0.1 instance (32-core CPU, 128 GB memory) with a database created. See Purchase a Hologres V4.0 instance and create a database
A Hologres user account. See User management
An Elastic Compute Service (ECS) instance (ecs.c9i.16xlarge or higher) in the same region, zone, and Virtual Private Cloud (VPC) as the Hologres instance
Git and Python 3.11 installed on the ECS instance

Hologres instances are deployed using Kubernetes containerization. Because QPS for vector retrieval is extremely high, actual performance may vary depending on network conditions and payload.

Test environment

The benchmark runs on a two-machine setup: the ECS instance acts as the test client, and the Hologres instance is the system under test.

Component	Specification
Hologres instance	32-core CPU, 128 GB memory, V4.0.1
ECS instance	ecs.c9i.16xlarge or higher
Network	Same region, zone, and VPC
Vector index	HGraph (Hologres's vector index)

The ECS and Hologres instances must share the same VPC so the benchmark client connects over the private network endpoint, not the public internet.

Set up the environment

Install Git

If Git is not already installed on the ECS instance, run:

apt-get update
apt install git

Install Python 3.11

If Python 3.11 is not already installed, run:

apt-get update
apt install python3-full python3-venv

cd /opt
python3 -m venv venv
source venv/bin/activate

Install VectorDBBench

Run the following commands on the ECS instance to install VectorDBBench with Hologres support:

git clone https://github.com/zilliztech/VectorDBBench.git
cd VectorDBBench

pip3 install -U pip -i https://mirrors.aliyun.com/pypi/simple
pip3 install --no-cache-dir -r install/requirements_py3.11.txt -i https://mirrors.aliyun.com/pypi/simple
pip3 install -e . -i https://mirrors.aliyun.com/pypi/simple

To see all available options for Hologres, run:

vectordbbench hologreshgraph --help

Run the benchmark

The following command runs a full benchmark against Hologres using a 768-dimensional, 10 million-vector dataset. It tests multiple concurrency levels (1 to 120) to show how QPS and latency change as the load increases.

NUM_PER_BATCH=100000 vectordbbench hologreshgraph \
  --host <hologres_instance_vpc_endpoint> \
  --port <port> \
  --user '<username>' \
  --password <password> \
  --database <database_name> \
  --m 64 \
  --ef-construction 400 \
  --case-type Performance768D10M \
  --index-type HGraph \
  --ef-search 400 \
  --k 10 \
  --num-concurrency 1,60,70,75,80,90,95,100,105,110,115,120 \
  --db-label 32c128g-4.0-hybrid-index

Replace the following placeholders before running:

Placeholder	Description	Where to find it
`<hologres_instance_vpc_endpoint>`	VPC endpoint hostname of the Hologres instance	Hologres management console > instance ID > Instance Details > Network Information
`<port>`	Port number of the Hologres instance	Same location as the endpoint above
`<username>`	Hologres username	Hologres management console > instance ID > Instance Details > Users
`<password>`	Password for the Hologres user	Same location as the username above
`<database_name>`	Name of the target database	Hologres management console > instance ID > Instance Details > Database Management

What the concurrency levels measure

The --num-concurrency parameter controls how many parallel client threads send requests simultaneously.

Concurrency = 1: Measures single-request latency — how fast Hologres responds to one request at a time. This reflects real-time or latency-sensitive use cases.
Concurrency > 1: Measures throughput (QPS) — how many requests per second Hologres can handle under parallel load. This reflects high-concurrency web application use cases.

Running multiple concurrency levels in one command (for example, 1,60,70,...,120) gives you a full picture of the performance curve, so you can match the results to your actual workload.

Index and search parameters

Parameter	Value	Description
`--m`	64	Number of connections per node in the HGraph index. Higher values improve recall but increase index size and build time.
`--ef-construction`	400	Search depth during index construction. Higher values improve index quality at the cost of longer build time.
`--ef-search`	400	Search depth during query. Higher values improve recall but increase query latency.
`--case-type`	Performance768D10M	Benchmark dataset: 768-dimensional vectors, 10 million records.
`--index-type`	HGraph	Hologres's vector index type.
`--k`	10	Number of nearest neighbors to retrieve per query.
`--num-concurrency`	1,60,70,75,80,90,95,100,105,110,115,120	Comma-separated list of concurrency levels to test sequentially.
`--db-label`	32c128g-4.0-hybrid-index	Label for this test run, used to identify results in the report.
`NUM_PER_BATCH`	100000	Number of vectors loaded per batch during data import.

Re-run without reloading data

If you run the benchmark again after the first run, add --skip-drop-old --skip-load to skip data import and reuse the existing dataset:

NUM_PER_BATCH=100000 vectordbbench hologreshgraph \
  --host <hologres_instance_vpc_endpoint> \
  --port <port> \
  --user '<username>' \
  --password <password> \
  --database <database_name> \
  --m 64 \
  --ef-construction 400 \
  --case-type Performance768D10M \
  --index-type HGraph \
  --ef-search 400 \
  --k 10 \
  --num-concurrency 1,60,70,75,80,90,95,100,105,110,115,120 \
  --db-label 32c128g-4.0-hybrid-index \
  --skip-drop-old --skip-load

--skip-drop-old: Keeps the existing table instead of dropping and recreating it.
--skip-load: Skips the data import step and queries the existing data.

What's next

If you have questions or want to share feedback, join the Hologres communication group. For details on how to join, see How can I get more online support?.