ApsaraDB RDS:Use the pgvector extension to test performance based on HNSW indexes

Step 1: Configure the RDS connection

Append the following settings to ~/ann-benchmarks/ann_benchmarks/algorithms/pgvector/module.py, replacing the placeholder values with your actual RDS connection details:

# RDS instance connection settings
os.environ['ANN_BENCHMARKS_PG_USER'] = 'ann_testuser'
os.environ['ANN_BENCHMARKS_PG_PASSWORD'] = 'testPawword'       # Replace with your password
os.environ['ANN_BENCHMARKS_PG_DBNAME'] = 'ann_testdb'
os.environ['ANN_BENCHMARKS_PG_HOST'] = 'pgm-****.pg.rds.aliyuncs.com'  # Replace with your internal endpoint
os.environ['ANN_BENCHMARKS_PG_PORT'] = '5432'
os.environ['ANN_BENCHMARKS_PG_START_SERVICE'] = 'false'        # Disable automatic startup

Step 2: Configure HNSW test parameters

Edit ~/ann-benchmarks/ann_benchmarks/algorithms/pgvector/config.yml to define the parameter combinations to test. This test covers three groups: M-16(100), M-16(200), and M-24(200).

float:
  any:
  - base_args: ['@metric']
    constructor: PGVector
    disabled: false
    docker_tag: ann-benchmarks-pgvector
    module: ann_benchmarks.algorithms.pgvector
    name: pgvector
    run_groups:
      M-16(100):
        arg_groups: [{M: 16, efConstruction: 100}]
        args: {}
        query_args: [[10, 20, 40, 80, 120, 200, 400, 800]]
      M-16(200):
        arg_groups: [{M: 16, efConstruction: 200}]
        args: {}
        query_args: [[10, 20, 40, 80, 120, 200, 400, 800]]
      M-24(200):
        arg_groups: [{M: 24, efConstruction: 200}]
        args: {}
        query_args: [[10, 20, 40, 80, 120, 200, 400, 800]]

Parameter descriptions:

Tuning direction at a glance:

• To increase recall rate: raise M, efConstruction, or ef_search.

• To reduce query latency: lower ef_search.

• To shorten index build time: lower efConstruction or increase parallel workers (see Appendix: Parameter impact on index build time).

• Default values (M=16, ef_construction=64, ef_search=40) often produce suboptimal recall. Tune from these defaults based on your requirements.

Step 3: Build the test Docker image

1. (Optional) Modify ~/ann-benchmarks/ann_benchmarks/algorithms/pgvector/Dockerfile to skip the built-in PostgreSQL setup and use only the psycopg and pgvector packages:

   FROM ann-benchmarks
   USER root
   RUN pip install psycopg[binary] pgvector

2. Build the Docker image:

   cd ~/ann-benchmarks/
   python install.py --algorithm pgvector

   Run python install.py --help to see all available options.

Step 4: Download the dataset

When you run the test script, ANN-Benchmarks automatically downloads the specified dataset. This test uses the nytimes-256-angular dataset. For details on available datasets, see ANN-Benchmarks datasets.

If the public datasets do not match your workload, convert your vector data to Hierarchical Data Format version 5 (HDF5) format and use it as a custom dataset. See Custom datasets.

Step 5: Run the test and collect results

1. Run the benchmark:

   Parameter	Description
   --dataset	Dataset to test
   -k	LIMIT value in the SQL query — number of results to return
   --algorithm	Algorithm to benchmark (pgvector in this test)
   --runs	Number of runs; the best result set is selected
   --parallelism	Number of parallel workers (default: 1)

   cd ~/ann-benchmarks
   nohup python run.py --dataset nytimes-256-angular -k 10 --algorithm pgvector --runs 1 > ann_benchmark_test.log 2>&1 &
   tail -f ann_benchmark_test.log

2. Generate result plots:

   cd ~/ann-benchmarks
   python plot.py --dataset nytimes-256-angular --recompute

3. (Optional) Export detailed results including recall rate, QPS, response time (RT), index build time, and index size:

   cd ~/ann-benchmarks
   python data_export.py --out res.csv

Recall rate and QPS

The following table shows results from the nytimes-256-angular dataset. All three groups use the same query_args values (ef_search = 10, 20, 40, 80, 120, 200, 400, 800).

Index build time

Effect of maintenance_work_mem

maintenance_work_mem sets the maximum memory for maintenance operations such as VACUUM and CREATE INDEX (unit: KB). Increasing this value shortens build time — but only up to the size of the dataset. Once maintenance_work_mem exceeds the dataset size, build time stops improving.

The following results use the pg.x8.2xlarge.2c instance type (16 cores, 128 GB memory), max_parallel_maintenance_workers=8, and the nytimes-256-angular dataset (~324 MB).

Build time plateaus between 512 MB and 1 GB because both values exceed the ~324 MB dataset size.

Effect of max_parallel_maintenance_workers

max_parallel_maintenance_workers controls the number of parallel workers for a single CREATE INDEX operation. Build time decreases as this value increases.

The following results use the same instance type, maintenance_work_mem=2048 (2 GB), and the nytimes-256-angular dataset.

Effect of vector dimensions

The following results use the GloVe dataset (1,183,514 rows), m=16, efConstruction=64, and ef_search=40, with maintenance_work_mem=8 GB and max_parallel_maintenance_workers=16.

As vector dimension increases, index build time increases, recall rate decreases, QPS decreases, and query latency increases.

P99 is the 99th percentile latency — 99% of all query requests complete within this time.

Effect of dataset size

The following results use the dbpedia-openai-{n}k-angular dataset with m=48, efConstruction=256, and ef_search=200.

As the number of rows increases, index build time increases nonlinearly, recall rate decreases, QPS decreases, and query latency increases.