Scalable Vector Search Benchmarking

Scalable Vector Search Benchmarking enables the benchmarking or evaluation of the Scalable Vector Search library.

Installation

Requires Python >= 3.12.

python -m pip install \
    git+https://github.com/IntelLabs/ScalableVectorSearchBenchmarking

Usage

Building an index

python -m svsbench.build \
    --vecs_file /path/to/vectors.fvecs \
    --svs_type leanvec4x8 --leanvec_dims -4 \
    --proportion_vectors_init 0.5 --batch_size 10000

Computing the ground truth

For the query vectors used in performance measurements:

python -m svsbench.generate_ground_truth \
    --vecs_file vectors.fvecs \
    --query_file query_vectors.fvecs

For the query vectors used in the calibration of search parameters:

python -m svsbench.generate_ground_truth \
    --vecs_file vectors.fvecs \
    --query_file calibration_query_vectors.fvecs

Searching

Calibrating the search parameters for a given recall and then searching:

python -m svsbench.search \
    --idx_dir /path/to/index_dir_from_build \
    --svs_type leanvec4x8 \
    --query_file /path/to/query_vectors.fvecs \
    --ground_truth_file /path/to/ground_truth.ivecs \
    -k 5 \
    --recall 0.95 \
    --calibration_query_file /path/to/calibration_query_vectors.fvecs \
    --calibration_ground_truth_file /path/to/calibration_ground_truth.ivecs

Searching using specified search parameters:

python -m svsbench.search \
    --idx_dir /path/to/index_dir_from_build \
    --svs_type leanvec4x8 \
    --query_file /path/to/query_vectors.fvecs \
    --ground_truth_file /path/to/ground_truth.ivecs \
    -k 5 \
    --batch_size 1 \
    --search_window_size 14 \
    --search_buffer_capacity 34 \
    --prefetch_lookahead 1 \
    --prefetch_step 0 \
    --batch_size 10000 \
    --search_window_size 15 \
    --search_buffer_capacity 36 \
    --prefetch_lookahead 10 \
    --prefetch_step 4