This repo contains the associated raw benchmark results of the benchANT blog post Vector Search Performance Benchmark of SingleStore, Pinecone and Zilliz.
The results folder contains the data for the six benchmark cases:
SingleStore S2SingleStore S4Pinecone (S2 price equal)Pinecone (S4 price equal)Zilliz (recommended)Zilliz (S4 price equal)
For more details on the method and the benchmark objectives please refer to the blog post.
Each folder contains the data of a single run benchmark of one configuration.
In order to ensure full transparency and reproducibility, each data folder contains benchmark configuration data, performance data, monitoring data, cloud provider metadata, VM metadata and DBMS configuration data.
All configurable benchmark parameters for each DBaaS and VectorDBBench are defined in the evaluationScenario.json.
The benchANT_versions contains the used versions of the benchANT software components to execute the benchmarks.
The execution logs of the individual benchmark steps are contained in airflowTaskInstanceDetails.json.
The raw performance data output of the VectorDBBench is contained in the 0_load.txt for the LOAD, OPTIMIZE and QUERY phase.
The DBMS cluster and the benchmark instances are monitored with Telegraf and the data is stored in InfluxDB.
A full snapshot of the monitoring data of each run is contained in the influx_data.zip file.
The DBaaS metadata for each DBMS deployment is available in dbaas_resources.json.
The cloud provider metadata for the VectorDBBench benchmark deployment in the benchmark_resources.json.
The VM metadata for the DBMS deployment is contained in the dbms_data_hardware_facts.json / dbms_management_hardware_facts.json and for the benchmark deployment in the benchmark_hardware_facts.json.
In case of questions or feedback on the data feel free to create an issue or reach out to [email protected]