[RELEASE] Release version 2.9.0 #944

prudhvigodithi · 2023-06-27T00:33:32Z

This is a component issue for {REPLACE_WITH_RELEASE_VERSION}.
Coming from opensearch-project/opensearch-build#3616. Please follow the following checklist.
Please refer to the DATES in that post.

How to use this issue

This Component Release Issue

This issue captures the state of the OpenSearch release, on component/plugin level; its assignee is responsible for driving the release. Please contact them or @mention them on this issue for help.
Any release related work can be linked to this issue or added as comments to create visiblity into the release status.

Release Steps

There are several steps to the release process; these steps are completed as the whole component release and components that are behind present risk to the release. The component owner resolves the tasks in this issue and communicate with the overall release owner to make sure each component are moving along as expected.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release. The most current set of dates is on the overall release issue linked at the top of this issue.

The Overall Release Issue

Linked at the top of this issue, the overall release issue captures the state of the entire OpenSearch release including references to this issue, the release owner which is the assignee is responsible for communicating the release status broadly. Please contact them or @mention them on that issue for help.

What should I do if my plugin isn't making any changes?

If including changes in this release, increment the version on __{REPLACE_MAJOR_MINOR_PATCH}__ branch to __{REPLACE_MAJOR_MINOR_PATCH}__ for Min/Core, and __{REPLACE_MAJOR_MINOR_PATCH_0}__ for components. Otherwise, keep the version number unchanged for both.

Preparation

Assign this issue to a release owner.
Finalize scope and feature set and update the Public Roadmap.
All the tasks in this issue have been reviewed by the release owner.
Create, update, triage and label all features and issues targeted for this release with v__REPLACE_MAJOR_MINOR_PATCH__.

CI/CD

All code changes for __{REPLACE_MAJOR_MINOR_PATCH}__ are complete.
Ensure working and passing CI.
Check that this repo is included in the distribution manifest.

Pre-Release

Update to the __REPLACE_MAJOR_MINOR__ release branch in the distribution manifest.
Increment the version on the parent branch to the next development iteration.
Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
Confirm that all changes for __{REPLACE_MAJOR_MINOR_PATCH}__ have been merged.
Add this repo to the manifest for the next developer iteration.

Release Testing

Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
Sanity Testing: Sanity testing and fixing of critical issues found.
File issues for all intermittent test failures.

Release

Complete documentation.
Verify all issued labeled for this release are closed or labelled for the next release.
Verify the release date mentioned in release notes is correct and matches actual release date.

Post Release

Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
Add this repo to the manifest of the next eventual security patch version.
Suggest improvements to this template.
Conduct a retrospective, and publish its results.

The text was updated successfully, but these errors were encountered:

junqiu-lei · 2023-07-24T22:51:09Z

Performance Test Results 2.9

Configurations

Data Sets

Dataset	Train size	Dimensions	Distance	Format	K
sift	1,000,000	128	L2	hdf5	100

Total Queries: 10,000

Cluster Configuration

Config item	Value
Leader Nodes	3
Leader Node Type	c5.xlarge
Leader Node Disk Space	20
Data Nodes	3
Data Node Type	r5.4xlarge
Data Node Disk Space	500
Primary Shard Count	24
Replica Count	1
AZ	us-east-1

Client Configuration

Config item	Value
OS	Amazon Linux
Instance Type	c5.9xlarge
Count	1
EBS Size	800G
AZ	us-east-1

Visualizations

Perf Results Comparison Between 2.8.0 and 2.9.0

Ingest took total

Query took p50

Query took p99

Perf Results Comparison in 2.9.0

Query took total

Query took p50

Query took p99

Faiss filters

Lucene filters

Results

nmslib-hnsw

{
    "metadata": {
      "test_name": "index-workflow",
      "test_id": "Index workflow",
      "date": "07/20/2023 16:59:56",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "952307712 (used) / 72055181312 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 146577.0981901011,
      "delete_index_took_total": 611.5554230011185,
      "create_index_took_total": 343.65535489923786,
      "ingest_took_total": 57827.77802130004,
      "refresh_index_store_kb_total": 2890968.9713867186,
      "refresh_index_took_total": 39971.90939090069,
      "query_took_total": 47822.2,
      "query_took_p50": 5.0,
      "query_took_p90": 5.0,
      "query_took_p99": 6.0,
      "query_took_p99.9": 16.5,
      "query_took_p100": 41.0,
      "query_client_time_total": 81725.0,
      "query_client_time_p50": 8.0,
      "query_client_time_p90": 8.8,
      "query_client_time_p99": 9.7,
      "query_client_time_p99.9": 43.1,
      "query_client_time_p100": 258.5,
      "query_memory_kb_total": 1305837.5,
      "query_recall@K_total": 0.9998685,
      "query_recall@1_total": 1.0
    }
  }

faiss-hnsw

{
    "metadata": {
      "test_name": "Faiss HNSW Test",
      "test_id": "Faiss HNSW Test",
      "date": "07/20/2023 08:47:37",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "810930176 (used) / 72200650752 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 399815.01608850004,
      "delete_index_took_total": 264.56064920002973,
      "create_index_took_total": 198.03830149999158,
      "ingest_took_total": 57706.42572260003,
      "refresh_index_store_kb_total": 2916527.0458984375,
      "refresh_index_took_total": 37616.86943390001,
      "force_merge_took_total": 251333.92198130005,
      "query_took_total": 52695.2,
      "query_took_p50": 5.0,
      "query_took_p90": 5.1,
      "query_took_p99": 6.1,
      "query_took_p99.9": 10.0,
      "query_took_p100": 1115.1,
      "query_client_time_total": 86212.0,
      "query_client_time_p50": 8.0,
      "query_client_time_p90": 8.7,
      "query_client_time_p99": 9.5,
      "query_client_time_p99.9": 149.1,
      "query_client_time_p100": 1120.4,
      "query_memory_kb_total": 1294728.3,
      "query_recall@K_total": 0.9998886999999999,
      "query_recall@1_total": 1.0
    }
}

faiss-hnsw relaxed-filter

{
    "metadata": {
      "test_name": "Faiss HNSW Relaxed Filter Test",
      "test_id": "Faiss HNSW Relaxed Filter Test",
      "date": "07/20/2023 20:15:46",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "3215032320 (used) / 69707624448 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 216196.63614569238,
      "delete_index_took_total": 281.3794626999879,
      "create_index_took_total": 185.3892246945179,
      "ingest_multi_field_took_total": 61322.30727869901,
      "refresh_index_store_kb_total": 2959247.2978515625,
      "refresh_index_took_total": 35772.960179598886,
      "query_with_filter_took_total": 118634.6,
      "query_with_filter_took_p50": 11.9,
      "query_with_filter_took_p90": 13.2,
      "query_with_filter_took_p99": 14.4,
      "query_with_filter_took_p99.9": 24.3,
      "query_with_filter_took_p100": 54.7,
      "query_with_filter_client_time_total": 156684.5,
      "query_with_filter_client_time_p50": 15.3,
      "query_with_filter_client_time_p90": 16.8,
      "query_with_filter_client_time_p99": 18.4,
      "query_with_filter_client_time_p99.9": 56.1,
      "query_with_filter_client_time_p100": 417.2,
      "query_with_filter_memory_kb_total": 1296683.2,
      "query_with_filter_recall@K_total": 0.9998979,
      "query_with_filter_recall@1_total": 0.99997
    }
}

faiss-hnsw restrictive-filter

{
    "metadata": {
      "test_name": "Faiss HNSW Restrictive Filter Test",
      "test_id": "Faiss HNSW Restrictive Filter Test",
      "date": "07/20/2023 20:52:31",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "920506368 (used) / 72086974464 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 400951.101299801,
      "delete_index_took_total": 313.93581539959996,
      "create_index_took_total": 207.38219340200885,
      "ingest_multi_field_took_total": 60584.180823600036,
      "refresh_index_store_kb_total": 2993302.6438476564,
      "refresh_index_took_total": 37546.422699699906,
      "force_merge_took_total": 243785.17976769945,
      "query_with_filter_took_total": 58514.0,
      "query_with_filter_took_p50": 5.0,
      "query_with_filter_took_p90": 6.0,
      "query_with_filter_took_p99": 6.7,
      "query_with_filter_took_p99.9": 9.4,
      "query_with_filter_took_p100": 1128.0,
      "query_with_filter_client_time_total": 94834.7,
      "query_with_filter_client_time_p50": 8.7,
      "query_with_filter_client_time_p90": 9.5,
      "query_with_filter_client_time_p99": 10.5,
      "query_with_filter_client_time_p99.9": 199.4,
      "query_with_filter_client_time_p100": 1133.0,
      "query_with_filter_memory_kb_total": 1297436.0,
      "query_with_filter_recall@K_total": 0.18980632427320202,
      "query_with_filter_recall@1_total": 0.4012699802934092
    }
  }

faiss-ivf

{
    "metadata": {
      "test_name": "index-workflow",
      "test_id": "index workflow",
      "date": "07/20/2023 08:38:18",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "914161664 (used) / 72095412224 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 100289.60986629999,
      "delete_model_took_total": 183.7637167000139,
      "delete_index_took_total": 707.2901364000245,
      "train_model_took_total": 1593.3622581998861,
      "create_index_took_total": 246.1298418998922,
      "ingest_took_total": 58624.54737009993,
      "refresh_index_store_kb_total": 2464553.5473632812,
      "refresh_index_took_total": 356.8165430002409,
      "query_took_total": 38577.7,
      "query_took_p50": 4.0,
      "query_took_p90": 4.0,
      "query_took_p99": 5.6,
      "query_took_p99.9": 17.2,
      "query_took_p100": 60.0,
      "query_client_time_total": 73840.5,
      "query_client_time_p50": 7.2,
      "query_client_time_p90": 8.0,
      "query_client_time_p99": 9.8,
      "query_client_time_p99.9": 55.1,
      "query_client_time_p100": 255.0,
      "query_memory_kb_total": 1075410.5,
      "query_recall@K_total": 0.9471388000000001,
      "query_recall@1_total": 1.0
    }
}

faiss-ivfpq

{
    "metadata": {
      "test_name": "index-workflow",
      "test_id": "index workflow",
      "date": "07/20/2023 16:58:36",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "828354560 (used) / 72094392320 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 263657.03240199864,
      "delete_model_took_total": 69.48158480081474,
      "delete_index_took_total": 664.398852699378,
      "train_model_took_total": 162104.747317599,
      "create_index_took_total": 190.47825660090894,
      "ingest_took_total": 58211.593049098155,
      "refresh_index_store_kb_total": 1903604.0291992188,
      "refresh_index_took_total": 2371.4333412004635,
      "query_took_total": 40044.9,
      "query_took_p50": 4.0,
      "query_took_p90": 4.3,
      "query_took_p99": 5.0,
      "query_took_p99.9": 22.8,
      "query_took_p100": 109.6,
      "query_client_time_total": 73778.9,
      "query_client_time_p50": 7.0,
      "query_client_time_p90": 8.0,
      "query_client_time_p99": 8.9,
      "query_client_time_p99.9": 73.7,
      "query_client_time_p100": 289.9,
      "query_memory_kb_total": 126500.8,
      "query_recall@K_total": 0.6135270000000002,
      "query_recall@1_total": 0.9940999999999998
    }
  }

lucene-hnsw

{
    "metadata": {
      "test_name": "index-workflow",
      "test_id": "Index workflow",
      "date": "07/20/2023 17:02:03",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "805933056 (used) / 72202358784 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 264153.68206990056,
      "delete_index_took_total": 585.3750425005273,
      "create_index_took_total": 328.73113790046773,
      "ingest_took_total": 173861.47249149944,
      "refresh_index_store_kb_total": 2237889.4716796875,
      "refresh_index_took_total": 410.2033980001579,
      "query_took_total": 88967.9,
      "query_took_p50": 8.9,
      "query_took_p90": 10.1,
      "query_took_p99": 11.1,
      "query_took_p99.9": 18.5,
      "query_took_p100": 51.5,
      "query_client_time_total": 124045.8,
      "query_client_time_p50": 12.2,
      "query_client_time_p90": 13.4,
      "query_client_time_p99": 14.7,
      "query_client_time_p99.9": 51.7,
      "query_client_time_p100": 267.3,
      "query_memory_kb_total": 0.0,
      "query_recall@K_total": 0.9995793999999998,
      "query_recall@1_total": 1.0
    }
}

lucene-hnsw restrictive-filter

{
    "metadata": {
      "test_name": "Lucene HNSW Restrictive Filter Test",
      "test_id": "Lucene HNSW Restrictive Filter Test",
      "date": "07/20/2023 22:41:50",
      "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
      "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
      "processor": "x86_64, 36 cores",
      "memory": "1006227456 (used) / 71916462080 (available) / 73716973568 (total)"
    },
    "results": {
      "test_took": 416076.4546975993,
      "delete_index_took_total": 265.0550271006068,
      "create_index_took_total": 292.8249509015586,
      "ingest_multi_field_took_total": 170225.01544579866,
      "refresh_index_store_kb_total": 2238015.027246094,
      "refresh_index_took_total": 427.53609320061514,
      "force_merge_took_total": 187610.8231805978,
      "query_with_filter_took_total": 57255.2,
      "query_with_filter_took_p50": 5.0,
      "query_with_filter_took_p90": 6.0,
      "query_with_filter_took_p99": 6.0,
      "query_with_filter_took_p99.9": 11.5,
      "query_with_filter_took_p100": 1032.2,
      "query_with_filter_client_time_total": 93825.7,
      "query_with_filter_client_time_p50": 8.6,
      "query_with_filter_client_time_p90": 9.4,
      "query_with_filter_client_time_p99": 10.0,
      "query_with_filter_client_time_p99.9": 241.6,
      "query_with_filter_client_time_p100": 1036.3,
      "query_with_filter_memory_kb_total": 0.0,
      "query_with_filter_recall@K_total": 1.0,
      "query_with_filter_recall@1_total": 1.0
    }
  }

lucene-hnsw relaxed-filter

{
  "metadata": {
    "test_name": "Lucene HNSW Relaxed Filter Test",
    "test_id": "Lucene HNSW Relaxed Filter Test",
    "date": "07/20/2023 20:39:31",
    "python_version": "3.8.17 (default, Jul  5 2023, 21:04:15) \n[GCC 11.2.0]",
    "os_version": "Linux-6.1.34-59.116.amzn2023.x86_64-x86_64-with-glibc2.17",
    "processor": "x86_64, 36 cores",
    "memory": "1051287552 (used) / 71956045824 (available) / 73716973568 (total)"
  },
  "results": {
    "test_took": 303027.87016130274,
    "delete_index_took_total": 412.14946289983345,
    "create_index_took_total": 220.56818470155122,
    "ingest_multi_field_took_total": 171905.49855670033,
    "refresh_index_store_kb_total": 2225311.6084960937,
    "refresh_index_took_total": 430.2539570009685,
    "query_with_filter_took_total": 130059.4,
    "query_with_filter_took_p50": 13.1,
    "query_with_filter_took_p90": 14.3,
    "query_with_filter_took_p99": 15.6,
    "query_with_filter_took_p99.9": 28.2,
    "query_with_filter_took_p100": 67.1,
    "query_with_filter_client_time_total": 166145.9,
    "query_with_filter_client_time_p50": 16.4,
    "query_with_filter_client_time_p90": 18.1,
    "query_with_filter_client_time_p99": 19.5,
    "query_with_filter_client_time_p99.9": 53.1,
    "query_with_filter_client_time_p100": 357.3,
    "query_with_filter_memory_kb_total": 0.0,
    "query_with_filter_recall@K_total": 0.999678,
    "query_with_filter_recall@1_total": 1.0
  }
}

navneet1v · 2023-08-11T18:53:37Z

K = 100 in all the above experiments.

prudhvigodithi added the v2.9.0 label Jun 27, 2023

github-actions bot added the untriaged label Jun 27, 2023

prudhvigodithi mentioned this issue Jul 4, 2023

[RELEASE] Release version 2.9.0 opensearch-project/opensearch-build#3616

Closed

42 tasks

junqiu-lei self-assigned this Jul 4, 2023

junqiu-lei removed the untriaged label Jul 4, 2023

junqiu-lei mentioned this issue Jul 10, 2023

Add 2.9.0 release notes #961

Merged

prudhvigodithi changed the title ~~Release version 2.9.0~~ [RELEASE] Release version 2.9.0 Jul 18, 2023

junqiu-lei closed this as completed Aug 11, 2023

navneet1v mentioned this issue Aug 16, 2023

[Feature] [META] Add Efficient filtering support for Faiss Engine #903

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RELEASE] Release version 2.9.0 #944

[RELEASE] Release version 2.9.0 #944

prudhvigodithi commented Jun 27, 2023 •

edited by junqiu-lei

Loading

This Component Release Issue

Release Steps

The Overall Release Issue

What should I do if my plugin isn't making any changes?

junqiu-lei commented Jul 24, 2023 •

edited by navneet1v

Loading

navneet1v commented Aug 11, 2023

[RELEASE] Release version 2.9.0 #944

[RELEASE] Release version 2.9.0 #944

Comments

prudhvigodithi commented Jun 27, 2023 • edited by junqiu-lei Loading

This Component Release Issue

Release Steps

The Overall Release Issue

What should I do if my plugin isn't making any changes?

Preparation

CI/CD

Pre-Release

Release Testing

Release

Post Release

junqiu-lei commented Jul 24, 2023 • edited by navneet1v Loading

Performance Test Results 2.9

Configurations

Data Sets

Cluster Configuration

Client Configuration

Visualizations

Perf Results Comparison Between 2.8.0 and 2.9.0

Ingest took total

Query took p50

Query took p99

Perf Results Comparison in 2.9.0

Query took total

Query took p50

Query took p99

Faiss filters

Lucene filters

Results

nmslib-hnsw

faiss-hnsw

faiss-hnsw relaxed-filter

faiss-hnsw restrictive-filter

faiss-ivf

faiss-ivfpq

lucene-hnsw

lucene-hnsw restrictive-filter

lucene-hnsw relaxed-filter

navneet1v commented Aug 11, 2023

prudhvigodithi commented Jun 27, 2023 •

edited by junqiu-lei

Loading

junqiu-lei commented Jul 24, 2023 •

edited by navneet1v

Loading