Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] LangPainlessClientYamlTestSuiteIT test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} failing #110290

Closed
elasticsearchmachine opened this issue Jun 28, 2024 · 5 comments · Fixed by #110521
Closed

[CI] LangPainlessClientYamlTestSuiteIT test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} failing #110290

elasticsearchmachine opened this issue Jun 28, 2024 · 5 comments · Fixed by #110521
Assignees
@mayya-sharipova
Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache medium-risk An open issue or test failure that is a medium risk to future releases :Search Relevance/Vectors Vector search Team:Core/Infra Meta label for core/infra team Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

Build Scans:

Reproduction Line:

./gradlew ':modules:lang-painless:yamlRestTest' --tests "org.elasticsearch.painless.LangPainlessClientYamlTestSuiteIT.test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported}" -Dtests.seed=ED89F8CD62B783B9 -Dtests.locale=es-ES -Dtests.timezone=America/Lower_Princes -Druntime.java=22

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Failure at [painless/146_dense_vector_bit_basic:136]: expected [400] status code but api [search] returned [200 OK] [{"took":2,"timed_out":false,"_shards":{"total":2,"successful":1,"skipped":0,"failed":1,"failures":[{"shard":0,"index":"test-index","node":"3RCBJqISShqsIk6rXEn1DA","reason":{"type":"script_exception","reason":"runtime error","script_stack":["[email protected]/org.elasticsearch.script.field.vectors.BitBinaryDenseVector.cosineSimilarity(BitBinaryDenseVector.java:71)","[email protected]/org.elasticsearch.script.VectorScoreScriptUtils$ByteCosineSimilarity.cosineSimilarity(VectorScoreScriptUtils.java:385)","[email protected]/org.elasticsearch.script.VectorScoreScriptUtils$CosineSimilarity.cosineSimilarity(VectorScoreScriptUtils.java:428)","cosineSimilarity(params.query_vector, 'vector')","                       ^---- HERE"],"script":"cosineSimilarity(params.query_vector, 'vector')","lang":"painless","position":{"offset":23,"start":0,"end":47},"caused_by":{"type":"unsupported_operation_exception","reason":"cosineSimilarity is not supported for bit vectors."}}}]},"hits":{"total":0,"max_score":null,"hits":[]}}]

Issue Reasons:

  • [main] 3 failures in test test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} (1.0% fail rate)
  • [main] 1.0% failure rate in test test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} (1.0% fail rate)

Note:
This issue was created using new, in-development muting automation. Please mind any issues.
@elasticsearchmachine elasticsearchmachine added :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >test-failure Triaged test failures from CI labels Jun 28, 2024
elasticsearchmachine added a commit that referenced this issue Jun 28, 2024
@elasticsearchmachine
Mute org.elasticsearch.painless.LangPainlessClientYamlTestSuiteIT tes…
ac740e4 
…t {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} #110290
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 3 failures in test test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} (1.0% fail rate)
  • [main] 1.0% failure rate in test test {yaml=painless/146_dense_vector_bit_basic/Cosine Similarity is not supported} (1.0% fail rate)

Build Scans:

@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Jun 28, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-core-infra (Team:Core/Infra)

@ldematte
Copy link
Contributor

ldematte commented Jul 1, 2024

@benwtrent could this be related to #110059?

@thecoop thecoop added medium-risk An open issue or test failure that is a medium risk to future releases :Search Relevance/Vectors Vector search and removed needs:risk Requires assignment of a risk label (low, medium, blocker) labels Jul 5, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Jul 5, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search (Team:Search)

@mayya-sharipova mayya-sharipova self-assigned this Jul 5, 2024
@mayya-sharipova
Copy link
Contributor

mayya-sharipova commented Jul 5, 2024 •
edited
Loading

This happens in cases where an index has more than 1 shards, even if we get expected failures on 1 shard, no failures on other empty shards make the whole request return 200.

I will fix the test to ensure that index has always 1 shard.

mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Jul 5, 2024
@mayya-sharipova
Fix bit vector tests
9310a35 
Bit vector tests were failing in cases where an index has more than 1 shards.
For error cases when we expected a failure of the whole request, shards
with empty documents returned success and the whoel request unexpectedly returned 200.

Ensuring that index contains only 1 shard fixes these failures.

Closes elastic#110290, elastic#110291
@mayya-sharipova mayya-sharipova mentioned this issue Jul 5, 2024
Merged
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Jul 5, 2024
@mayya-sharipova
Fix bit vector tests (elastic#110521)
c2ee49c 
Bit vector tests were failing in cases where an index has more than 1 shards.
For error cases when we expected a failure of the whole request, shards
with empty documents returned success and the whoel request unexpectedly returned 200.

Ensuring that index contains only 1 shard fixes these failures.

Closes elastic#110290, elastic#110291
mayya-sharipova added a commit that referenced this issue Jul 5, 2024
@mayya-sharipova
Fix bit vector tests (#110521) (#110525)
3f2c81e 
Bit vector tests were failing in cases where an index has more than 1 shards.
For error cases when we expected a failure of the whole request, shards
with empty documents returned success and the whoel request unexpectedly returned 200.

Ensuring that index contains only 1 shard fixes these failures.

Closes #110290, #110291
@javanna javanna added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search Meta label for search team labels Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mayya-sharipova mayya-sharipova

Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache medium-risk An open issue or test failure that is a medium risk to future releases :Search Relevance/Vectors Vector search Team:Core/Infra Meta label for core/infra team Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix bit vector tests mayya-sharipova/elasticsearch
5 participants
@javanna @ldematte @thecoop @mayya-sharipova @elasticsearchmachine