Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adding a test for metadata filtering.
WIP failing test:
So we have metadata from vespa but then we're also querying the db for metadata. In these tests we are getting a disparity between the metadata in vespa and that in the db. Thus, the search results when using a metadata filter aren't correct. I.e. filtering for
sector: Price
returns results without that metadata being present._populate_db_families
is non-deterministic as well.vespa_families[0].hits[0].metadata -> [{'name': 'sector', 'value': 'Price'}...
>> family_and_family_metadata[0][1].value['sector'] ['Agriculture', 'Urban', 'LULUCF', 'Cross Cutting Area', 'Finance', 'Social development'...]
>>> db_family_tuple[1].value['sector'] ['Transport', 'Economy-wide', 'Disaster Risk Management (Drm)'.... Not sector: ['Price']
In theory this shouldn't happen in practice due to the db_state being from the db but is there a possibility we update metadata and before vespa is updated start getting some odd search results?
Validating this theory that there's a disparity.
Ran the following yql and proved that the vespa search results are correct against the db.
vespa query 'select * from family_document where family_import_id contains "CCLW.family.8633.0"
Validated postgres by adding a breakpoint and adding running the sql alchemy query.
Proposed version
Please select the option below that is most relevant from the list below. This
will be used to generate the next tag version name during auto-tagging.
Visit the Semver website to understand the
difference between
MAJOR
,MINOR
, andPATCH
versions.Notes:
used -- e.g. Major > Minor > Patch
sure your selected option is marked
[x]
with no spaces in between thebrackets and the
x
Type of change
Please select the option(s) below that are most relevant:
How Has This Been Tested?
Please describe the tests that you added to verify your changes.
Reviewer Checklist