keyword search can not be combined with other search parameters

[msbentley]

🐛 Describe the bug

I'm not clear if this is by design, but if I run a keyword search, e.g.

https://pds.nasa.gov/api/search/1.0/products?keyword=magnetic

I get a number of hits of different product types. If I want to repeat this search but get only observational products with this keyword, I assumed I could do:

https://pds.nasa.gov/api/search/1.0/products?keyword=magnetic&q=pds:Identification_Area.pds:product_class eq “Product_Observational”

but this returns the same number of results, implying that when keyword is used, other search parameters are ignored? It would be great to clarify (and document) if this is expected behaviour, or else allow combinations of search parameters.

📜 To Reproduce

Steps to reproduce the behavior:

• Run API searches as above

🕵️ Expected behavior

Unless told otherwise, I would assume that I could constrain a keyword search with other search terms (e.g. q),

📚 Version of Software Used

PDS API V1, tested on 22/11/2022

---

🦄 Related requirements

⚙️ Engineering Details

[jordanpadams]

@tloubrieu-jpl we should maybe discuss this at the API WG whether or not this is something that needs to be supported by the API. i definitely see where this could be helpful for narrowing responses, but maybe we use faceting to do this instead?

[tloubrieu-jpl]

Thanks @jordanpadams I'll keep that as a discussion topic for tomorrow.

[alexdunnjpl]

@jordanpadams am I remembering correctly from yesterday that the conclusion was that this functionality will not be implemented?

[jordanpadams]

@alexdunnjpl correct. we are going to table this implementation for now.

@msbentley for now, I think the workaround would be use to use the new API endpoints (should be deployed soon), which I believe will be:

https://pds.nasa.gov/api/search/1.1/classes/observationals?keyword=magnetic

@tloubrieu-jpl can correct me if I am wrong.

[tloubrieu-jpl]

The work around will be to integrate the keyword search into the q parameter value. We still need to document what the workaround will be, @msbentley might be interested.

[msbentley]

Fine with me - either search syntax updates or document updates would be great, I was just not sure what to expect.

[jordanpadams]

Moved from pds-api repo

[alexdunnjpl]

@jordanpadams @tloubrieu-jpl could one of y'all please give me a rundown on what PDS keywords are and how they're supposed to work in search? Based on the swaggerdocs description of "free text search of title/description" I'd guess relevance-based ordering of results?

Are we sure this requirement is actually possible to fulfill? As far as I understand it, introducing a non-deterministic value (relevance) as a sort field means that reliable search-after pagination is not possible when keyword qparam is used.

[alexdunnjpl]

Related tickets:

#460
#17
NASA-PDS/registry-api-service#60

[jordanpadams]

@alexdunnjpl the original developer of Harvest created some combined field for the keyword field or search that allows search of a couple specific fields. We have not done any kind of detailed analysis as to the usefulness of this field or if other information should be included there (target names, instrument names, primary result summary information, keyword attributes, etc.). the ability to perform some sort of keyword-like search seems very doable from OpenSearch since it is really more suited for that than it is the form-based search we are trying to enable with the rest of the API. Long story short, getting it to work at all is a start, and then we can use cases like above to help hone the search capability.

introducing a non-deterministic value (relevance) as a sort field means that reliable search-after pagination is not possible when keyword qparam is used.

hmmm, yeah this search after fails here. We will need to figure out a workaround for search support here. Or maybe we just say for keyword search we only support X number of results being returned.

[alexdunnjpl]

@jordanpadams in light of what you've just said, is "working at all" still part of the MVP for #424? Seems like maybe it shouldn't be blocking, though I guess at-least- @msbentley is using the feature.

[jordanpadams]

@alexdunnjpl we definitely need this to work eventually, but not highest priority. In the MVP, this should probably be replaced with like query support.