Add schema for embed API requests

[bjester]

Summary

• Defines a JSON schema for the structure of recommendations API requests that will be received by TorchServe
• Fields level, parent, and has_content will be generated in preprocessing in the TorchServe handler
• TODO: unit tests to ensure schema does what we expect

Reference

Closes learningequality/studio#4392

[bjester]

@jamalex :: @akolson and I worked on a JSON schema to nail down the format of the data sent to TorchServe for topic recommendations. We followed your example in the example pipeline code. Does it look sufficient?

Other questions:

• Do we need has_content to represent whether a topic has non-topic resources? Or is that only representative of whether it has subtopics?
• What is category and do we need it?

[jamalex]

I added some clarifying comments in here -- the main thing is that the current schema seems designed to send the entire tree at once, whereas we only need the target topic(s) and their ancestors.

Fields level, parent, and has_content will be generated in preprocessing in the TorchServe handler

Level and has_content aren't important for embedding. The parent/ancestors are needed, though.

[jamalex]

Thanks, great changes -- getting closer! I had a few high level points/questions (such as this being the schema for calling an embedding vs recommender endpoint, and needing to support embedding of content resources in addition to topics). And then some smaller notes. Happy to hop on a call to chat through any of my notes that may be confusing! Thanks.

[jamalex]

Just wondering about naming: is this endpoint for recommendations, or embeddings? (I know there was discussion of having endpoints for making actual recommendations, as well -- but then we'd probably want to add a few additional parameters in addition to what's here, e.g. around desired number of recommended items to return, etc. So it may be best to have this named as an embedding endpoint schema?)

[akolson]

If I understand you correctly, I think "$id": "/schemas/recommendations_request", here is a URI for the schema to refer to elements of the schema from inside the same document or from external JSON documents. However, your comment on endpoint naming still stands and will be put into consideration for the endpoints we intend to implement.

[akolson]

Thus said, this schema is for embedding purposes only. Serve responds with embeddings that we hope to store and make comparisons against later in studio. I will change the URI to embed_request for better clarity.

[akolson]

Hi @jamalex I was able to incorporate the feedback. It was pretty clear so need for the call. We should now be more closer to the final request body 🤞. Please let me know incase there is anything that may require an update. @bjester, a few files have been renamed as noted by Jamie in his comment. Also, any feedback is welcome. Thanks

Resolved

[jamalex]

Looks good to me -- thanks! I left one small comment that may be good to address if you have a chance, but not a blocker.