phase 2: creative-mode and aux-graph/result-analyses refactor

[colleenXu]

Background

• info on TRAPI 1.4 KP aux-graph / result.analyses expectations in phase 1: aux-graph/result.analyses refactor for basic querying #603. That issue includes the provenance that should be added to "new" edges (with the new format phase 1: provenance refactor for x-bte operations  #604 )

Overview

Slides: https://docs.google.com/presentation/d/1aHm1Ndk1hUX9AU7SquhtjLvN4tlQdL4HnocH05gu-PE/edit?usp=sharing

Notes:

• remember that in creative-mode, we get a QGraph with 2 QNodes, 1 of which has 1 ID on it
• we still want one result per unique "answer" node (this is partially defined by the templates + which QNode has an ID on it)
• we still want to deal with "final" score / "individual" scores from templates the same way. So we only report the "final" score...
• in a result, each edge represents the data from 1 template-run (aka "collapses" the result sub-graph from that template-run). IF that template was a multi-hop or that result involved ID-expansion, the edge will have an auxiliary-graph reference in its attributes.
  • we'll want to construct and keep those auxiliary-graphs in the auxiliary_graphs section of the TRAPI response
  • we'll want to keep the edges listed in the auxiliary-graphs in the knowledge_graph.edges section of the TRAPI response
  • we'll maybe do nesting of auxiliary-graphs/edges when a result from a template-run is BOTH a multi-hop and involves ID-expansion? My slides give an example with nesting and w/o...

[tokebe]

@colleenXu Just wanted to double-check on this: Obviously, we need to create "creative edges" that represent a given template's answer. I've forgotten why we would want each of these to be within the same analysis? Because we have individual scores per-template, we could conceivably represent them each as a separate analysis, which at least feels like it makes sense semantically (calling each "way we found this answer" a single "analysis"). We could theoretically order our results by the sum of their analyses scores?

I'm pretty sure we had solid reasoning for using just one analysis, just wanted to have it written down somewhere for my forgetfulness.

[colleenXu]

@andrewsu

@tokebe and I are stuck on a design decision and would like your input. We reference this recently-edited slide deck

When a BTE creative-mode result is supported by multiple templates, do we want to represent it as:

• Slide 2: 1 Analysis Object, so there's 1 score that encompasses info from all templates. I find this easier to "find all info from BTE in one place" and less verbose; or...
• Slide 3: multiple Analysis Objects, 1 per template, so there's 1 score per template. Jackson says this seems to match the meaning of an Analysis better (see his comment above), and it may better match what the ARS / UI team are going to do with the Analysis Objects. I countered that it's not clear what the ARS / UI team are going to do...

Jackson says that both options are about equal in development time/effort.

notes: some background we agree on

• when a result is supported by a multi-hop template, 1 creative-edge will be made that references 1 auxiliary graph. That auxiliary graph stores all the edges from that template's result (e_creative_orange in the slides)
• If a result is supported by MULTIPLE such templates, then 1 creative-edge (w/ 1 auxiliary graph) will be made for each template (e_creative_orange vs e_creative_purple in the slides)
  • But the TRAPI spec has a LOT of ambiguity here. It may also be okay to have 1 creative-edge total, then have it reference multiple auxiliary-graphs (1 per template). But Jackson and I weren't really in favor of doing this...it shoves the "multiple-template info" deeper into the data structures...

[andrewsu]

As I understand it, either slide 2 or slide 3 are TRAPI-compliant. I wish TRAPI was more opinionated / prescriptive about the right solution. But absent that, I favor the solution in slide 2 (mostly because I don't like the idea of multiple scores for the same result).

I'm not entrenched in this position, so I'm open to being convinced of another option.

[colleenXu]

Note: the last two slides of the slide deck describe two possible ways we could handle aux-graphs when we have BOTH subclassing + a multi-hop creative template.

Jackson and I agree that the "nested" option is easier to implement but more verbose (edge->aux-graph->edge->aux-graph) .So we'll probably do that...

• musing: could first handle subclassing, then handle multi-hop template

[colleenXu]

From 2023-05-10 meeting:

• Noted feedback: adjusting how creative-mode is handled
  • direct edges in edge bindings, not in support graphs
  • For multi-hop templates (aka real “inferred” stuff), result.analyses.edge_bindings can have 1 single “creative” edge. Then it’ll reference multiple supporting graphs (1 per template) <- decision. Other option was edge_bindings having multiple “creative” edges, each referencing single supporting graph (aka 1 of both per template)

[colleenXu]

@tokebe oops one format issue:

In the auxiliary graph section of the TRAPI response, each auxiliary graph is an object; their property "edges" is the list of Edges. See TRAPI spec here. I've also corrected the slides linked in this issue.

(brought up by Guthrie Price of UI team)

[tokebe]

I'll have to fix this briefly before returning to subclassing so I can have it on dev/ready for CI.

[tokebe]

Marking as completed.

Minor improvements to aux graphs are possible, that topic is now tracked in #682. Any undiscovered bugs may be tracked in future issues.