Next steps End-to-end: Implementation Proposal + Implementation

[wochinge]

Short scope document

Policies have to be able to return events to store information about the used featurization in the tracker.

[wochinge]

I think it would be cool to use a context manager to save the tracker. Currently we save the tracker quite often in between which is 1) Complex from a code-perspective and 2) Slow.

I'd imagine something like

with tracker_store.get_tracker(sender_id) as tracker:
   # ... do things

This would mean that a tracker would only be saved after at the end of the interaction (aka we're waiting for the next user message).

[wochinge]

@Ghostvv I thought about it some more:

• I'd implement "policies returning events" against master. This keeps the diff for end-to-end smaller and you can just use it in your e2e implementation then
• We need a deprecation warning for policies not returning the new items
• I think DefinePrevUserUtteredFeaturization could use apply_to to update the UserUttered event instead of applied_events. This way the whole logic would be nicely encapsulated within the event and not spread out too much.
• I think a separate event is indeed better than manipulating the UserUttered event directly. This has two advantages for me
  • more in line with our philosophy of unmodifiable events and that the tracker events give a precise history of what happened
  • allows the policies to manipulate the tracker by only using events. Directly modifying the event would require some less abstracted operations which are less generalizable / reusable.

My idea from the comment above would be a nice refactoring, but it's not required for these changes

[Ghostvv]

I'd implement "policies returning events" against master.

ok, the only problem, that you couldn't properly test it

I think DefinePrevUserUtteredFeaturization could use apply_to to update the UserUttered event instead of applied_events.

as we discussed I prefer this one as well, but it'll work only if trackers are stored at the right moment. I don't think it'll work reliably without your idea above

more in line with our philosophy of unmodifiable events and that the tracker events give a precise history of what happened

but if you do it with DefinePrevUserUtteredFeaturization.apply_to you will modify UserUttered event

[wochinge]

ok, the only problem, that you couldn't properly test it

I mean your approach already works, right? That would simply be the same changes just a bit more polished. And we can write tests for it and so on. Worst case is that you can cherry pick the changes and we quickly try it out in e2e?

as we discussed I prefer this one as well, but it'll work only if trackers are stored at the right moment. I don't think it'll work reliably without your idea above

When we do DialogueStateTracker.from_events we call tracker.update for every event which in turn calls apply_to for every event (same applies to TrackerWithCachedStates). So that would be applied independent of when events are persisted.

but if you do it with DefinePrevUserUtteredFeaturization.apply_to you will modify UserUttered event

But not in the database. applied_events in your branch is currently doing the same modification.