Support a way to do technical maintenance of URLs within the dialog aggregate

[elsand]

Introduction

The dialogs contain several URLs, some of which are immutable (transmissions). Over time, there might arise a need to update the URLs due to system changes/updates which causes the original URLs to become invalid.

Transmissions are immutable and cannot be changed at all. Actions, content references and dialoglevel attachments are immutable, and may have the URLs changed, but it is not desirable to invoke the full range change handling mechanisms (bump updatedAt, produce events, reset system labels etc) for a change that is only done due to technical maintenance related need.

Description

Endpoints should be established that allow for only URL changing, that also bypass the normal change pipeline.

There are some options:

1. Expose per actions/attachment-url/content reference endpoints, eg PUT actions on

/api/v1/serviceowner/dialogs/{dialogId}/mainContentReference/url
/api/v1/serviceowner/dialogs/{dialogId}/attachments/{attachmentId}/urls/{urlId}/url
/api/v1/serviceowner/dialogs/{dialogId}/guiActions/{guiActionId}/url
/api/v1/serviceowner/dialogs/{dialogId}/apiActions/{apiActionId}/endpoints/{endpointId}/url
/api/v1/serviceowner/dialogs/{dialogId}/transmissions/{transmissionId}/attachments/{attachmentId}/urls/{urlId}/url
/api/v1/serviceowner/dialogs/{dialogId}/transmissions/{transmissionId}/{contentReference}/url

2. Expose a common "URL maintenance" endpoint, that looks something like:

/api/v1/serviceowner/dialogs/{dialogId}/actions/url-maintentance

Which accepts a body like:

{
   "updatedUrls": [ 
      { "id": "{urlId}", "url": "https://newurl ..." }
   ]
}

Dialogporten will then match the {urlId} to the corresponding URL within any part of the aggreate

In both cases, IDs will have to be exposed on all URLs (including values used for content references).

Implementation

TBD

Tasks

Preview Give feedback

1. Implementation tasks are added here
2. Prepare documentation (if relevant - either update working document, or add a new file in docs)
3. Add e2e-test (if relevant)

Threat modelling

Preview Give feedback

1. Does this change introduce any potential security issues?

Acceptance criteria

GIVEN ...
WHEN ....
THEN ...

GIVEN ...
WHEN ....
THEN ...

[MagnusSandgren]

Some suggestions arose during refinement:

1. Create a new url template aggregate which other parts of the system may reference
2. Split the above suggestion into Host and PathTemplate entities

GuiAction uri reference is used as an example in the image below, however this could be any part of the system which reference an external resource.

Templates makes it easy for a service owner to update url references without having to deep dive into every dialog they've ever created to do so. Suggestion 2 is better for updating the host. However, looping over all UrlTemplates in suggestion 1 and changing host that way is a viable alternative while keeping the structure simple. The assumption is that the UrlTemplates set for a service owner is small.

Changing SO api target version would also be easy through templates.

The only challenge would be altering the template placeholders on existing templates.

• Add: Say a SO wants to add a query parameter template placeholder to /api/v1/entity/{entityId} so that it becomes /api/v1/entity/{entityId}?deleted={deleted}. Now we have two template placeholders, however existing entities (say GuiAction) would only have the value for {entityId}, not {deleted}. If we connect placeholder-value by name we can offer the possibility to define default values, thereby not breaking existing references.
• Rename: Given a template placeholder is referenced by its name, a rename would break the placeholder-value reference. However, if the template placeholder is referenced by order (string.Format("{0} {1}", "Hello", "World")) a rename would not be an issue.
• Remove: Remove would only be an issue if the template placeholder is referenced by order. The placeholder value would just be ignored if it's referenced by name.

How we connect the template placeholder with the actual value affects migration strategy when altering the template.

Needs more consideration...

[elsand]

Refinement:

We want to investigate whether or not we can have a more generic templating system to be used across all properties that use more or less free-form text (content and URLs). Something akin to:

"value": "sdf sdlfk jsdlf kjsdf {supersimpleservice.summary.nb(navn='Bjørn Langfors', age=12)} sdfs"
// "supersimpleservice.summary.nb": "Hei #navn#! på #age# år! Her er greiene dine. {boilerplate}"
// "boilerplate": "Kontaktinfo: {tlf}"
// "tlf": "212345678" 
// "---> rendered": "sdf sdlfk jsdlf kjsdf Hei Bjørn Langfors på 12 år! Her er greiene dine! Kontaktinfo: 212345678 sdfs"

This can be used in a backwards-compatible way (ie. no need to change the API contract), as it introduces a separate root for these placeholders, that is defined on a per-serviceowner basis.

Ie. support arbitrary parameterized placeholders, with recursion support. This will solve both this issue, and potentially some of #1650 , where we can drastically reduce disk space usage caused by content/URLs by eliminating much of the redundancy.

This will however cause search to become more complex (see #1651), and will probably require that we maintain a separate index in ElasticSearch or similar, which is populated with the resolved text-data whenever a given dialog is created/updated. Whenever a placeholder is changed, this will have to cause all dialogs that (indirectly) refer that placeholder to be reindexed. But this will also could greatly improve search performance and functionality for end users.

Disaster recovery will have to be considered, as we need to be have ES and PG in sync. During recovery/resync, we might have to be able to fall back to more simple PG-based search.

Some initial considerations:

• We would probably want the initial version of this without ES
• Search capability will either require expensive and complex run-time resolving of placeholders, or some sort of separate ES index.
• URLs are not searchable, nor do they have the same need for recursion or parameterization, so this can be more easily implemented, whilst allowing for future expansion into a more generic content parameterization mechanism.