Add callback url to exchanges?

[brianorwhatever]

To avoid polling architecture should we think about a way to include a callbackUrl for the workflow to hit when an exchange has been complete?

[TallTed]

I suggest adding a question mark to the end of this issue's title, and closing #346 as a dupe of this issue, as this issue is more completely described.

[brianorwhatever]

Oops definitely accidentally duped this. Thanks for the catch!

[dlongley]

One reason to add a callback URL might be to try and eliminate a need to poll an exchange for updates to its state. I imagine that's the main draw.

However, even if we had a callback URL, it seems to me that using it could fail for any variety of reasons, leading to a need to poll an exchange anyway. So if we were thinking that having a callback URL could allow for a push-only notification experience, that might be not be correct. It may instead just reduce the amount of polling necessary in the average case.

If this reasoning is correct, then we should consider the added complexity of having this feature according to this less appealing "average case" trade off instead of considering that it would eliminate polling entirely.

Other considerations around "callback URLs" involve what kind of authz would be required (could we keep it simple with capability URLs?), how many times can they be called, with what request data and what response data, etc.

[jrhender]

I think this issue might be a duplicate (it might not though) of #295 . I'd say it certainly relates to it.

[msporny]

Yes, this issue is a duplicate of #295.

[msporny]

The group discussed this on 2024-05-28 call.

@PatStLouis noted that this is a more specific case of issue #295 and perhaps we could discuss in #295. Cover callback URLs in issue #295. @dlongley recommended closing #295 and keeping this one since it's more specific. The most useful place to put callbacks right now is in exchanges. @jandrieu +1 to solve the more specific use case (this issue). @PatStLouis reading #295, use case identified, report to external system on progress in an exchange, somewhat of a different use case. Make sure threads in other issue are captured here. @dlongley didn't think that there is a use case to report back on progress of issuance (atomic operation of issuance API). Use cases are around exchanges, other issue has been opened for a long time, it's still open/linked if it's not exchange related, let's close to make progress. @jandrieu noted that point of exchanges is to be abstraction, right layer to handle callbacks in general, this isn't just simpler use case, among APIs that are synchronous, we can move async stuff to exchanges (where all callback stuff moves).

The group decided to focus the callback URL discussion wrt. exchanges and close issue #295 because it was believed that by addressing callbacks in exchanges that it would address most, if not all, callback use cases.

[msporny]

There are additional challenges to create a PR for this issue:

• authorization challenges
• ephemeral exchanges needing oauth2 credentials
• using capability URLs
• error handling (should exchange be halted on bad callback, or option to configure, what is UX like if callback fires and nothing happens)

The group isn't sure if its ready for a PR on this just yet before becoming more certain on the challenges above. The group is going to wait until we have PR #382 merged and then the group will see where it can go from there.

[msporny]

The group discussed this briefly on the 2024-06-18 call, but decided to wait for more people to be present to process it further.

[jrhender]

A point I would like to add from my recollection of the 2024-06-18 call is that @msporny noted that the callback payload may consist of the submitted presentation and other metadata about the presentation/workflow.

In my opinion, this description of the payload is similar to the payload the OpenWalletFoundation implementation sends in its callbacks. An example of this callback payload can be seen here.

[dlongley]

Note that including information such as the payload in the callback can create problems for consumers that have strict limits on incoming payload size. These same limits are usually more relaxed for outgoing calls, suggesting that a more acceptable pattern would involve having the workflow service call a callback with a minimal payload of metadata ... followed by the callback system optionally making a call to ask for the full exchange state (presuming it is authorized to do so).

We could also support both approaches if we want the additional complexity, but it might be worth only spec'ing the "minimal payload" version.

[dlongley]

I should also note that limits on incoming payload size could be lifted -- but this is usually only done based on authz credentials, which only further adds complexity. Ideally, an approach here could be designed that doesn't require additional authz.

Additional authz comes with many headaches, e.g., ensuring authz-mechanism compatibility between workflow services and callback systems (where the client/server roles are reversed), keeping access tokens for a workflow instance up-to-date, proper secret-management of per-workflow-instance authz credentials, ensuring timely responses when access tokens need to be refreshed, and so on.

I think it would be best to have a design that uses capability URLs for callback URLs (similar to how exchange URLs can be capability URLs), where the security requirements can be kept low by reducing the information transmitted (both for size and content) to the callback system. Then the callback system can use its (almost-always) pre-existing authz-credentials to query the workflow service for exchange state as needed.