[pickup 3.0] internal error 500 on delivery request

[antonbaliasnikov]

Issue overview

After recipient successfully sets up a mediation, when a sender sends a message to recipient, recipient is unable to pick up the message, because blocktrust mediator responds with error 500 on a delivery-request.

Here is the scenario:

Given Recipient sends a mediate request message to the mediator
And Mediator responds to Recipient with mediate grant message
Given Sender sent a forward message to Recipient
When Recipient sends a delivery-request message
Then Mediator delivers message of Sender to Recipient

How to reproduce

Mediation request

Mediation is successful, request:

{
    "id": "4064f2a6-bc3b-4e0b-a328-4d559ae433d9",
    "typ": "application\/didcomm-plain+json",
    "type": "https:\/\/didcomm.org\/coordinate-mediation\/2.0\/mediate-request",
    "from": "did:peer:2.Ez6LSkXGpxBFGdGkj5xZGrkiRLSJJXfNvze7d5hhQ8aT6kHXp.Vz6MkwKjkLPuesP9s3WPnArbyerkYM2aYy8egSqsbBS7oRcGr.SeyJ0IjoiZG0iLCJzIjoiaHR0cDovL2hvc3QuZG9ja2VyLmludGVybmFsOjk5OTkvIiwiciI6W10sImEiOlsiZGlkY29tbS92MiJdfQ",
    "to": [
        "did:peer:2.Ez6LSeUYyDHMTbWoMGCKyqntPR95TB3N6ic2A27YLmwZHchxY.Vz6MkgRyq89zDCmXEcg8LmdqKjoaanxK4MUVbbtembDa4fLpK.SeyJpZCI6Im5ldy1pZCIsInQiOiJkbSIsInMiOiJodHRwczovL21lZGlhdG9yLmJsb2NrdHJ1c3QuZGV2LyIsInIiOltdLCJhIjpbImRpZGNvbW0vdjIiXX0"
    ],
    "body": {},
    "return_route": "all"
}

Response:

{
    "id": "acfe9df7-45b5-4eb5-ab52-1b37ba15107f",
    "typ": "application\/didcomm-plain+json",
    "type": "https:\/\/didcomm.org\/coordinate-mediation\/2.0\/mediate-grant",
    "body": {
        "routing_did": "did:peer:2.Ez6LSeVvARjb7oQmFsJJaiGz85KZVAfvHpvZHYNeiLv2agorY.Vz6Mkkd27LDZ15HZVPaSzLTVCMfTRnia48XY1cM9X2ieF6iSD.SeyJpZCI6Im5ldy1pZCIsInQiOiJkbSIsInMiOiJodHRwczovL21lZGlhdG9yLmJsb2NrdHJ1c3QuZGV2LyIsInIiOltdLCJhIjpbImRpZGNvbW0vdjIiXX0"
    },
    "thid": "4064f2a6-bc3b-4e0b-a328-4d559ae433d9"
}

Sender sends a forward message

Then, Sender successfully sends a forward message to the mediator and the mediator responds with 202.
I will skip this part, because the forward message is encrypted.

Recipient requests delivery

Sending delivery request message:

{
    "id": "63021717-5e6d-4b30-b29d-889a83a3fdbb",
    "typ": "application\/didcomm-plain+json",
    "type": "https:\/\/didcomm.org\/messagepickup\/3.0\/delivery-request",
    "from": "did:peer:2.Ez6LSkXGpxBFGdGkj5xZGrkiRLSJJXfNvze7d5hhQ8aT6kHXp.Vz6MkwKjkLPuesP9s3WPnArbyerkYM2aYy8egSqsbBS7oRcGr.SeyJ0IjoiZG0iLCJzIjoiaHR0cDovL2hvc3QuZG9ja2VyLmludGVybmFsOjk5OTkvIiwiciI6W10sImEiOlsiZGlkY29tbS92MiJdfQ",
    "to": [
        "did:peer:2.Ez6LSeVvARjb7oQmFsJJaiGz85KZVAfvHpvZHYNeiLv2agorY.Vz6Mkkd27LDZ15HZVPaSzLTVCMfTRnia48XY1cM9X2ieF6iSD.SeyJpZCI6Im5ldy1pZCIsInQiOiJkbSIsInMiOiJodHRwczovL21lZGlhdG9yLmJsb2NrdHJ1c3QuZGV2LyIsInIiOltdLCJhIjpbImRpZGNvbW0vdjIiXX0"
    ],
    "body": {
        "recipient_did": "did:peer:2.Ez6LSkXGpxBFGdGkj5xZGrkiRLSJJXfNvze7d5hhQ8aT6kHXp.Vz6MkwKjkLPuesP9s3WPnArbyerkYM2aYy8egSqsbBS7oRcGr.SeyJ0IjoiZG0iLCJzIjoiaHR0cDovL2hvc3QuZG9ja2VyLmludGVybmFsOjk5OTkvIiwiciI6W10sImEiOlsiZGlkY29tbS92MiJdfQ",
        "limit": 3
    },
    "return_route": "all"
}

In encrypted format:

{
    "ciphertext": "7T-5mkdm6Yh_kSizL7Gn_PtrslaU5VIk1hUWbnzssHD4WhaMFv2mLP5ZKp5jaw0jTJlTSJD4tv70Wz__kP6fK12z3IUwDB1H7MDLa7ZMyK9syc8eNv3XEkfONpn_5WdJFXM0aYzHyxqjF_M4EUID9wD5BuOW62PnIV4jogGtJwfFWzt305fMYwk2E5_drCOkgxm9h6FLn_PlOdui3ME5UKGWisnvK_4a4aDioOy73mcoaYlq1QqemKX_qMFnC-AWlHuMLuN8W3NHyaIzb8g6zFCDvh7ApOy581SuzZ56fm6fedoUCBDaEmJd7RqJiyaMTBq8xEzukNzGEhKEjz403QIeQZBGNKmOTRSI5_l1cnPc_OYkCNVDE3_gAJnvocWyPwWdUAQBdT855hfe9uWgTlXeGAX6IpQssxQm7Tlm-xHaa7a4pGZYfT72ij-KAgGp1ueTrHStCVDc30zwpaILElXf2x9r4uWD-BaLB7l1zLLJhtsoYLQuu8cs4xlaVDj1-osuUwxJk-WHfqFzXXVuUglpdKHfUCNnPjmpp2VLRWXsaWlEBB8f0VoKWzeB3t1eYAMuxhsgdQu_GD5i-lhemqpUrmK1pYMUXbDzCjRK0RjLlWbc6FsIJC2p5tzeURdsGx4zR_5ky_QNVkTpyCcTG5Bjsc4iGLw763S7ix5OvKQCoq5QnBgQnDrQGYTeKeuEYufBKtXk-fSnDwQMvDYGa0olJuaKY5nxDRpmUkA2cgZyn2IKOs_N-PC6ukgWzIcclvciM9J8xjk4ogqoWYnqU13Vj1b0RiFCH-mMCqYLpKyCKGxWbzX8O39XeIU6xQ8kSF1_146g1zw36D1gNH-LthuFt4Qyh9sc2wzkC49mOuMEIDB-VayEgDVNk7Klnyy2yINqAZ-8Ney4XfXzxp32RDrz0jwitgcdkUyBp5hdbR6l1Muw8iYPSSpgNza3KKz2UtuMjiE079LtgtPFr3KjM_5JQDpE1SCN-arN5aGrDLLXmvMT38SQoddwjpscrhf6kNYlQR4DrTZD9zCx892PNmX43hHmBcWuaW5AYcPevr1Xz8N9cAQilDo5JTMT_tmQV3GyLpA-2cH5iRQUumJKTBR7ssDVmGfZdRiMR3TyD0fSHCJaxZ6y9OlLlZztuNbJ0u_cPFTDfCXh3m3YSguJ4vF31OcMdteERu4f7snvpFg",
    "protected": "eyJlcGsiOnsia3R5IjoiT0tQIiwiY3J2IjoiWDI1NTE5IiwieCI6Il9WT0lXbnp4ZlJLT212QXpaVTVIdUZUZjh5c3FZUnZJT1QtTFY1ZmN3ajgifSwiYXB2IjoiREdHWEV4a09DV3pWeldnMzAzYUZsTVZDUWRBaW9CbVBNQVpWYV9Vd3kyYyIsInNraWQiOiJkaWQ6cGVlcjoyLkV6NkxTa1hHcHhCRkdkR2tqNXhaR3JraVJMU0pKWGZOdnplN2Q1aGhROGFUNmtIWHAuVno2TWt3S2prTFB1ZXNQOXMzV1BuQXJieWVya1lNMmFZeThlZ1Nxc2JCUzdvUmNHci5TZXlKMElqb2laRzBpTENKeklqb2lhSFIwY0RvdkwyaHZjM1F1Wkc5amEyVnlMbWx1ZEdWeWJtRnNPams1T1Rrdklpd2ljaUk2VzEwc0ltRWlPbHNpWkdsa1kyOXRiUzkyTWlKZGZRIzZMU2tYR3B4QkZHZEdrajV4Wkdya2lSTFNKSlhmTnZ6ZTdkNWhoUThhVDZrSFhwIiwiYXB1IjoiWkdsa09uQmxaWEk2TWk1RmVqWk1VMnRZUjNCNFFrWkhaRWRyYWpWNFdrZHlhMmxTVEZOS1NsaG1Ublo2WlRka05XaG9VVGhoVkRaclNGaHdMbFo2TmsxcmQwdHFhMHhRZFdWelVEbHpNMWRRYmtGeVlubGxjbXRaVFRKaFdYazRaV2RUY1hOaVFsTTNiMUpqUjNJdVUyVjVTakJKYW05cFdrY3dhVXhEU25wSmFtOXBZVWhTTUdORWIzWk1NbWgyWXpOUmRWcEhPV3BoTWxaNVRHMXNkV1JIVm5saWJVWnpUMnByTlU5VWEzWkphWGRwWTJsSk5sY3hNSE5KYlVWcFQyeHphVnBIYkd0Wk1qbDBZbE01TWsxcFNtUm1VU00yVEZOcldFZHdlRUpHUjJSSGEybzFlRnBIY210cFVreFRTa3BZWms1MmVtVTNaRFZvYUZFNFlWUTJhMGhZY0EiLCJ0eXAiOiJhcHBsaWNhdGlvblwvZGlkY29tbS1lbmNyeXB0ZWQranNvbiIsImVuYyI6IkEyNTZDQkMtSFM1MTIiLCJhbGciOiJFQ0RILTFQVStBMjU2S1cifQ",
    "recipients": [
        {
            "encrypted_key": "YdaoD-h2VNit96y6Hw1mKspZHpgusffI_NxuNcduLJzj7aKjeTz3uEW53rzSU03vUOZajTA-Kt19Pis7dbXL1RD-akQaQZka",
            "header": {
                "kid": "did:peer:2.Ez6LSeVvARjb7oQmFsJJaiGz85KZVAfvHpvZHYNeiLv2agorY.Vz6Mkkd27LDZ15HZVPaSzLTVCMfTRnia48XY1cM9X2ieF6iSD.SeyJpZCI6Im5ldy1pZCIsInQiOiJkbSIsInMiOiJodHRwczovL21lZGlhdG9yLmJsb2NrdHJ1c3QuZGV2LyIsInIiOltdLCJhIjpbImRpZGNvbW0vdjIiXX0#6LSeVvARjb7oQmFsJJaiGz85KZVAfvHpvZHYNeiLv2agorY"
            }
        }
    ],
    "tag": "6gKhxU_mGpuqAT8ikt_-QC7t_KF8mP14eo_Kc6a3JGw",
    "iv": "dU2GPrvysyP1mUluevsiwg"
}

And the mediator returns error 500 with empty response body.

Expected behavior

Mediator successfully delivers the forward message to recipient.

Notes

RootsID mediator successfully delivers the message with the same flow.

[antonbaliasnikov]

Hey @bsandmann !
Could you, please, also check why the delivery request is not working?

[bsandmann]

Hey @antonbaliasnikov
I have my own Testsuite and it was indeed also failing, but I believe due to an issue with Messages and the "To" attributes which has tripping us all up for some time. I pushed a fix to that, so please test again. Thanks for finding these interop issues and reporting them. That's awesome work!

One note to the above request: We don't support the 'limit' yet. We parse it but the database doesn't filter it yet. Minor thing, but not important at the moment I would say. Same thing with paging.

[antonbaliasnikov]

Hey @bsandmann !

Now, it's working, but reports a problem that DID was not found, but mediation was successfully granted. Here is what's going on:

{
    "id": "a97a5202-1c81-41bc-a625-141a078ff4e5",
    "typ": "application\/didcomm-plain+json",
    "type": "https:\/\/didcomm.org\/report-problem\/2.0\/problem-report",
    "body": {
        "escalate_to": "mailto:[email protected]",
        "code": "e.m.me",
        "comment": "Internal error: Recipient DID 'did:peer:2.Ez6LSc6RL3vgLZFgka19kacfiRqwPgd41sUMqweZvT7vTiNxx.Vz6MkjQuarYjsws3t8rWJk1wwKpZbnhnnyHFNVnRbLxXgjwpa.SeyJ0IjoiZG0iLCJzIjoiaHR0cDovL2hvc3QuZG9ja2VyLmludGVybmFsOjk5OTkvIiwiciI6W10sImEiOlsiZGlkY29tbS92MiJdfQ' was not found."
    },
    "pthid": "e6cdc8e0-58fa-433b-aac9-2a9c74adf23d"
}

But at the same time, mediation was successfully granted:

Request:

{
    "id": "110204db-9785-4e91-ae9b-eee3e5ea0d2e",
    "typ": "application\/didcomm-plain+json",
    "type": "https:\/\/didcomm.org\/coordinate-mediation\/2.0\/mediate-request",
    "from": "did:peer:2.Ez6LSc6RL3vgLZFgka19kacfiRqwPgd41sUMqweZvT7vTiNxx.Vz6MkjQuarYjsws3t8rWJk1wwKpZbnhnnyHFNVnRbLxXgjwpa.SeyJ0IjoiZG0iLCJzIjoiaHR0cDovL2hvc3QuZG9ja2VyLmludGVybmFsOjk5OTkvIiwiciI6W10sImEiOlsiZGlkY29tbS92MiJdfQ",
    "to": [
        "did:peer:2.Ez6LSeUYyDHMTbWoMGCKyqntPR95TB3N6ic2A27YLmwZHchxY.Vz6MkgRyq89zDCmXEcg8LmdqKjoaanxK4MUVbbtembDa4fLpK.SeyJpZCI6Im5ldy1pZCIsInQiOiJkbSIsInMiOiJodHRwczovL21lZGlhdG9yLmJsb2NrdHJ1c3QuZGV2LyIsInIiOltdLCJhIjpbImRpZGNvbW0vdjIiXX0"
    ],
    "body": {},
    "return_route": "all"
}

Response:

{
    "id": "69799421-588e-4ce8-b982-c712e7b06fbe",
    "typ": "application\/didcomm-plain+json",
    "type": "https:\/\/didcomm.org\/coordinate-mediation\/2.0\/mediate-grant",
    "body": {
        "routing_did": "did:peer:2.Ez6LSgSy1mqFYhWVa5AsVTHmGn3UngFwwPhsVgfUbA1w2qzoi.Vz6MkoLz7G46Wb8Kq3mYEUKchZsjm2ybe2GG9ZUVYMqBYiY3W.SeyJpZCI6Im5ldy1pZCIsInQiOiJkbSIsInMiOiJodHRwczovL21lZGlhdG9yLmJsb2NrdHJ1c3QuZGV2LyIsInIiOltdLCJhIjpbImRpZGNvbW0vdjIiXX0"
    },
    "thid": "110204db-9785-4e91-ae9b-eee3e5ea0d2e"
}

[bsandmann]

Hey @antonbaliasnikov

I'm currently not so much into the DIDComm topic, so sorry for not having these things correctly in my mind. But I believe the issue is, that you are trying to pickup messages for a recipient which isn't registered yet. At least that is, what the problem report states. Before you can pickup a message or ask for a status you have to update the recipient list.

I believe the flow should go like this:

Locally create Your PeerDID A to connect to the Mediator

MediateRequest for PeerDID A against the Mediator M
-> Mediator M responds with a RoutingDID R

Locally create your new PeerDID A' to be used with someone else, which includes the RoutingDID R as a service endpoint

Send a Recipient Update request against the Mediator M, adding A' to the list for which you can pickup messages 

Pickup message using A with the recipientDID A' (or leave it empty to get the messages for all recipients)

One could argue, that the problem report is the wrong response, and you could also respond with an empty list, but maybe it's more helpful this way. Am I missing something?

[antonbaliasnikov]

Hey @bsandmann !

My flow is as follows (as described in the protocols):

1. Recipient sends mediate-request to register for mediation
2. Mediator answers with mediation-granted (I demonstrated this in the previous thread)
3. Sender sends a message to this already registered recipient
4. When Recipient tries to pick up this message, the mediator responds with report problem that Recipient DID is not registered which is not true

Send a Recipient Update request against the Mediator M, adding A' to the list for which you can pickup messages

Why this is required if a Recipient just registered and mediation was successfully granted? And if it's required, how recipient updates the mediator M?

[bsandmann]

@antonbaliasnikov
The idea is to have the first mediate-request to get the routing DID, to put it into your new DID as the service-endpoint. Maybe there is a way to do this without using the routing_did, but thats currently not supported by us.

After that you have a second request to register the new DID you want to use using the keylist-update for 2.0 or recipient-update for 3.0 (we currently only support 2.0).

Why this is required if a Recipient just registered and mediation was successfully granted?

When you create your DID A, you use that soley for communication between you and the Mediator. The DID A' is soley for the communication with you and Bob. What you are proposing would be using A for the commication between you and the Mediator as well as between you and Bob. Theoretically you could do that, and the mediator could also support that, but that wouldn't be so clean I guess.

And if it's required, how recipient updates the mediator M?

https://didcomm.org/mediator-coordination/2.0/

I know that Roots Mediator also works that way, but I'm not sure how Roots Mediator will respond to your scenario. Did you try that for passing forward messages around?

[antonbaliasnikov]

Thanks @bsandmann

Yeah, the RootsID mediator will correctly deliver a message in my scenario.

But I understood your idea, it makes total sense.
I will take a look at how it works with your mediator further.

[bsandmann]

@antonbaliasnikov
I'll look into 'fixing' the issue, but I'm a bit hesitant if that fix would really be an improvement. It would then primarily be just to be compatible across the board.

[antonbaliasnikov]

@bsandmann , yeah, nothing to "fix" really... but interesting to understand how it's supposed to work.

We will be discussing this further with DIDComm community.

For reference, see: decentralized-identity/didcomm.org#88

[FabioPinheiro]

@antonbaliasnikov I'll look into 'fixing' the issue, but I'm a bit hesitant if that fix would really be an improvement. It would then primarily be just to be compatible across the board.

I wouldn't fix it right yet. Since we don't know what should be the behavior, I can argue from both sides.
We feel that are a lot of ambiguities on the protocols ATM. That's why @antonbaliasnikov is invested time in making a generic test suite for the Mediator protocols.

[FabioPinheiro]

Seems like Blocktrust's Mediator has the correct behavior.
This was the conclusion from the last DIF DID Comm meeting. Here is the recording https://us02web.zoom.us/rec/play/UW_8y9H8nfALu18XV5zJ7jCYvVw7PV8fbxUk3TVXXsRuQXs71vu[…]IIeSpagNEJDjJC4b55d8P3o4MWZDTthFA8ehMTovnThy.qUtcyz8pPHOlo9YG

@antonbaliasnikov we need the change our implementation and the test.

[bsandmann]

@FabioPinheiro @antonbaliasnikov
Thanks for bringing up the issue in the DIF meeting and clearing that up. If you discover any other issues where our Mediator might not show a fully compatible behavior, please continue notifying me. You are doing a great job with testing and ensuring compatibility!