unbans do not propagate to federated servers (SYN-725)

[matrixbot]

Submitted by @​richvdh:sw1v.org

1. ban user on remote server
2. re-invite user on remote server
3. user receives invite but their HS still thinks they are banned

(Imported from https://matrix.org/jira/browse/SYN-725)

---

Complement test blocked by this issue: https://github.com/matrix-org/complement/blob/3865b81da267f9c5e904e58b234c0a57a262ec04/tests/federation_room_ban_test.go?rgh-link-date=2016-07-13T19%3A37%3A37Z#L15

[richvdh]

Ideas for workarounds (currently untested):

method 1

• Create second user on homeserver.
• Join room using that user. That should pull the latest room state
• You should then be able to join with the original user.
• If desired, leave the room from the second user, and deactivate the second user account.

method 2

Alternatively, if you are sure that this is the only user in the room on the affected homeserver, you can try some surgery on your database. Take a backup first, in case of mistakes.

• Run this query on your homeserver database (substitute !room_id:domain appropriately)

select * from event_forward_extremities where room_id='<!room_id:domain>';

• We expect to see one result. Keep note of it, in case this blows up. If there is more than one, or zero, you have other problems.
• Shut down synapse
• Run the following query. Make sure the 'where' part exactly matches what you had in the 'select' query above:

delete from event_forward_extremities where room_id='<!room_id:domain>';

• Restart synapse
• Try joining the room again.

---

Please let me know if you have any success with either of these methods

[MacLemon]

Thanks to richvdh for getting back to me on that issue via Twitter/Matrix.

I tried Method 1 with my federated homeserver and it worked fine.
Joining the room with the temporary account synched the room and allowed the accidentally banned, then unbanned account to join again immediately.

My homeserver was running synapse 0.20.0 on FreeBSD 11 at that time.
Interestingly when the unban message is missing on a homeserver and the user gets an invitation to join the room in the meantime, you end up with Schroedingers invitation. You cannot accept the invitation, you cannot decline the invitation and you can't get rid of the invitation.

I haven't tried method 2.

[richvdh]

neil: really? we want this for v1.0? it's not easy :/

[Xananax]

I tried both method 1 and method 2, to no avail. My attempts sometimes still show me as banned, sometimes tell me I can't preview the room, sometimes allow me to preview, but joining just keeps spinning forever to no avail.

Other people see me as having joined (multiple times), but the spinning wheel never stops, the room never gets added to my list, and I can never speak. I've tried multiple flavors of Riot.

More context can be found in this issue

The event that gets received in the rooms looks like:

{
  "content": {
    "avatar_url": "mxc://mutnt.io/xtfOWjOPxlUDntuZcsbpnRbc",
    "displayname": "Xananax",
    "membership": "join"
  },
  "event_id": "$15698602680XKphH:mutnt.io",
  "origin_server_ts": 1569860268236,
  "sender": "@xananax:mutnt.io",
  "state_key": "@xananax:mutnt.io",
  "type": "m.room.member",
  "unsigned": {
    "age": 108,
    "replaces_state": "$1569573974171540mjBIK:matrix.org",
    "prev_content": {
      "membership": "ban"
    },
    "prev_sender": "@abuse:matrix.org"
  },
  "room_id": "!QtykxKocfZaZOUrTwp:matrix.org"
}

[anoadragon453]

FYI this is still a problem.

[grinapo]

It is a bug since 2016. Still bites hard.

Can't invites trigger unban propagation? :-(

Actually why does synapse keep a ban info for a room he's not member of?

[ghost]

What is the status of this? We recently added ban to our matrix client application, then ran into this issue and had to disable ban again. This is a real blocker and makes the whole ban feature unusable. Leaving users with zombie invites they cannot get rid of is not exactly great user experience.

EDIT: Ok, it's kinda edge-case. As soon as more than one user from that other federated server exist in the room, the issue does not occur. I assume it's because the server thinks "no more people in this room, no need to send federated information". Probably a non-issue in real life usage.

[callahad]

Probably a non-issue in real life usage.

We agree that this is a valid issue, but as you noted, it should be relatively rare in practice. We'd gladly accept patches, but have not currently scheduled time to work on it ourselves.

[callahad]

(Apologies, corrected a typo above. We have not currently scheduled time to work on it this ourselves. Patches very welcome.)

[subiol]

I can confirm this. It just happened to me in an invite only room. I was testing stuff, and I was banned. Then unbanned and reinvited. Now I can not join or reject the invitation, reason given is I am banned from the room (why would being banned not allow you to reject the invitation?), despite not really being banned in the room.

[joepie91]

Just to confirm, I also seem to have been hit by this issue a while ago. Worked around it at the time in some way I don't remember (SDK API method from Electron devtools, I think?), and then it reappeared after clearing my cache. Shutting down the room on the homeserver side "fixed" it.

[grinapo]

Would knock #6739 resolve this?

[ShadowJonathan]

I don't think so, to me this looks like local homeserver validation, where the perceived state is still ban, and you can't change your membership state from ban to knock

[maxinasa]

Hello! I think I also stumbled upon this issue. I found a workaround for this by disinviting the user and inviting it again. On the second invite, after the unban, I can accept or decline the invitation.

[anoadragon453]

Would knock #6739 resolve this?

Knock rejections over federation also suffer from this issue.

[timokoesters]

My idea to fix this in Conduit is to always use the /make_join /send_join endpoints to join if there are currently no joined users in the room.

[MTRNord]

This seems to still happen. We just hit this bug while unbanning a user. They kept seeing that they are banned

[ShadowJonathan]

Interestingly, while a test for this exists in complement, the test only skips for Synapse, which implies that it runs well on Dendrite.

Does that mean that there's a way for Synapse to copy the behaviour/implementation for unbanning from Dendrite to achieve the same behaviour? Or is it a problem when receiving an unban event through federation transactions?

[anoadragon453]

For what it's worth, as a modern workaround you should be able to delete the local copy of the room via the Admin API, and then rejoin the room.

Interestingly, while a test for this exists in complement, the test only skips for Synapse, which implies that it runs well on Dendrite.

In this case, I expect that Synapse is just not sending the unban over federation (because the homeserver is no longer in the room). Without that, the remote Synapse just responds to join attempts from its own users with "sorry, you're banned" because it looks at the local copy of the room that is now stale. Remember, rooms are replicated between homeservers as you participate in them, and becoming disconnected from the federation and later rejoining is considered a valid scenario (the room should keep working).

So, when a user is unbanned, we should special-case and send that unban over federation to the unbanned user's homeserver, right?

Well, sending the unban or knock rejection (aka a leave membership) over federation is a little tricky. The remote homeserver may not be able to validate that unban membership event because it doesn't have the full state of the room, and won't be able to request the missing state as its not joined to the room. To solve this, we need to ignore the local room state and try to rejoin the room, thus overwriting our local state of the room. Perhaps an unban could signal to the remote homeserver to do this on the next join attempt.

You can invite the user after unbanning them, as the linked Complement test does, and the remote homeserver will receive that invite. I imagine Synapse now has some internal confusion about inserting that invite event into its local copy of the room, whereas Dendrite just accepts it.

[ShadowJonathan]

Personally, I believe that that (getting unbans propagated by invites) is an acceptable and user-workable solution to this issue, even though it might not close it.

How can I best ask the synapse team to work on that specific behaviour (invite after ban triggers recalc and fetch) for synapse? If only to get this test unskipped on complement.

[anoadragon453]

invite after ban triggers recalc and fetch

Note that you would only attempt to rejoin the room if a client asked you to (after the invite). Otherwise you'd end up with a DoS vector.

How can I best ask the synapse team to work on that specific behaviour.

Well, this issue's already been triaged; we'll relabel it if the priority changes. As it stands in this case, we'd accept PRs for it. And if you want implementation guidance for one, feel free to pop into #synapse-dev:matrix.org.

[anoadragon453]

(And feel free to poke on issues as you did here if you believe things have become more urgent or on fire than they were when they were triaged.)

[deepbluev7]

Hi, let me poke this: I am currently facing the issue, that every time I ban someone from a small homeserver, I can't unban them. This basically makes it impossible to redact bans in heavily federated rooms. This is not an issue for matrix.org users, since usually there is another matrix.org user in the room, but it heavily impacts small homeservers. My suggestion is to just ignore the local room state for joins, if all the local users are banned or no user from our server is currently in the "join" state. The easiest is probably to just ignore the local state, if all local users are in the ban state, since that should fix the cases I usually run into, where only a single user is in the room from that server (which is basically what usually happens, when you selfhost).