POST /read_markers fails after history purge if cache is cleared

[PiotrKozimor]

Description

POST /read_markers fails when event at which m.fully_read points is cleared using history purge.

Steps to reproduce

1. have a two rooms with two users joined each
2. type messages in Element in first room with first user
3. switch to second room
4. send new events with second user
5. purge history up to latest event in a first room
6. restart synapse so that cache is cleared
7. on first user switch to previous room
8. POST /read_marker FAILS with 404
9. sending new message doesn't modify read marker

Looks like the only current workaround is to edit database manually :/ If synapse is not restarted, event is fetched from cache and read marker can be edited

$ curl --location --request POST 'https://hs.stg.globekeeper.com/_matrix/client/r0/rooms/!DZjumzOmWaqeeaDKtc:hs.stg.globekeeper.com/read_markers' \
--header 'Authorization: Bearer SECRET' \
--header 'Content-Type: application/json' \
--data-raw '{
  "m.fully_read": "$O8Mg-Vo5krkxojC1T0zf4q_h7k21s93qHbCnWVBFGAs"
}'
{"errcode":"M_UNKNOWN","error":"Could not find event $TV2s-Rsp0BWwTzWBHo-4FSCk-q0MAA5HfsHNi1ls4zo"}

m.fully_read points to non-existing event after history purge:

synapse1=# select * from room_account_data where user_id = '@tcayxbre:hs.stg.globekeeper.com' and room_id = '!DZjumzOmWaqeeaDKtc:hs.stg.globekeeper.com';
             user_id              |                  room_id                   | account_data_type | stream_id |                           content           | instance_name
----------------------------------+--------------------------------------------+-------------------+-----------+------------------------------------------------------------+---------------
 @tcayxbre:hs.stg.globekeeper.com | !DZjumzOmWaqeeaDKtc:hs.stg.globekeeper.com | m.fully_read      |      4051 | {"event_id":"$TV2s-Rsp0BWwTzWBHo-4FSCk-q0MAA5HfsHNi1ls4zo"} |
(1 row)

synapse1=# select * from events where event_id = '$TV2s-Rsp0BWwTzWBHo-4FSCk-q0MAA5HfsHNi1ls4zo';
 topological_ordering | event_id | type | room_id | content | unrecognized_keys | processed | outlier | depth | origin_server_ts | received_ts | sender | contains_url | instance_name | stream_ordering
----------------------+----------+------+---------+---------+-------------------+-----------+---------+-------+-----------------+-------------+--------+--------------+---------------+-----------------
(0 rows)

Version information

• Homeserver: hs.stg.globekeeper.com

• Version: 1.49.2 (with small, irrelevant modifications: https://github.com/globekeeper/synapse/commits/develop)

• Install method:docker

• Platform: GKE/containerd

[H-Shay]

I am attempting to reproduce this and just had a few questions on the instructions:

• have a room (room 1)
• type messages in Element (log in to Element with account 1 and send some messages in room 1)
• switch to different room (room 2)
• send new events with second user (log user 1 out of Element and log user 2 in, send some messages in room 2)
• purge history up to latest event in a room (which room? room 1, 2, both or either?)
• restart synapse so that cache is cleared
• on first user switch to previous room (log in to Element with account 1 and send some messages in room 1?)
• POST /read_marker FAILS with 404
• sending new message doesn't modify read marker (sending new messages in which room with which user?)

Can you answer the questions and let me know if my interpretation of the instructions are clear? Thanks!

[PiotrKozimor]

I have numbered steps for clarity.
4. I used guest account in Chrome and had two clients opened at once. Log out is not necessary.
5. In a first room.
7. See 4. - please switch between clients.
8,9. First room, first client.

[richvdh]

related: #11521

[marvinwankersteen]

The tips here have saved me a lot of work.
My database crashed unexpectedly, which caused Element on two PCs to have an eventID in the cache that was not present in the database.

I have displayed an existing eventID in the raw data in Element and set this in the database:

update room_account_data set content = '{"event_id":"$<event-id>:<server>"}' where user_id = '@<user>:<server>' and room_id = '!<room-id>:<server>' and account_data_type = 'm.fully_read';

Synapse must be restarted after the change or stop Synapse before the change.

Additionally, I had to log off the users in Element and clear the local cache from the browser/electron via the Developer tools.

[cristianyxen]

For me I fixed it like that cristianyxen@d5aed2c

[rettichschnidi]

For me I fixed it like that cristianyxen@d5aed2c

Seems like a reasonable workaround, and is working for me. Thanks!

[ThoreKr]

Is this the root cause of element-hq/element-web#24111 as well?