Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch load messages with filters #2006

Closed
toger5 opened this issue Oct 28, 2021 · 1 comment
Closed

Batch load messages with filters #2006

toger5 opened this issue Oct 28, 2021 · 1 comment

Comments

@toger5
Copy link
Contributor

toger5 commented Oct 28, 2021

Is your feature request related to a problem? Please describe.
For widgets and other applications where the messages are used to store information that is required to recreate data, It would be very helpful to load all the relevant events at once.
Examples for those situations are:

Describe the solution you'd like
An endpoint that allows to load multiple filtered events at once (a more sophisticated scroll back function):

function loadMessages(originServerTsFilter=[0,now], 
ordering="oldToNew"/"NewToOld", 
typeFilter=["all"/"m.room.message"], 
contentFilter=[{"msgtype":"m.text", ... }], 
senderFilter=["@matrixId1","@matrixId2",...], 
pageSize, page)

The filtering is based on the mandatory field of a matrix message: type, content, sender, origin_server_ts. An array of dictionaries/strings is provided and all matching events get collected on the server and send to the client.
Idealy there would also be a pagination system in place. Setting pageSize and page request a subset of the filtered events. To guarantee that page X always contains the same elements it is necessary to use the same value for the originServerTsFilter [constTimeA, constTimeB] for each request.
The response should be the array of events and the amount of events that conform to the filter. The server should have the option to respond with an error (too many events) if the filter is set in a way that the servers performance would suffer from collecting the events. The client than is responsible to set other conditions (a smaller time window, smaller page size) to get the desired events.
Describe alternatives you've considered
Alternatively the filtering can be done on the client. A lot of scrollback requests need to be send and the relevant events need to be selected locally. Dependent on the situation this can lead to a lot of unnecessary overhead. It also does not allow for any optimizations. The proposed solution could allow a stripped flag so only the event content is send for example.

Additional context
This is also required to (in the future) give widgets the option to read all existing events of a certain type as requested in:
element-hq/element-web#17994

@toger5
Copy link
Contributor Author

toger5 commented Dec 16, 2021

This is already part of the spec:
https://spec.matrix.org/v1.1/client-server-api/#get_matrixclientv3roomsroomidmessages
The filter parameter can be used for that.

@toger5 toger5 closed this as completed Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant