Do not consider events by ignored users for bundled aggregations #12235

clokep · 2022-03-16T14:45:55Z

This is the last bit needed for #11753; we need to ensure that if a user ignores another user, their related events are properly ignored in bundled aggregations.

See #12285 for filtering events from ignored users in /relations.

Note that for the cached methods we per-calculate the list of ignored users and pass it into each other method. The reasoning for this is that users who have the same set of ignored users will then share the cache (as opposed to if we passed in the requester) -- my thought being that most users will not have anyone ignored or that it is highly likely for a small set of users to be ignored. (I.e. essentially that there's a small set of people who get ignored for being spammy.)

Should be reviewable commit-by-commit.

clokep · 2022-03-21T18:27:33Z

synapse/handlers/relations.py

+        # Note that there is no use in limiting edits by ignored users since the
+        # parent event should be ignored in the first place if the user is ignored.


Does this sound like sound logic? I couldn't come up with a situation where we would need to check edits,

Yeah, I think it's probably fine. The only thing is if we decided in future to allow e.g. moderators to edit messages.

clokep · 2022-03-21T18:29:26Z

synapse/handlers/relations.py

+        # Note that ignored users are not passed into get_relations_for_event
+        # below. Ignored users are handled in filter_events_for_client (and by
+        # noy passing them in here we should get a better cache hit rate).


Does this sound reasonable or do we think we should do the initial filtering by ignoring users too? (I think we would essentially do it twice then -- once in get_relations_for_event, and then again in filter_events_for_client.

Makes sense so long as we have this comment here.

erikjohnston · 2022-03-24T10:40:17Z

Note that for the cached methods we per-calculate the list of ignored users and pass it into each other method. The reasoning for this is that users who have the same set of ignored users will then share the cache (as opposed to if we passed in the requester) -- my thought being that most users will not have anyone ignored or that it is highly likely for a small set of users to be ignored. (I.e. essentially that there's a small set of people who get ignored for being spammy.)

My hunch is that we'd pretty much only hit the cache for users with zero ignored users, as I think its going to be relatively rare for different users to have the exact same set of ignored users (even if they are mostly the same). It also means that we cache the set of ignored users in the cache key, which feels weird and very bleurgh for people with large numbers of ignored users (which would be rare tbf)

For get_relations_for_event can we instead lift the filtering of the ignored users out of the DB query? (You might want to do it in a loop to fetch more results to replace any filtered out).

I'm not really sure what to do about calculating the counts in the other queries. Some thoughts I have are:

Filter ignored users to those that are also in the room, to hopefully increase cache hits
Don't bother caching the query if ignored_users is non-empty
Have a separate cached query that looks up the counts by user, e.g. for get_aggregation_groups_for_event could have a query to look up (event_id, sender) -> (type, aggregation_key) so that you could call that for each ignored user, and use the results to fiddle with the counts. 🤷

Thoughts?

clokep · 2022-03-24T11:24:04Z

For get_relations_for_event can we instead lift the filtering of the ignored users out of the DB query? (You might want to do it in a loop to fetch more results to replace any filtered out).

I originally thought about having it also use the filter_events_for_client, but that would require resolving the event IDs to events, which is not cheap. I suppose we could also request the sender and do the filtering in Python though. This might need a bit more of refactoring...as right now the results of get_relations_for_event are essentially splatted out into responses (and bundled aggregations).

Filter ignored users to those that are also in the room, to hopefully increase cache hits

That could maybe work?

Don't bother caching the query if ignored_users is non-empty

This sounds like the "easy" thing to do, but I'd be curious what percentage of users ignore at least someone... otherwise we might just be making the cache useless.

Have a separate cached query that looks up the counts by user, e.g. for get_aggregation_groups_for_event could have a query to look up (event_id, sender) -> (type, aggregation_key) so that you could call that for each ignored user, and use the results to fiddle with the counts. 🤷

Something like that might work, I'll investigate. I think it is a bit more fiddly, but probably the best plan?

Thanks for the thoughts! I'll poke a bit more at this.

erikjohnston · 2022-03-24T11:32:57Z

This sounds like the "easy" thing to do, but I'd be curious what percentage of users ignore at least someone... otherwise we might just be making the cache useless.

Yeah, no idea, though if you also filter the ignored users to those in the room it might not be too bad? 🤷

erikjohnston

(Bah, forgot to hit the submit review button)

erikjohnston · 2022-03-24T10:40:59Z

synapse/handlers/relations.py

+        # Note that there is no use in limiting edits by ignored users since the
+        # parent event should be ignored in the first place if the user is ignored.


Yeah, I think it's probably fine. The only thing is if we decided in future to allow e.g. moderators to edit messages.

erikjohnston · 2022-03-24T10:41:29Z

synapse/handlers/relations.py

+        # Note that ignored users are not passed into get_relations_for_event
+        # below. Ignored users are handled in filter_events_for_client (and by
+        # noy passing them in here we should get a better cache hit rate).


Makes sense so long as we have this comment here.

…rs-relations

clokep · 2022-04-06T13:23:46Z

synapse/storage/databases/main/relations.py


            return [{"type": row[0], "key": row[1], "count": row[2]} for row in txn]

        return await self.db_pool.runInteraction(
            "get_aggregation_groups_for_event", _get_aggregation_groups_for_event_txn
        )

+    async def get_aggregation_groups_for_users(


Not sure if this needs caching?

I don't think these need caching, given that the common case will be a no-op anyway. We can always revisit it if we we see it taking time

clokep · 2022-04-06T13:23:54Z

synapse/storage/databases/main/relations.py

@@ -521,6 +587,67 @@ def _get_thread_summaries_txn(

        return summaries

+    async def get_threaded_messages_per_user(


Not sure if this needs caching?

clokep · 2022-04-06T13:24:53Z

@erikjohnston Could you take another look at this? I attempted to take the idea of having the "normal" query, which is cached, and then substracting out info from ignored users, as you proposed. Let me know if you want to discuss!

erikjohnston

This makes sense to me, modulo checking that the queries are good

synapse/storage/database.py

synapse/storage/databases/main/relations.py

synapse/handlers/relations.py

erikjohnston · 2022-04-08T09:41:08Z

synapse/storage/databases/main/relations.py


            return [{"type": row[0], "key": row[1], "count": row[2]} for row in txn]

        return await self.db_pool.runInteraction(
            "get_aggregation_groups_for_event", _get_aggregation_groups_for_event_txn
        )

+    async def get_aggregation_groups_for_users(


I don't think these need caching, given that the common case will be a no-op anyway. We can always revisit it if we we see it taking time

erikjohnston · 2022-04-08T09:42:32Z

synapse/storage/databases/main/relations.py

+            WHERE relates_to_id = ? AND room_id = ? AND relation_type = ? AND {users_sql}
+            GROUP BY relation_type, type, aggregation_key
+            ORDER BY COUNT(*) DESC
+            LIMIT ?


Have you checked the query plans for these? I'm struggling to reason about if these are "safe" or not?

Good call! Will do.

@erikjohnston and I took a look at this and it seems like the query plans used are reasonable.

…rs-relations

clokep · 2022-04-11T13:37:05Z

@erikjohnston I made the changes requested and we discussed the query plans, I think this is probably good to go. I've just merged in develop again to double check CI!

erikjohnston

Synapse 1.57.0 (2022-04-19) =========================== This version includes a [change](matrix-org/synapse#12209) to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#upgrading-to-v1570) for more details. No significant changes since 1.57.0rc1. Synapse 1.57.0rc1 (2022-04-12) ============================== Features -------- - Send device list changes to application services as specified by [MSC3202](matrix-org/matrix-spec-proposals#3202), using unstable prefixes. The `msc3202_transaction_extensions` experimental homeserver config option must be enabled and `org.matrix.msc3202: true` must be present in the application service registration file for device list changes to be sent. The "left" field is currently always empty. ([\#11881](matrix-org/synapse#11881)) - Optimise fetching large quantities of missing room state over federation. ([\#12040](matrix-org/synapse#12040)) - Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. ([\#12251](matrix-org/synapse#12251)) - Move `update_client_ip` background job from the main process to the background worker. ([\#12252](matrix-org/synapse#12252)) - Add a module callback to react to new 3PID (email address, phone number) associations. ([\#12302](matrix-org/synapse#12302)) - Add a configuration option to remove a specific set of rooms from sync responses. ([\#12310](matrix-org/synapse#12310)) - Add a module callback to react to account data changes. ([\#12327](matrix-org/synapse#12327)) - Allow setting user admin status using the module API. Contributed by Famedly. ([\#12341](matrix-org/synapse#12341)) - Reduce overhead of restarting synchrotrons. ([\#12367](matrix-org/synapse#12367), [\#12372](matrix-org/synapse#12372)) - Update `/messages` to use historic pagination tokens if no `from` query parameter is given. ([\#12370](matrix-org/synapse#12370)) - Add a module API for reading and writing global account data. ([\#12391](matrix-org/synapse#12391)) - Support the stable `v1` endpoint for `/relations`, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#12403](matrix-org/synapse#12403)) - Include bundled aggregations in search results ([MSC3666](matrix-org/matrix-spec-proposals#3666)). ([\#12436](matrix-org/synapse#12436)) Bugfixes -------- - Fix a long-standing bug where updates to the server notices user profile (display name/avatar URL) in the configuration would not be applied to pre-existing rooms. Contributed by Jorge Florian. ([\#12115](matrix-org/synapse#12115)) - Fix a long-standing bug where events from ignored users were still considered for bundled aggregations. ([\#12235](matrix-org/synapse#12235), [\#12338](matrix-org/synapse#12338)) - Fix non-member state events not resolving for historical events when used in [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` `state_events_at_start`. ([\#12329](matrix-org/synapse#12329)) - Fix a long-standing bug affecting URL previews that would generate a 500 response instead of a 403 if the previewed URL includes a port that isn't allowed by the relevant blacklist. ([\#12333](matrix-org/synapse#12333)) - Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. ([\#12350](matrix-org/synapse#12350)) - Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. ([\#12364](matrix-org/synapse#12364), [\#12410](matrix-org/synapse#12410)) - Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. ([\#12390](matrix-org/synapse#12390)) Updates to the Docker image --------------------------- - Bundle locked versions of dependencies into the Docker image. ([\#12385](matrix-org/synapse#12385), [\#12439](matrix-org/synapse#12439)) - Fix up healthcheck generation for workers docker image. ([\#12405](matrix-org/synapse#12405)) Improved Documentation ---------------------- - Clarify documentation for running SyTest against Synapse, including use of Postgres and worker mode. ([\#12271](matrix-org/synapse#12271)) - Document the behaviour of `LoggingTransaction.call_after` and `LoggingTransaction.call_on_exception` methods when transactions are retried. ([\#12315](matrix-org/synapse#12315)) - Update dead links in `check-newsfragment.sh` to point to the correct documentation URL. ([\#12331](matrix-org/synapse#12331)) - Upgrade the version of `mdbook` in CI to 0.4.17. ([\#12339](matrix-org/synapse#12339)) - Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. ([\#12345](matrix-org/synapse#12345)) - Update the link to Redis pub/sub documentation in the workers documentation. ([\#12369](matrix-org/synapse#12369)) - Remove documentation for converting a legacy structured logging configuration to the new format. ([\#12392](matrix-org/synapse#12392)) Deprecations and Removals ------------------------- - Remove the unused and unstable `/aggregations` endpoint which was removed from [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#12293](matrix-org/synapse#12293)) Internal Changes ---------------- - Remove lingering unstable references to MSC2403 (knocking). ([\#12165](matrix-org/synapse#12165)) - Avoid trying to calculate the state at outlier events. ([\#12191](matrix-org/synapse#12191), [\#12316](matrix-org/synapse#12316), [\#12330](matrix-org/synapse#12330), [\#12332](matrix-org/synapse#12332), [\#12409](matrix-org/synapse#12409)) - Omit sending "offline" presence updates to application services after they are initially configured. ([\#12193](matrix-org/synapse#12193)) - Switch to using a sequence to generate AS transaction IDs. Contributed by Nick @ Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. ([\#12209](matrix-org/synapse#12209)) - Add missing type hints for storage. ([\#12267](matrix-org/synapse#12267)) - Add missing type definitions for scripts in docker folder. Contributed by Jorge Florian. ([\#12280](matrix-org/synapse#12280)) - Move [MSC2654](matrix-org/matrix-spec-proposals#2654) support behind an experimental configuration flag. ([\#12295](matrix-org/synapse#12295)) - Update docstrings to explain how to decipher live and historic pagination tokens. ([\#12317](matrix-org/synapse#12317)) - Add ground work for speeding up device list updates for users in large numbers of rooms. ([\#12321](matrix-org/synapse#12321)) - Fix typechecker problems exposed by signedjson 1.1.2. ([\#12326](matrix-org/synapse#12326)) - Remove the `tox` packaging job: it will be redundant once #11537 lands. ([\#12334](matrix-org/synapse#12334)) - Ignore `.envrc` for `direnv` users. ([\#12335](matrix-org/synapse#12335)) - Remove the (broadly unused, dev-only) dockerfile for pg tests. ([\#12336](matrix-org/synapse#12336)) - Remove redundant `get_success` calls in test code. ([\#12346](matrix-org/synapse#12346)) - Add type annotations for `tests/unittest.py`. ([\#12347](matrix-org/synapse#12347)) - Move single-use methods out of `TestCase`. ([\#12348](matrix-org/synapse#12348)) - Remove broken and unused development scripts. ([\#12349](matrix-org/synapse#12349), [\#12351](matrix-org/synapse#12351), [\#12355](matrix-org/synapse#12355)) - Convert `Linearizer` tests from `inlineCallbacks` to async. ([\#12353](matrix-org/synapse#12353)) - Update docstrings for `ReadWriteLock` tests. ([\#12354](matrix-org/synapse#12354)) - Refactor `Linearizer`, convert methods to async and use an async context manager. ([\#12357](matrix-org/synapse#12357)) - Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. ([\#12358](matrix-org/synapse#12358)) - Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. ([\#12366](matrix-org/synapse#12366)) - Add type hints to tests files. ([\#12371](matrix-org/synapse#12371)) - Allow specifying the Postgres database's port when running unit tests with Postgres. ([\#12376](matrix-org/synapse#12376)) - Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. ([\#12379](matrix-org/synapse#12379)) - Add opentracing spans to calls to external cache. ([\#12380](matrix-org/synapse#12380)) - Lay groundwork for using `poetry` to manage Synapse's dependencies. ([\#12381](matrix-org/synapse#12381), [\#12407](matrix-org/synapse#12407), [\#12412](matrix-org/synapse#12412), [\#12418](matrix-org/synapse#12418)) - Make missing `importlib_metadata` dependency explicit. ([\#12384](matrix-org/synapse#12384), [\#12400](matrix-org/synapse#12400)) - Update type annotations for compatiblity with prometheus_client 0.14. ([\#12389](matrix-org/synapse#12389)) - Remove support for the unstable identifiers specified in [MSC3288](matrix-org/matrix-spec-proposals#3288). ([\#12398](matrix-org/synapse#12398)) - Add missing type hints to configuration classes. ([\#12402](matrix-org/synapse#12402)) - Add files used to build the Docker image used for complement testing into the Synapse repository. ([\#12404](matrix-org/synapse#12404)) - Do not include groups in the sync response when disabled. ([\#12408](matrix-org/synapse#12408)) - Improve type hints related to HTTP query parameters. ([\#12415](matrix-org/synapse#12415)) - Stop maintaining a list of lint targets. ([\#12420](matrix-org/synapse#12420)) - Make `synapse._scripts` pass type checks. ([\#12421](matrix-org/synapse#12421), [\#12422](matrix-org/synapse#12422)) - Add some type hints to datastore. ([\#12423](matrix-org/synapse#12423)) - Enable certificate checking during complement tests. ([\#12435](matrix-org/synapse#12435)) - Explicitly specify the `tls` extra for Twisted dependency. ([\#12444](matrix-org/synapse#12444))

Synapse 1.57.0 (2022-04-19) =========================== This version includes a [change](matrix-org#12209) to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#upgrading-to-v1570) for more details. No significant changes since 1.57.0rc1. Synapse 1.57.0rc1 (2022-04-12) ============================== Features -------- - Send device list changes to application services as specified by [MSC3202](matrix-org/matrix-spec-proposals#3202), using unstable prefixes. The `msc3202_transaction_extensions` experimental homeserver config option must be enabled and `org.matrix.msc3202: true` must be present in the application service registration file for device list changes to be sent. The "left" field is currently always empty. ([\matrix-org#11881](matrix-org#11881)) - Optimise fetching large quantities of missing room state over federation. ([\matrix-org#12040](matrix-org#12040)) - Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. ([\matrix-org#12251](matrix-org#12251)) - Move `update_client_ip` background job from the main process to the background worker. ([\matrix-org#12252](matrix-org#12252)) - Add a module callback to react to new 3PID (email address, phone number) associations. ([\matrix-org#12302](matrix-org#12302)) - Add a configuration option to remove a specific set of rooms from sync responses. ([\matrix-org#12310](matrix-org#12310)) - Add a module callback to react to account data changes. ([\matrix-org#12327](matrix-org#12327)) - Allow setting user admin status using the module API. Contributed by Famedly. ([\matrix-org#12341](matrix-org#12341)) - Reduce overhead of restarting synchrotrons. ([\matrix-org#12367](matrix-org#12367), [\matrix-org#12372](matrix-org#12372)) - Update `/messages` to use historic pagination tokens if no `from` query parameter is given. ([\matrix-org#12370](matrix-org#12370)) - Add a module API for reading and writing global account data. ([\matrix-org#12391](matrix-org#12391)) - Support the stable `v1` endpoint for `/relations`, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\matrix-org#12403](matrix-org#12403)) - Include bundled aggregations in search results ([MSC3666](matrix-org/matrix-spec-proposals#3666)). ([\matrix-org#12436](matrix-org#12436)) Bugfixes -------- - Fix a long-standing bug where updates to the server notices user profile (display name/avatar URL) in the configuration would not be applied to pre-existing rooms. Contributed by Jorge Florian. ([\matrix-org#12115](matrix-org#12115)) - Fix a long-standing bug where events from ignored users were still considered for bundled aggregations. ([\matrix-org#12235](matrix-org#12235), [\matrix-org#12338](matrix-org#12338)) - Fix non-member state events not resolving for historical events when used in [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` `state_events_at_start`. ([\matrix-org#12329](matrix-org#12329)) - Fix a long-standing bug affecting URL previews that would generate a 500 response instead of a 403 if the previewed URL includes a port that isn't allowed by the relevant blacklist. ([\matrix-org#12333](matrix-org#12333)) - Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. ([\matrix-org#12350](matrix-org#12350)) - Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. ([\matrix-org#12364](matrix-org#12364), [\matrix-org#12410](matrix-org#12410)) - Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. ([\matrix-org#12390](matrix-org#12390)) Updates to the Docker image --------------------------- - Bundle locked versions of dependencies into the Docker image. ([\matrix-org#12385](matrix-org#12385), [\matrix-org#12439](matrix-org#12439)) - Fix up healthcheck generation for workers docker image. ([\matrix-org#12405](matrix-org#12405)) Improved Documentation ---------------------- - Clarify documentation for running SyTest against Synapse, including use of Postgres and worker mode. ([\matrix-org#12271](matrix-org#12271)) - Document the behaviour of `LoggingTransaction.call_after` and `LoggingTransaction.call_on_exception` methods when transactions are retried. ([\matrix-org#12315](matrix-org#12315)) - Update dead links in `check-newsfragment.sh` to point to the correct documentation URL. ([\matrix-org#12331](matrix-org#12331)) - Upgrade the version of `mdbook` in CI to 0.4.17. ([\matrix-org#12339](matrix-org#12339)) - Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. ([\matrix-org#12345](matrix-org#12345)) - Update the link to Redis pub/sub documentation in the workers documentation. ([\matrix-org#12369](matrix-org#12369)) - Remove documentation for converting a legacy structured logging configuration to the new format. ([\matrix-org#12392](matrix-org#12392)) Deprecations and Removals ------------------------- - Remove the unused and unstable `/aggregations` endpoint which was removed from [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\matrix-org#12293](matrix-org#12293)) Internal Changes ---------------- - Remove lingering unstable references to MSC2403 (knocking). ([\matrix-org#12165](matrix-org#12165)) - Avoid trying to calculate the state at outlier events. ([\matrix-org#12191](matrix-org#12191), [\matrix-org#12316](matrix-org#12316), [\matrix-org#12330](matrix-org#12330), [\matrix-org#12332](matrix-org#12332), [\matrix-org#12409](matrix-org#12409)) - Omit sending "offline" presence updates to application services after they are initially configured. ([\matrix-org#12193](matrix-org#12193)) - Switch to using a sequence to generate AS transaction IDs. Contributed by Nick @ Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. ([\matrix-org#12209](matrix-org#12209)) - Add missing type hints for storage. ([\matrix-org#12267](matrix-org#12267)) - Add missing type definitions for scripts in docker folder. Contributed by Jorge Florian. ([\matrix-org#12280](matrix-org#12280)) - Move [MSC2654](matrix-org/matrix-spec-proposals#2654) support behind an experimental configuration flag. ([\matrix-org#12295](matrix-org#12295)) - Update docstrings to explain how to decipher live and historic pagination tokens. ([\matrix-org#12317](matrix-org#12317)) - Add ground work for speeding up device list updates for users in large numbers of rooms. ([\matrix-org#12321](matrix-org#12321)) - Fix typechecker problems exposed by signedjson 1.1.2. ([\matrix-org#12326](matrix-org#12326)) - Remove the `tox` packaging job: it will be redundant once matrix-org#11537 lands. ([\matrix-org#12334](matrix-org#12334)) - Ignore `.envrc` for `direnv` users. ([\matrix-org#12335](matrix-org#12335)) - Remove the (broadly unused, dev-only) dockerfile for pg tests. ([\matrix-org#12336](matrix-org#12336)) - Remove redundant `get_success` calls in test code. ([\matrix-org#12346](matrix-org#12346)) - Add type annotations for `tests/unittest.py`. ([\matrix-org#12347](matrix-org#12347)) - Move single-use methods out of `TestCase`. ([\matrix-org#12348](matrix-org#12348)) - Remove broken and unused development scripts. ([\matrix-org#12349](matrix-org#12349), [\matrix-org#12351](matrix-org#12351), [\matrix-org#12355](matrix-org#12355)) - Convert `Linearizer` tests from `inlineCallbacks` to async. ([\matrix-org#12353](matrix-org#12353)) - Update docstrings for `ReadWriteLock` tests. ([\matrix-org#12354](matrix-org#12354)) - Refactor `Linearizer`, convert methods to async and use an async context manager. ([\matrix-org#12357](matrix-org#12357)) - Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. ([\matrix-org#12358](matrix-org#12358)) - Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. ([\matrix-org#12366](matrix-org#12366)) - Add type hints to tests files. ([\matrix-org#12371](matrix-org#12371)) - Allow specifying the Postgres database's port when running unit tests with Postgres. ([\matrix-org#12376](matrix-org#12376)) - Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. ([\matrix-org#12379](matrix-org#12379)) - Add opentracing spans to calls to external cache. ([\matrix-org#12380](matrix-org#12380)) - Lay groundwork for using `poetry` to manage Synapse's dependencies. ([\matrix-org#12381](matrix-org#12381), [\matrix-org#12407](matrix-org#12407), [\matrix-org#12412](matrix-org#12412), [\matrix-org#12418](matrix-org#12418)) - Make missing `importlib_metadata` dependency explicit. ([\matrix-org#12384](matrix-org#12384), [\matrix-org#12400](matrix-org#12400)) - Update type annotations for compatiblity with prometheus_client 0.14. ([\matrix-org#12389](matrix-org#12389)) - Remove support for the unstable identifiers specified in [MSC3288](matrix-org/matrix-spec-proposals#3288). ([\matrix-org#12398](matrix-org#12398)) - Add missing type hints to configuration classes. ([\matrix-org#12402](matrix-org#12402)) - Add files used to build the Docker image used for complement testing into the Synapse repository. ([\matrix-org#12404](matrix-org#12404)) - Do not include groups in the sync response when disabled. ([\matrix-org#12408](matrix-org#12408)) - Improve type hints related to HTTP query parameters. ([\matrix-org#12415](matrix-org#12415)) - Stop maintaining a list of lint targets. ([\matrix-org#12420](matrix-org#12420)) - Make `synapse._scripts` pass type checks. ([\matrix-org#12421](matrix-org#12421), [\matrix-org#12422](matrix-org#12422)) - Add some type hints to datastore. ([\matrix-org#12423](matrix-org#12423)) - Enable certificate checking during complement tests. ([\matrix-org#12435](matrix-org#12435)) - Explicitly specify the `tls` extra for Twisted dependency. ([\matrix-org#12444](matrix-org#12444))

Per implementation: matrix-org/synapse#12235

* Thread bundles exclude ignored users Per implementation: matrix-org/synapse#12235 * Clarify that the threads list API returns a 403 too Per implementation: matrix-org/synapse#13394 * Reference a token that actually exists

clokep force-pushed the clokep/ignored-users-relations branch 3 times, most recently from c9ff174 to a3597b9 Compare March 18, 2022 18:44

clokep added 8 commits March 21, 2022 10:38

Filter out events from ignored users in /relations.

75b1b5f

Add tests for ignored users in bundled aggregations.

3a51cfe

Filter out ignored users for aggregation groups.

5e73a5e

Rename a variable.

56dd70a

Filter out ignored users for threads.

3357181

Filter out ignored users for references.

9eef5cb

Add a note about edits.

156ef7a

Newsfragment

3bb8071

clokep force-pushed the clokep/ignored-users-relations branch from a3597b9 to 3bb8071 Compare March 21, 2022 14:52

clokep marked this pull request as ready for review March 21, 2022 18:26

clokep requested a review from a team as a code owner March 21, 2022 18:26

clokep commented Mar 21, 2022

View reviewed changes

erikjohnston self-assigned this Mar 24, 2022

erikjohnston reviewed Mar 24, 2022

View reviewed changes

erikjohnston removed their assignment Mar 24, 2022

clokep mentioned this pull request Mar 24, 2022

Do not consider events by ignored users for relations #12285

Merged

clokep changed the title ~~Do not consider events by ignored users for relations~~ Do not consider events by ignored users for bundled aggregations Mar 24, 2022

clokep added 2 commits March 24, 2022 10:30

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

89c89df

…rs-relations

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

fffa6ca

…rs-relations

clokep mentioned this pull request Mar 24, 2022

Remove the unused and unstable /aggregations endpoint. #12293

Merged

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

887fcb0

…rs-relations

clokep mentioned this pull request Mar 30, 2022

Remove an unnecessary class from the relations code. #12338

Merged

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

fc7d14b

…rs-relations

clokep added 8 commits March 31, 2022 08:08

Do not cache on the ignored users parameter when fetching relations.

ecae2ad

Newsfragment

f03a6a8

Do not cache on the ignored users parameter when fetching annotations.

622b621

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

b60fded

…rs-relations

Add an intermediate method for threads.

30ce317

Do not cache on the ignored users parameter when fetching threads.

b0d4474

Add missing docstrings.

1b2c9a1

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

44f6975

…rs-relations

clokep commented Apr 6, 2022

View reviewed changes

clokep requested a review from erikjohnston April 6, 2022 13:24

erikjohnston reviewed Apr 8, 2022

View reviewed changes

clokep added 5 commits April 8, 2022 08:19

Revert unused changes.

96d9215

Docstring.

e2910a4

Add logging.

7d6fa1e

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

61c7526

…rs-relations

Merge remote-tracking branch 'origin/develop' into clokep/ignored-use…

e95bd07

…rs-relations

clokep requested a review from erikjohnston April 11, 2022 13:36

erikjohnston approved these changes Apr 11, 2022

View reviewed changes

clokep merged commit 772bad2 into develop Apr 11, 2022

clokep deleted the clokep/ignored-users-relations branch April 11, 2022 14:09

turt2live added a commit to matrix-org/matrix-spec-proposals that referenced this pull request Sep 27, 2022

Thread bundles exclude ignored users

5a06d60

Per implementation: matrix-org/synapse#12235

turt2live mentioned this pull request Sep 27, 2022

Misc clarifications to threading MSCs matrix-org/matrix-spec-proposals#3899

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not consider events by ignored users for bundled aggregations #12235

Do not consider events by ignored users for bundled aggregations #12235

clokep commented Mar 16, 2022 •

edited

Loading

clokep Mar 21, 2022

erikjohnston Mar 24, 2022

clokep Mar 21, 2022

erikjohnston Mar 24, 2022

erikjohnston commented Mar 24, 2022

clokep commented Mar 24, 2022

erikjohnston commented Mar 24, 2022

erikjohnston left a comment

erikjohnston Mar 24, 2022

erikjohnston Mar 24, 2022

clokep Apr 6, 2022

erikjohnston Apr 8, 2022

clokep Apr 6, 2022

clokep commented Apr 6, 2022

erikjohnston left a comment

erikjohnston Apr 8, 2022

erikjohnston Apr 8, 2022

clokep Apr 8, 2022

clokep Apr 11, 2022

clokep commented Apr 11, 2022

erikjohnston left a comment

		# Note that there is no use in limiting edits by ignored users since the
		# parent event should be ignored in the first place if the user is ignored.

		@@ -521,6 +587,67 @@ def _get_thread_summaries_txn(

		return summaries

		async def get_threaded_messages_per_user(

Do not consider events by ignored users for bundled aggregations #12235

Do not consider events by ignored users for bundled aggregations #12235

Conversation

clokep commented Mar 16, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erikjohnston commented Mar 24, 2022

clokep commented Mar 24, 2022

erikjohnston commented Mar 24, 2022

erikjohnston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clokep commented Apr 6, 2022

erikjohnston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clokep commented Apr 11, 2022

erikjohnston left a comment

Choose a reason for hiding this comment

clokep commented Mar 16, 2022 •

edited

Loading