Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Avoid blocking lazy-loading /syncs during partial joins #13477

Merged
merged 12 commits into from
Aug 18, 2022

Conversation

squahtx
Copy link
Contributor

@squahtx squahtx commented Aug 8, 2022

Use a state filter or accept partial state in a few places where we
request state, to avoid blocking.

To make lazy-loading /syncs work, we need to provide the memberships
of event senders, which are not guaranteed to be in the room state.
Instead we dig through auth events for memberships to present to
clients. The auth events of an event are guaranteed to contain a
passable membership event, otherwise the event would have been rejected.

Note that this only covers the common code paths encountered during
testing. There has been no exhaustive checking of all sync code paths.

Fixes #13146.

Signed-off-by: Sean Quah [email protected]


May or may not be easier to review commit by commit.

Complement tests:

@squahtx squahtx requested a review from a team as a code owner August 8, 2022 17:17
@squahtx squahtx marked this pull request as draft August 8, 2022 17:17
@squahtx squahtx removed the request for review from a team August 8, 2022 17:17
@squahtx squahtx force-pushed the squah/faster_room_joins_unblock_lazy_loading_sync branch from a9b61c9 to 4fd024b Compare August 10, 2022 12:03
@squahtx squahtx force-pushed the squah/faster_room_joins_unblock_lazy_loading_sync branch from 4fd024b to ad8a2b3 Compare August 10, 2022 12:09
Comment on lines 939 to 953
timeline_state = {}

members_to_fetch = {}
for event in batch.events:
# We need the event's sender, unless their membership was in a
# previous timeline event.
if (
EventTypes.Member,
event.sender,
) not in timeline_state and event.sender not in members_to_fetch:
members_to_fetch[event.sender] = event
# FIXME: we also care about invite targets etc.

if event.is_state():
timeline_state[(event.type, event.state_key)] = event.event_id
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change calculates members_to_fetch more accurately, by excluding event senders whose membership appears previously in the timeline.

Comment on lines 1105 to 1173
# If we only have partial state for the room, `state_ids` may be missing the
# memberships we wanted. We attempt to find some by digging through the auth
# events of timeline events.
if lazy_load_members:
assert members_to_fetch is not None

is_partial_state = await self.store.is_partial_state_room(room_id)
if is_partial_state:
additional_state_ids: MutableStateMap[str] = {}

# Tracks the missing members for logging purposes.
missing_members = {}

# Pick out the auth events of timeline events whose sender
# memberships are missing.
auth_event_ids: Set[str] = set()
for member, first_referencing_event in members_to_fetch.items():
if (
first_referencing_event is None
or (EventTypes.Member, member) in state_ids
):
continue

missing_members[member] = first_referencing_event
auth_event_ids.update(first_referencing_event.auth_event_ids())

auth_events = await self.store.get_events(auth_event_ids)

# Run through the events with missing sender memberships once more,
# picking out the memberships from the pile of auth events we have
# just fetched.
for member, first_referencing_event in members_to_fetch.items():
if (
first_referencing_event is None
or (EventTypes.Member, member) in state_ids
):
continue

# Dig through the auth events to find the sender's membership.
for auth_event_id in first_referencing_event.auth_event_ids():
# We only store events once we have all their auth events,
# so the auth event must be in the pile we have just
# fetched.
auth_event = auth_events[auth_event_id]

if (
auth_event.type == EventTypes.Member
and auth_event.state_key == event.sender
):
missing_members.pop(member)
additional_state_ids[
(EventTypes.Member, event.sender)
] = auth_event.event_id
break

# Now merge in the state we have scrounged up.
state_ids = {**state_ids, **additional_state_ids}

if missing_members:
# There really shouldn't be any missing memberships now.
# For an event to appear in the timeline, we must have its auth
# events, which must include its sender's membership.
logger.error(
"Failed to find memberships for %s in partial state room "
"%s in the auth events of %s.",
list(missing_members.keys()),
room_id,
list(missing_members.values()),
)
Copy link
Contributor Author

@squahtx squahtx Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since all three branches (initial, gappy and incremental sync) above have been modified to return partial state, the property below (

# At this point, if `lazy_load_members` is enabled, `state_ids` includes
# the memberships of all event senders in the timeline

) no longer holds. We try to find membership events from auth events to restore the property.

Comment on lines 1163 to 1173
if missing_members:
# There really shouldn't be any missing memberships now.
# For an event to appear in the timeline, we must have its auth
# events, which must include its sender's membership.
logger.error(
"Failed to find memberships for %s in partial state room "
"%s in the auth events of %s.",
list(missing_members.keys()),
room_id,
list(missing_members.values()),
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this error to be accurate, we need to change the calculation of members_to_fetch to be more accurate (done above).

Comment on lines +1867 to 1872
old_state_ids = await self.get_state_at(
room_id,
since_token,
state_filter=StateFilter.from_types([(EventTypes.Member, user_id)]),
)
old_mem_ev_id = old_state_ids.get((EventTypes.Member, user_id), None)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only ever pull this one local membership out of the state, so the state filter is okay to add. The same applies to the next instance below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like a valuable optimisation irrespective of the faster-joins case

@squahtx squahtx changed the title Avoid blocking lazy-loading syncs during partial joins Avoid blocking lazy-loading /syncs during partial joins Aug 10, 2022
@squahtx squahtx marked this pull request as ready for review August 10, 2022 12:43
@squahtx squahtx requested a review from a team August 10, 2022 12:49
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm in general, a few suggestions

synapse/storage/controllers/state.py Outdated Show resolved Hide resolved
Comment on lines +1867 to 1872
old_state_ids = await self.get_state_at(
room_id,
since_token,
state_filter=StateFilter.from_types([(EventTypes.Member, user_id)]),
)
old_mem_ev_id = old_state_ids.get((EventTypes.Member, user_id), None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like a valuable optimisation irrespective of the faster-joins case

synapse/handlers/sync.py Outdated Show resolved Hide resolved
synapse/handlers/sync.py Outdated Show resolved Hide resolved
Comment on lines 1111 to 1173
is_partial_state = await self.store.is_partial_state_room(room_id)
if is_partial_state:
additional_state_ids: MutableStateMap[str] = {}

# Tracks the missing members for logging purposes.
missing_members = {}

# Pick out the auth events of timeline events whose sender
# memberships are missing.
auth_event_ids: Set[str] = set()
for member, first_referencing_event in members_to_fetch.items():
if (
first_referencing_event is None
or (EventTypes.Member, member) in state_ids
):
continue

missing_members[member] = first_referencing_event
auth_event_ids.update(first_referencing_event.auth_event_ids())

auth_events = await self.store.get_events(auth_event_ids)

# Run through the events with missing sender memberships once more,
# picking out the memberships from the pile of auth events we have
# just fetched.
for member, first_referencing_event in members_to_fetch.items():
if (
first_referencing_event is None
or (EventTypes.Member, member) in state_ids
):
continue

# Dig through the auth events to find the sender's membership.
for auth_event_id in first_referencing_event.auth_event_ids():
# We only store events once we have all their auth events,
# so the auth event must be in the pile we have just
# fetched.
auth_event = auth_events[auth_event_id]

if (
auth_event.type == EventTypes.Member
and auth_event.state_key == event.sender
):
missing_members.pop(member)
additional_state_ids[
(EventTypes.Member, event.sender)
] = auth_event.event_id
break

# Now merge in the state we have scrounged up.
state_ids = {**state_ids, **additional_state_ids}

if missing_members:
# There really shouldn't be any missing memberships now.
# For an event to appear in the timeline, we must have its auth
# events, which must include its sender's membership.
logger.error(
"Failed to find memberships for %s in partial state room "
"%s in the auth events of %s.",
list(missing_members.keys()),
room_id,
list(missing_members.values()),
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move all this out to a separate function which returns additional_state_ids?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave that a go in the latest commit.

@squahtx squahtx requested a review from richvdh August 15, 2022 18:57
Comment on lines 920 to 926
# The memberships needed for events in the timeline.
# A dictionary with user IDs as keys and the first event in the timeline
# requiring each member as values.
# The syncing user's own membership is always implicitly required for
# `full_state` syncs and may not be present in the dictionary.
# Only calculated when `lazy_load_members` is on.
members_to_fetch = None
members_to_fetch: Optional[Dict[str, EventBase]] = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given you've now repurposed this a bit, I think it needs a different name.

Maybe it should be called first_event_by_sender_map or something?

We should also remove the comment that says it is "the memberships needed for events in the timeline" - that is no longer entirely correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first_event_by_sender_map isn't quite right either, since it excludes senders whose memberships appear in the timeline.

I've instead split members_to_fetch into two: a set of members (still excluding senders whose memberships appear in the timeline) and a mapping of user ID -> first event sent, for all senders. Let me know what you think.

Comment on lines 915 to 970
members_to_fetch.add(sync_config.user.to_string())
state_filter = StateFilter.from_lazy_load_member_list(
itertools.chain(
members_to_fetch.keys(), [sync_config.user.to_string()]
)
)
else:
state_filter = StateFilter.from_lazy_load_member_list(
members_to_fetch
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's incosistent use of .keys() between the two arms here: put it in both or neither. Preferably both for explicitness?

I'm also wondering if this would be more clearly written as:

if full_state:
    # comments
    members_to_fetch = itertools.chain(first_event_by_sender_map.keys(), (sync_config.user.to_string(),))
else:
    members_to_fetch = first_event_by_sender_map.keys()

state_filter = StateFilter.from_lazy_load_member_list(members_to_fetch)

ymmv though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've ended up reverting back to the original code now that members_to_fetch has been split into two.

@squahtx squahtx requested a review from richvdh August 16, 2022 17:25
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup that seems much clearer. THanks!

@squahtx squahtx merged commit 84169a8 into develop Aug 18, 2022
@squahtx squahtx deleted the squah/faster_room_joins_unblock_lazy_loading_sync branch August 18, 2022 10:53
squahtx added a commit to matrix-org/complement that referenced this pull request Aug 18, 2022
matrix-org/synapse#13477 unblocked lazy-loading
`/sync`s while a room has partial state, which allows us to wait for
received events using `/sync`.

"Resync completes even when events arrive before their prev_events" is
now the only test that waits for events using `/event`, since the
outliers do not appear in the `/sync` timeline.
DMRobertson pushed a commit that referenced this pull request Aug 23, 2022
Synapse 1.66.0rc1 (2022-08-23)
==============================

This release removes the ability for homeservers to delegate email ownership
verification and password reset confirmation to identity servers. This removal
was originally planned for Synapse 1.64, but was later deferred until now.

See the [upgrade notes](https://matrix-org.github.io/synapse/v1.66/upgrade.html#upgrading-to-v1660) for more details.

Features
--------

- Improve validation of request bodies for the following client-server API endpoints: [`/account/password`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3accountpassword), [`/account/password/email/requestToken`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3accountpasswordemailrequesttoken), [`/account/deactivate`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3accountdeactivate) and [`/account/3pid/email/requestToken`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3account3pidemailrequesttoken). ([\#13188](#13188), [\#13563](#13563))
- Add forgotten status to [Room Details Admin API](https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#room-details-api). ([\#13503](#13503))
- Add an experimental implementation for [MSC3852 (Expose user agents on `Device`)](matrix-org/matrix-spec-proposals#3852). ([\#13549](#13549))
- Add `org.matrix.msc2716v4` experimental room version with updated content fields. Part of [MSC2716 (Importing history)](matrix-org/matrix-spec-proposals#2716).  ([\#13551](#13551))
- Add support for compression to federation responses. ([\#13537](#13537))
- Improve performance of sending messages in rooms with thousands of local users. ([\#13522](#13522), [\#13547](#13547))

Bugfixes
--------

- Faster room joins: make `/joined_members` block whilst the room is partial stated. ([\#13514](#13514))
- Fix a bug introduced in Synapse 1.21.0 where the [`/event_reports` Admin API](https://matrix-org.github.io/synapse/develop/admin_api/event_reports.html) could return a total count which was larger than the number of results you can actually query for. ([\#13525](#13525))
- Fix a bug introduced in Synapse 1.52.0 where sending server notices fails if `max_avatar_size` or `allowed_avatar_mimetypes` is set and not `system_mxid_avatar_url`. ([\#13566](#13566))
- Fix a bug where the `opentracing.force_tracing_for_users` config option would not apply to [`/sendToDevice`](https://spec.matrix.org/v1.3/client-server-api/#put_matrixclientv3sendtodeviceeventtypetxnid) and [`/keys/upload`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3keysupload) requests. ([\#13574](#13574))

Improved Documentation
----------------------

- Add `openssl` example for generating registration HMAC digest. ([\#13472](#13472))
- Tidy up Synapse's README. ([\#13491](#13491))
- Document that event purging related to the `redaction_retention_period` config option is executed only every 5 minutes. ([\#13492](#13492))
- Add a warning to retention documentation regarding the possibility of database corruption. ([\#13497](#13497))
- Document that the `DOCKER_BUILDKIT=1` flag is needed to build the docker image. ([\#13515](#13515))
- Add missing links in `user_consent` section of configuration manual. ([\#13536](#13536))
- Fix the doc and some warnings that were referring to the nonexistent `custom_templates_directory` setting (instead of `custom_template_directory`). ([\#13538](#13538))

Deprecations and Removals
-------------------------

- Remove the ability for homeservers to delegate email ownership verification
  and password reset confirmation to identity servers. See [upgrade notes](https://matrix-org.github.io/synapse/v1.66/upgrade.html#upgrading-to-v1660) for more details.

Internal Changes
----------------

- Update the rejected state of events during de-partial-stating. ([\#13459](#13459))
- Avoid blocking lazy-loading `/sync`s during partial joins due to remote memberships. Pull remote memberships from auth events instead of the room state. ([\#13477](#13477))
- Refuse to start when faster joins is enabled on a deployment with workers, since worker configurations are not currently supported. ([\#13531](#13531))

- Allow use of both `@trace` and `@tag_args` stacked on the same function. ([\#13453](#13453))
- Instrument the federation/backfill part of `/messages` for understandable traces in Jaeger. ([\#13489](#13489))
- Instrument `FederationStateIdsServlet` (`/state_ids`) for understandable traces in Jaeger. ([\#13499](#13499), [\#13554](#13554))
- Track HTTP response times over 10 seconds from `/messages` (`synapse_room_message_list_rest_servlet_response_time_seconds`). ([\#13533](#13533))
- Add metrics to track how the rate limiter is affecting requests (sleep/reject). ([\#13534](#13534), [\#13541](#13541))
- Add metrics to time how long it takes us to do backfill processing (`synapse_federation_backfill_processing_before_time_seconds`, `synapse_federation_backfill_processing_after_time_seconds`). ([\#13535](#13535), [\#13584](#13584))
- Add metrics to track rate limiter queue timing (`synapse_rate_limit_queue_wait_time_seconds`). ([\#13544](#13544))
- Update metrics to track `/messages` response time by room size. ([\#13545](#13545))

- Refactor methods in `synapse.api.auth.Auth` to use `Requester` objects everywhere instead of user IDs. ([\#13024](#13024))
- Clean-up tests for notifications. ([\#13471](#13471))
- Add some miscellaneous comments to document sync, especially around `compute_state_delta`. ([\#13474](#13474))
- Use literals in place of `HTTPStatus` constants in tests. ([\#13479](#13479), [\#13488](#13488))
- Add comments about how event push actions are rotated. ([\#13485](#13485))
- Modify HTML template content to better support mobile devices' screen sizes. ([\#13493](#13493))
- Add a linter script which will reject non-strict types in Pydantic models. ([\#13502](#13502))
- Reduce the number of tests using legacy TCP replication. ([\#13543](#13543))
- Allow specifying additional request fields when using the `HomeServerTestCase.login` helper method. ([\#13549](#13549))
- Make `HomeServerTestCase` load any configured homeserver modules automatically. ([\#13558](#13558))
Fizzadar added a commit to beeper/synapse-legacy-fork that referenced this pull request Sep 1, 2022
Synapse 1.66.0 (2022-08-31)
===========================

No significant changes since 1.66.0rc2.

This release removes the ability for homeservers to delegate email ownership
verification and password reset confirmation to identity servers. This removal
was originally planned for Synapse 1.64, but was later deferred until now. See
the [upgrade notes](https://matrix-org.github.io/synapse/v1.66/upgrade.html#upgrading-to-v1660) for more details.

Deployments with multiple workers should note that the direct TCP replication
configuration was deprecated in Synapse v1.18.0 and will be removed in Synapse
v1.67.0. In particular, the TCP `replication` [listener](https://matrix-org.github.io/synapse/v1.66/usage/configuration/config_documentation.html#listeners)
type (not to be confused with the `replication` resource on the `http` listener
type) and the `worker_replication_port` config option will be removed .

To migrate to Redis, add the [`redis` config](https://matrix-org.github.io/synapse/v1.66/workers.html#shared-configuration),
then remove the TCP `replication` listener from config of the master and
`worker_replication_port` from worker config. Note that a HTTP listener with a
`replication` resource is still required. See the
[worker documentation](https://matrix-org.github.io/synapse/v1.66/workers.html)
for more details.

Synapse 1.66.0rc2 (2022-08-30)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.66.0rc1 where the new rate limit metrics were misreported (`synapse_rate_limit_sleep_affected_hosts`, `synapse_rate_limit_reject_affected_hosts`). ([\matrix-org#13649](matrix-org#13649))

Synapse 1.66.0rc1 (2022-08-23)
==============================

Features
--------

- Improve validation of request bodies for the following client-server API endpoints: [`/account/password`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3accountpassword), [`/account/password/email/requestToken`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3accountpasswordemailrequesttoken), [`/account/deactivate`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3accountdeactivate) and [`/account/3pid/email/requestToken`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3account3pidemailrequesttoken). ([\matrix-org#13188](matrix-org#13188), [\matrix-org#13563](matrix-org#13563))
- Add forgotten status to [Room Details Admin API](https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#room-details-api). ([\matrix-org#13503](matrix-org#13503))
- Add an experimental implementation for [MSC3852 (Expose user agents on `Device`)](matrix-org/matrix-spec-proposals#3852). ([\matrix-org#13549](matrix-org#13549))
- Add `org.matrix.msc2716v4` experimental room version with updated content fields. Part of [MSC2716 (Importing history)](matrix-org/matrix-spec-proposals#2716).  ([\matrix-org#13551](matrix-org#13551))
- Add support for compression to federation responses. ([\matrix-org#13537](matrix-org#13537))
- Improve performance of sending messages in rooms with thousands of local users. ([\matrix-org#13522](matrix-org#13522), [\matrix-org#13547](matrix-org#13547))

Bugfixes
--------

- Faster room joins: make `/joined_members` block whilst the room is partial stated. ([\matrix-org#13514](matrix-org#13514))
- Fix a bug introduced in Synapse 1.21.0 where the [`/event_reports` Admin API](https://matrix-org.github.io/synapse/develop/admin_api/event_reports.html) could return a total count which was larger than the number of results you can actually query for. ([\matrix-org#13525](matrix-org#13525))
- Fix a bug introduced in Synapse 1.52.0 where sending server notices fails if `max_avatar_size` or `allowed_avatar_mimetypes` is set and not `system_mxid_avatar_url`. ([\matrix-org#13566](matrix-org#13566))
- Fix a bug where the `opentracing.force_tracing_for_users` config option would not apply to [`/sendToDevice`](https://spec.matrix.org/v1.3/client-server-api/#put_matrixclientv3sendtodeviceeventtypetxnid) and [`/keys/upload`](https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3keysupload) requests. ([\matrix-org#13574](matrix-org#13574))

Improved Documentation
----------------------

- Add `openssl` example for generating registration HMAC digest. ([\matrix-org#13472](matrix-org#13472))
- Tidy up Synapse's README. ([\matrix-org#13491](matrix-org#13491))
- Document that event purging related to the `redaction_retention_period` config option is executed only every 5 minutes. ([\matrix-org#13492](matrix-org#13492))
- Add a warning to retention documentation regarding the possibility of database corruption. ([\matrix-org#13497](matrix-org#13497))
- Document that the `DOCKER_BUILDKIT=1` flag is needed to build the docker image. ([\matrix-org#13515](matrix-org#13515))
- Add missing links in `user_consent` section of configuration manual. ([\matrix-org#13536](matrix-org#13536))
- Fix the doc and some warnings that were referring to the nonexistent `custom_templates_directory` setting (instead of `custom_template_directory`). ([\matrix-org#13538](matrix-org#13538))

Deprecations and Removals
-------------------------

- Remove the ability for homeservers to delegate email ownership verification
  and password reset confirmation to identity servers. See [upgrade notes](https://matrix-org.github.io/synapse/v1.66/upgrade.html#upgrading-to-v1660) for more details.

Internal Changes
----------------

- Update the rejected state of events during de-partial-stating. ([\matrix-org#13459](matrix-org#13459))
- Avoid blocking lazy-loading `/sync`s during partial joins due to remote memberships. Pull remote memberships from auth events instead of the room state. ([\matrix-org#13477](matrix-org#13477))
- Refuse to start when faster joins is enabled on a deployment with workers, since worker configurations are not currently supported. ([\matrix-org#13531](matrix-org#13531))

- Allow use of both `@trace` and `@tag_args` stacked on the same function. ([\matrix-org#13453](matrix-org#13453))
- Instrument the federation/backfill part of `/messages` for understandable traces in Jaeger. ([\matrix-org#13489](matrix-org#13489))
- Instrument `FederationStateIdsServlet` (`/state_ids`) for understandable traces in Jaeger. ([\matrix-org#13499](matrix-org#13499), [\matrix-org#13554](matrix-org#13554))
- Track HTTP response times over 10 seconds from `/messages` (`synapse_room_message_list_rest_servlet_response_time_seconds`). ([\matrix-org#13533](matrix-org#13533))
- Add metrics to track how the rate limiter is affecting requests (sleep/reject). ([\matrix-org#13534](matrix-org#13534), [\matrix-org#13541](matrix-org#13541))
- Add metrics to time how long it takes us to do backfill processing (`synapse_federation_backfill_processing_before_time_seconds`, `synapse_federation_backfill_processing_after_time_seconds`). ([\matrix-org#13535](matrix-org#13535), [\matrix-org#13584](matrix-org#13584))
- Add metrics to track rate limiter queue timing (`synapse_rate_limit_queue_wait_time_seconds`). ([\matrix-org#13544](matrix-org#13544))
- Update metrics to track `/messages` response time by room size. ([\matrix-org#13545](matrix-org#13545))

- Refactor methods in `synapse.api.auth.Auth` to use `Requester` objects everywhere instead of user IDs. ([\matrix-org#13024](matrix-org#13024))
- Clean-up tests for notifications. ([\matrix-org#13471](matrix-org#13471))
- Add some miscellaneous comments to document sync, especially around `compute_state_delta`. ([\matrix-org#13474](matrix-org#13474))
- Use literals in place of `HTTPStatus` constants in tests. ([\matrix-org#13479](matrix-org#13479), [\matrix-org#13488](matrix-org#13488))
- Add comments about how event push actions are rotated. ([\matrix-org#13485](matrix-org#13485))
- Modify HTML template content to better support mobile devices' screen sizes. ([\matrix-org#13493](matrix-org#13493))
- Add a linter script which will reject non-strict types in Pydantic models. ([\matrix-org#13502](matrix-org#13502))
- Reduce the number of tests using legacy TCP replication. ([\matrix-org#13543](matrix-org#13543))
- Allow specifying additional request fields when using the `HomeServerTestCase.login` helper method. ([\matrix-org#13549](matrix-org#13549))
- Make `HomeServerTestCase` load any configured homeserver modules automatically. ([\matrix-org#13558](matrix-org#13558))

# -----BEGIN PGP SIGNATURE-----
#
# iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmMPT8QACgkQ+84Kzgcy
# GG9CUAv+Pv/iDpE2jKlV7zQ/cagaKCGsFK5jy0+K9Wr215nP89tuhU37bJXsgvVu
# GP3A8k1c/ENPhXwYHLCnnxV3jick1FuVE0W6h0j2PMYeIGNCQhDswytnsQO4JExg
# fGLL4ygCzpe8bFX9+mhIM4z8xkZjZX3lIa8CN2LtRLIo0m7qoT1ZWqdt7kAjj5yL
# XMk+3Y1yq/Y4SHHqgKurBNdwNcwnv7ynchWxTYa12WVTINt26dLV0Syk3p8u2SLl
# 5YNzcDs2TAM7+VxAu7E0AQl426+Ufi122Oj1ZBUG2FxTPLH8Xr18cN2M/at6WxoX
# 8pOkGiuahKKvahw1iCoHAGIC66gFIPxBE9xW4R2SKrQtG4sDuKJI0kvunRV8+cy5
# TuJ9cmdDmJR2vj3P3OULqLXGkWsGNJqfZZF8OWkHEI8LUIXZLrAZocFtlonkr9rV
# Y8r8LxL8Id1rbHAnCXcJnYdaJ6ol0RIObDFpitY/D8BDUONVw/byeOyAEkq/XPrZ
# Ke/9K8sy
# =eg1L
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed Aug 31 13:10:44 2022 BST
# gpg:                using RSA key 58C4E75BC67C92169A7FDD11FBCE0ACE0732186F
# gpg: Can't check signature: No public key

# Conflicts:
#	synapse/api/auth.py
#	synapse/push/baserules.py
#	synapse/push/bulk_push_rule_evaluator.py
#	synapse/push/push_rule_evaluator.py
#	synapse/storage/databases/main/event_push_actions.py
#	tests/server_notices/test_resource_limits_server_notices.py
squahtx added a commit that referenced this pull request Sep 27, 2022
During a `lazy_load_members` `/sync`, we look through auth events in
rooms with partial state to find prior membership events. When such a
membership is not found, an error is logged.

Since the first join event for a user never has a prior membership event
to cite, the error would always be logged when one appeared in the room
timeline.

Avoid logging errors for such events.

Introduced in #13477.

Signed-off-by: Sean Quah <[email protected]>
squahtx pushed a commit that referenced this pull request Mar 8, 2023
When pushing events in partial state rooms down incremental /sync, we
try to find the `m.room.member` state event for their senders by digging
through their auth events, so that we can present the membership to the
client. Events usually have a membership event in their auth events,
with the exception of the `m.room.create` event and a user's first join
into the room.

When implementing #13477, we took the case of a user's first join into
account, but forgot to handle the `m.room.create` case. This change
fixes that.

Signed-off-by: Sean Quah <[email protected]>
squahtx added a commit that referenced this pull request Mar 9, 2023
When pushing events in partial state rooms down incremental /sync, we
try to find the `m.room.member` state event for their senders by digging
through their auth events, so that we can present the membership to the
client. Events usually have a membership event in their auth events,
with the exception of the `m.room.create` event and a user's first join
into the room.

When implementing #13477, we took the case of a user's first join into
account, but forgot to handle the `m.room.create` case. This change
fixes that.

Signed-off-by: Sean Quah <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

lazy-load-members /sync blocks if an event arrives in a partial-state-room
2 participants