compute_state_delta pulls out the full state when doing gappy syncs

[erikjohnston]

This causes spikes in memory usage when handling /sync requests.

The callstack:

  File "/home/erikj/synapse/synapse/handlers/sync.py", line 1562, in handle_room_entries
    await self._generate_room_entry(
  File "/home/erikj/synapse/synapse/handlers/sync.py", line 2072, in _generate_room_entry
    state = await self.compute_state_delta(
  File "/home/erikj/synapse/synapse/handlers/sync.py", line 966, in compute_state_delta
    state_at_previous_sync = await self.get_state_at(
  File "/home/erikj/synapse/synapse/handlers/sync.py", line 680, in get_state_at
    state = await self.get_state_after_event(
  File "/home/erikj/synapse/synapse/handlers/sync.py", line 639, in get_state_after_event
    state_ids = await self._state_storage_controller.get_state_ids_for_event(
  File "/home/erikj/synapse/synapse/storage/controllers/state.py", line 303, in get_state_ids_for_event
    state_map = await self.get_state_ids_for_events(
  File "/home/erikj/synapse/synapse/storage/controllers/state.py", line 253, in get_state_ids_for_events
    group_to_state = await self.stores.state._get_state_for_groups(
  File "/home/erikj/synapse/synapse/storage/databases/state/store.py", line 288, in _get_state_for_groups
    group_to_state_dict = await self._get_state_groups_from_groups(
  File "/home/erikj/synapse/synapse/storage/databases/state/store.py", line 173, in _get_state_groups_from_groups

[squahtx]

It looks like we did this to handle a bug where membership changes in the gap get missed:

synapse/synapse/handlers/sync.py

Lines 948 to 951 in 845732b

	# for now, we disable LL for gappy syncs - see
	# https://github.com/vector-im/riot-web/issues/7211#issuecomment-419976346
	# N.B. this slows down incr syncs as we are now processing way
	# more state in the server than if we were LLing.

See the comment here: element-hq/element-web#7211 (comment)

I don't think there's an easy fix for this one.