-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Room Complexity Client Implementation #5783
Conversation
commit c2ca163 Author: Amber H. Brown <[email protected]> Date: Fri Jul 26 22:48:20 2019 +1000 don't block commit 8d9a56e Merge: b50d8a9 4a5fb54 Author: Amber H. Brown <[email protected]> Date: Fri Jul 26 21:35:42 2019 +1000 Merge branch 'shhs' of ssh://github.com/matrix-org/synapse into shhs commit 4a5fb54 Merge: 95a0386 992333b Author: Amber Brown <[email protected]> Date: Fri Jul 26 20:59:41 2019 +1000 Merge tag 'v1.2.1' into shhs Synapse 1.2.1 (2019-07-26) ========================== Security update --------------- This release includes *four* security fixes: - Prevent an attack where a federated server could send redactions for arbitrary events in v1 and v2 rooms. ([\#5767](#5767)) - Prevent a denial-of-service attack where cycles of redaction events would make Synapse spin infinitely. Thanks to `@lrizika:matrix.org` for identifying and responsibly disclosing this issue. ([0f2ecb9](0f2ecb961)) - Prevent an attack where users could be joined or parted from public rooms without their consent. Thanks to @dylanger for identifying and responsibly disclosing this issue. ([\#5744](#5744)) - Fix a vulnerability where a federated server could spoof read-receipts from users on other servers. Thanks to @dylanger for identifying this issue too. ([\#5743](#5743)) Additionally, the following fix was in Synapse **1.2.0**, but was not correctly identified during the original release: - It was possible for a room moderator to send a redaction for an `m.room.create` event, which would downgrade the room to version 1. Thanks to `/dev/ponies` for identifying and responsibly disclosing this issue! ([\#5701](#5701)) commit 95a0386 Author: Amber Brown <[email protected]> Date: Fri Jul 26 20:27:31 2019 +1000 don't have a circleci config commit b50d8a9 Author: Amber H. Brown <[email protected]> Date: Fri Jul 26 02:26:23 2019 +1000 fix merging forward commit 3edf6e9 Author: Amber H. Brown <[email protected]> Date: Fri Jul 26 02:07:05 2019 +1000 fix this commit f61cdc1 Merge: 43cf234 c0a1301 Author: Amber H. Brown <[email protected]> Date: Fri Jul 26 01:48:50 2019 +1000 Merge tag 'v1.2.0' into shhs No changes since v1.2.0rc2. commit 43cf234 Author: Amber H. Brown <[email protected]> Date: Fri Jul 26 01:48:20 2019 +1000 dockerfile update commit b7962f5 Author: Amber H. Brown <[email protected]> Date: Thu Jul 18 23:23:12 2019 +1000 add a wait commit 9bbf2d2 Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:46:20 2019 +1000 fix commit 5daee2e Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:41:00 2019 +1000 fix commit 14c8b03 Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:36:27 2019 +1000 fix commit 7fcd6c1 Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:32:50 2019 +1000 fix commit c43c1ad Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:28:11 2019 +1000 fix commit a025abe Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:02:15 2019 +1000 try now commit c1777f5 Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 04:00:34 2019 +1000 try now commit 646292c Author: Amber H. Brown <[email protected]> Date: Wed Jul 17 03:58:34 2019 +1000 see if we can do a build! commit a175e60 Merge: 9b3a63e 0e54342 Author: Amber H. Brown <[email protected]> Date: Fri Jul 5 23:49:13 2019 +1000 Merge remote-tracking branch 'origin/develop' into shhs commit 9b3a63e Author: Amber H. Brown <[email protected]> Date: Fri Jul 5 23:36:41 2019 +1000 linting commit 3d89feb Author: Amber H. Brown <[email protected]> Date: Fri Jul 5 23:34:24 2019 +1000 linting commit 400bc06 Author: Amber H. Brown <[email protected]> Date: Fri Jul 5 22:44:22 2019 +1000 linting commit a1de642 Merge: f4343c7 54283f3 Author: Amber H. Brown <[email protected]> Date: Fri Jul 5 19:46:11 2019 +1000 Merge tag 'v1.1.0' into shhs Synapse 1.1.0 (2019-07-04) ========================== As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4. See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details. This release also deprecates the use of environment variables to configure the docker image. See the [docker README](https://github.com/matrix-org/synapse/blob/release-v1.1.0/docker/README.md#legacy-dynamic-configuration-file-support) for more details. No changes since 1.1.0rc2. Synapse 1.1.0rc2 (2019-07-03) ============================= Bugfixes -------- - Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. ([\#5593](#5593)) - Removed the `SYNAPSE_SMTP_*` docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. ([\#5596](#5596)) - Fix a number of "Starting txn from sentinel context" warnings. ([\#5605](#5605)) Internal Changes ---------------- - Update github templates. ([\#5552](#5552)) Synapse 1.1.0rc1 (2019-07-02) ============================= As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4. See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details. Features -------- - Added possibilty to disable local password authentication. Contributed by Daniel Hoffend. ([\#5092](#5092)) - Add monthly active users to phonehome stats. ([\#5252](#5252)) - Allow expired user to trigger renewal email sending manually. ([\#5363](#5363)) - Statistics on forward extremities per room are now exposed via Prometheus. ([\#5384](#5384), [\#5458](#5458), [\#5461](#5461)) - Add --no-daemonize option to run synapse in the foreground, per issue #4130. Contributed by Soham Gumaste. ([\#5412](#5412), [\#5587](#5587)) - Fully support SAML2 authentication. Contributed by [Alexander Trost](https://github.com/galexrt) - thank you! ([\#5422](#5422)) - Allow server admins to define implementations of extra rules for allowing or denying incoming events. ([\#5440](#5440), [\#5474](#5474), [\#5477](#5477)) - Add support for handling pagination APIs on client reader worker. ([\#5505](#5505), [\#5513](#5513), [\#5531](#5531)) - Improve help and cmdline option names for --generate-config options. ([\#5512](#5512)) - Allow configuration of the path used for ACME account keys. ([\#5516](#5516), [\#5521](#5521), [\#5522](#5522)) - Add --data-dir and --open-private-ports options. ([\#5524](#5524)) - Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. ([\#5534](#5534)) - The minimum TLS version used for outgoing federation requests can now be set with `federation_client_minimum_tls_version`. ([\#5550](#5550)) - Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. ([\#5559](#5559)) - Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. ([\#5476](#5476)) Bugfixes -------- - Fix bug processing incoming events over federation if call to `/get_missing_events` fails. ([\#5042](#5042)) - Prevent more than one room upgrade happening simultaneously on the same room. ([\#5051](#5051)) - Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. ([\#5325](#5325)) - Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. ([\#5387](#5387)) - Fix email notifications for unnamed rooms with multiple people. ([\#5388](#5388)) - Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. ([\#5389](#5389)) - Fix handling of failures fetching remote content to not log failures as exceptions. ([\#5390](#5390)) - Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. ([\#5394](#5394)) - Fix missing invite state after exchanging 3PID invites over federaton. ([\#5464](#5464)) - Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. ([\#5498](#5498)) - Fix logging error when a tampered event is detected. ([\#5500](#5500)) - Fix bug where clients could tight loop calling `/sync` for a period. ([\#5507](#5507)) - Fix bug with `jinja2` preventing Synapse from starting. Users who had this problem should now simply need to run `pip install matrix-synapse`. ([\#5514](#5514)) - Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. ([\#5523](#5523)) - Fixed m.login.jwt using unregistred user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. ([\#5555](#5555), [\#5586](#5586)) - Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. ([\#5576](#5576)) Updates to the Docker image --------------------------- - Add ability to change Docker containers [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) with the `TZ` variable. ([\#5383](#5383)) - Update docker image to use Python 3.7. ([\#5546](#5546)) - Deprecate the use of environment variables for configuration, and make the use of a static configuration the default. ([\#5561](#5561), [\#5562](#5562), [\#5566](#5566), [\#5567](#5567)) - Increase default log level for docker image to INFO. It can still be changed by editing the generated log.config file. ([\#5547](#5547)) - Send synapse logs to the docker logging system, by default. ([\#5565](#5565)) - Open the non-TLS port by default. ([\#5568](#5568)) - Fix failure to start under docker with SAML support enabled. ([\#5490](#5490)) - Use a sensible location for data files when generating a config file. ([\#5563](#5563)) Deprecations and Removals ------------------------- - Python 2.7 is no longer a supported platform. Synapse now requires Python 3.5+ to run. ([\#5425](#5425)) - PostgreSQL 9.4 is no longer supported. Synapse requires Postgres 9.5+ or above for Postgres support. ([\#5448](#5448)) - Remove support for cpu_affinity setting. ([\#5525](#5525)) Improved Documentation ---------------------- - Improve README section on performance troubleshooting. ([\#4276](#4276)) - Add information about how to install and run `black` on the codebase to code_style.rst. ([\#5537](#5537)) - Improve install docs on choosing server_name. ([\#5558](#5558)) Internal Changes ---------------- - Add logging to 3pid invite signature verification. ([\#5015](#5015)) - Update example haproxy config to a more compatible setup. ([\#5313](#5313)) - Track deactivated accounts in the database. ([\#5378](#5378), [\#5465](#5465), [\#5493](#5493)) - Clean up code for sending federation EDUs. ([\#5381](#5381)) - Add a sponsor button to the repo. ([\#5382](#5382), [\#5386](#5386)) - Don't log non-200 responses from federation queries as exceptions. ([\#5383](#5383)) - Update Python syntax in contrib/ to Python 3. ([\#5446](#5446)) - Update federation_client dev script to support `.well-known` and work with python3. ([\#5447](#5447)) - SyTest has been moved to Buildkite. ([\#5459](#5459)) - Demo script now uses python3. ([\#5460](#5460)) - Synapse can now handle RestServlets that return coroutines. ([\#5475](#5475), [\#5585](#5585)) - The demo servers talk to each other again. ([\#5478](#5478)) - Add an EXPERIMENTAL config option to try and periodically clean up extremities by sending dummy events. ([\#5480](#5480)) - Synapse's codebase is now formatted by `black`. ([\#5482](#5482)) - Some cleanups and sanity-checking in the CPU and database metrics. ([\#5499](#5499)) - Improve email notification logging. ([\#5502](#5502)) - Fix "Unexpected entry in 'full_schemas'" log warning. ([\#5509](#5509)) - Improve logging when generating config files. ([\#5510](#5510)) - Refactor and clean up Config parser for maintainability. ([\#5511](#5511)) - Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the `synapse/` folder within it. ([\#5543](#5543)) - Update v1.0.0 release changelog to include more information about changes to password resets. ([\#5545](#5545)) - Remove non-functioning check_event_hash.py dev script. ([\#5548](#5548)) - Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. ([\#5550](#5550)) - Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. ([\#5557](#5557)) - Reduce the amount of stuff we send in the docker context. ([\#5564](#5564)) - Point the reverse links in the Purge History contrib scripts at the intended location. ([\#5570](#5570)) commit f4343c7 Merge: 4689408 463d5a8 Author: Amber H. Brown <[email protected]> Date: Wed Jul 3 22:39:30 2019 +1000 Merge remote-tracking branch 'origin/develop' into shhs commit 4689408 Merge: bed45ab b491468 Author: Amber H. Brown <[email protected]> Date: Tue Jul 2 18:31:29 2019 +1000 Merge remote-tracking branch 'origin/develop' into shhs commit bed45ab Author: Amber H. Brown <[email protected]> Date: Tue Jul 2 18:18:09 2019 +1000 release shhs on tags commit 0993b05 Author: Amber H. Brown <[email protected]> Date: Mon Jul 1 23:13:21 2019 +1000 improve error text when room is too large commit e001115 Author: Amber H. Brown <[email protected]> Date: Tue Jun 18 21:24:56 2019 +1000 fix commit e60aab1 Merge: e7c1171 82d9d52 Author: Amber H. Brown <[email protected]> Date: Tue Jun 18 21:20:13 2019 +1000 Merge remote-tracking branch 'origin/develop' into shhs commit e7c1171 Merge: 8fe26db c831748 Author: Amber Brown <[email protected]> Date: Tue Jun 4 20:41:59 2019 +1000 Merge remote-tracking branch 'origin/master' into shhs commit 8fe26db Merge: c99c105 4a30e4a Author: Amber Brown <[email protected]> Date: Tue May 21 14:30:47 2019 -0500 Merge remote-tracking branch 'origin/develop' into HEAD commit c99c105 Author: Amber Brown <[email protected]> Date: Mon May 20 17:01:50 2019 -0500 SHHS - Room Join Complexity (#5072) commit d142e51 Merge: d424ba9 24b93b9 Author: Amber Brown <[email protected]> Date: Mon May 20 15:43:08 2019 -0500 Merge remote-tracking branch 'origin/develop' into shhs commit d424ba9 Merge: a1b8767 f1e5b41 Author: Amber Brown <[email protected]> Date: Wed May 15 23:30:22 2019 -0500 Merge remote-tracking branch 'origin/develop' into shhs commit a1b8767 Merge: faee1e9 df2ebd7 Author: Amber Brown <[email protected]> Date: Mon May 13 15:01:58 2019 -0500 Merge remote-tracking branch 'origin/develop' into shhs commit faee1e9 Merge: 12875f9 d216a36 Author: Amber Brown <[email protected]> Date: Wed May 8 16:29:45 2019 -0500 Merge remote-tracking branch 'origin/develop' into shhs commit 12875f9 Merge: ed38141 c1799b0 Author: Amber Brown <[email protected]> Date: Wed May 1 10:55:14 2019 -0400 Merge remote-tracking branch 'origin/develop' into shhs commit ed38141 Author: Amber Brown <[email protected]> Date: Mon Apr 29 20:57:42 2019 +1000 target better for the shhs release docker hub, pt 3 commit bd5f624 Author: Amber Brown <[email protected]> Date: Mon Apr 29 20:43:17 2019 +1000 target better for the shhs release docker hub, pt 2 commit c0f57ca Author: Amber Brown <[email protected]> Date: Mon Apr 29 20:36:35 2019 +1000 target better for the shhs release docker hub commit 1d5cf66 Author: Amber Brown <[email protected]> Date: Mon Apr 29 20:33:36 2019 +1000 no media repo == no path checks commit 25256f9 Author: Amber Brown <[email protected]> Date: Mon Apr 29 20:30:55 2019 +1000 release shhs as a release commit a32aa2c Author: Amber Brown <[email protected]> Date: Mon Apr 29 20:28:40 2019 +1000 patch up docker commit cbc866a Author: Amber Brown <[email protected]> Date: Fri Apr 26 01:40:01 2019 +1000 Remove Python 2 from the SHHS branch CI (#5099)
Codecov Report
@@ Coverage Diff @@
## develop #5783 +/- ##
===========================================
+ Coverage 63.29% 63.34% +0.05%
===========================================
Files 331 331
Lines 36428 36477 +49
Branches 6017 6027 +10
===========================================
+ Hits 23056 23106 +50
+ Misses 11730 11723 -7
- Partials 1642 1648 +6 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some initial thoughts
# If limit_remote_rooms.enabled is True, the room complexity will be | ||
# checked before a user joins a new remote room. If it is above | ||
# limit_remote_rooms.complexity, it will disallow joining or | ||
# instantly leave. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have any docs on what these values mean? It would be good to give some form of hand wavey guidance on what people can set this to, since we're making this config option public.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1 == 500 state events but that kinda means nothing as well because lol :( it is basically a black box number and there's no "good" setting without looking at the complexity of the rooms you want to join and setting it higher than that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hawkowl How does one determine the complexity of a given room?
Also, I think it would be a good idea to document a few tested values for a few configurations (with specs), fe. one for very small HS (raspberry pi or an Atom NUC?), one for a medium one (SSD, modern full power Celeron G, 4GB RAM?), one for a big one, but still not big enough to not be wary of a possible failure (whatever a typical homelab type server looks like). You know, some base for quick set up and further optimization. I can test this a bit on my HS by joining some rooms and seeing how things behave for an extended period of time.
Synapse 1.3.0 (2019-08-15) ========================== Bugfixes -------- - Fix 500 Internal Server Error on `publicRooms` when the public room list was cached. ([\#5851](#5851)) Synapse 1.3.0rc1 (2019-08-13) ========================== Features -------- - Use `M_USER_DEACTIVATED` instead of `M_UNKNOWN` for errcode when a deactivated user attempts to login. ([\#5686](#5686)) - Add sd_notify hooks to ease systemd integration and allows usage of Type=Notify. ([\#5732](#5732)) - Synapse will no longer serve any media repo admin endpoints when `enable_media_repo` is set to False in the configuration. If a media repo worker is used, the admin APIs relating to the media repo will be served from it instead. ([\#5754](#5754), [\#5848](#5848)) - Synapse can now be configured to not join remote rooms of a given "complexity" (currently, state events) over federation. This option can be used to prevent adverse performance on resource-constrained homeservers. ([\#5783](#5783)) - Allow defining HTML templates to serve the user on account renewal attempt when using the account validity feature. ([\#5807](#5807)) Bugfixes -------- - Fix UISIs during homeserver outage. ([\#5693](#5693), [\#5789](#5789)) - Fix stack overflow in server key lookup code. ([\#5724](#5724)) - start.sh no longer uses deprecated cli option. ([\#5725](#5725)) - Log when we receive an event receipt from an unexpected origin. ([\#5743](#5743)) - Fix debian packaging scripts to correctly build sid packages. ([\#5775](#5775)) - Correctly handle redactions of redactions. ([\#5788](#5788)) - Return 404 instead of 403 when accessing /rooms/{roomId}/event/{eventId} for an event without the appropriate permissions. ([\#5798](#5798)) - Fix check that tombstone is a state event in push rules. ([\#5804](#5804)) - Fix error when trying to login as a deactivated user when using a worker to handle login. ([\#5806](#5806)) - Fix bug where user `/sync` stream could get wedged in rare circumstances. ([\#5825](#5825)) - The purge_remote_media.sh script was fixed. ([\#5839](#5839)) Deprecations and Removals ------------------------- - Synapse now no longer accepts the `-v`/`--verbose`, `-f`/`--log-file`, or `--log-config` command line flags, and removes the deprecated `verbose` and `log_file` configuration file options. Users of these options should migrate their options into the dedicated log configuration. ([\#5678](#5678), [\#5729](#5729)) - Remove non-functional 'expire_access_token' setting. ([\#5782](#5782)) Internal Changes ---------------- - Make Jaeger fully configurable. ([\#5694](#5694)) - Add precautionary measures to prevent future abuse of `window.opener` in default welcome page. ([\#5695](#5695)) - Reduce database IO usage by optimising queries for current membership. ([\#5706](#5706), [\#5738](#5738), [\#5746](#5746), [\#5752](#5752), [\#5770](#5770), [\#5774](#5774), [\#5792](#5792), [\#5793](#5793)) - Improve caching when fetching `get_filtered_current_state_ids`. ([\#5713](#5713)) - Don't accept opentracing data from clients. ([\#5715](#5715)) - Speed up PostgreSQL unit tests in CI. ([\#5717](#5717)) - Update the coding style document. ([\#5719](#5719)) - Improve database query performance when recording retry intervals for remote hosts. ([\#5720](#5720)) - Add a set of opentracing utils. ([\#5722](#5722)) - Cache result of get_version_string to reduce overhead of `/version` federation requests. ([\#5730](#5730)) - Return 'user_type' in admin API user endpoints results. ([\#5731](#5731)) - Don't package the sytest test blacklist file. ([\#5733](#5733)) - Replace uses of returnValue with plain return, as returnValue is not needed on Python 3. ([\#5736](#5736)) - Blacklist some flakey tests in worker mode. ([\#5740](#5740)) - Fix some error cases in the caching layer. ([\#5749](#5749)) - Add a prometheus metric for pending cache lookups. ([\#5750](#5750)) - Stop trying to fetch events with event_id=None. ([\#5753](#5753)) - Convert RedactionTestCase to modern test style. ([\#5768](#5768)) - Allow looping calls to be given arguments. ([\#5780](#5780)) - Set the logs emitted when checking typing and presence timeouts to DEBUG level, not INFO. ([\#5785](#5785)) - Remove DelayedCall debugging from the test suite, as it is no longer required in the vast majority of Synapse's tests. ([\#5787](#5787)) - Remove some spurious exceptions from the logs where we failed to talk to a remote server. ([\#5790](#5790)) - Improve performance when making `.well-known` requests by sharing the SSL options between requests. ([\#5794](#5794)) - Disable codecov GitHub comments on PRs. ([\#5796](#5796)) - Don't allow clients to send tombstone events that reference the room it's sent in. ([\#5801](#5801)) - Deny redactions of events sent in a different room. ([\#5802](#5802)) - Deny sending well known state types as non-state events. ([\#5805](#5805)) - Handle incorrectly encoded query params correctly by returning a 400. ([\#5808](#5808)) - Handle pusher being deleted during processing rather than logging an exception. ([\#5809](#5809)) - Return 502 not 500 when failing to reach any remote server. ([\#5810](#5810)) - Reduce global pauses in the events stream caused by expensive state resolution during persistence. ([\#5826](#5826)) - Add a lower bound to well-known lookup cache time to avoid repeated lookups. ([\#5836](#5836)) - Whitelist history visbility sytests in worker mode tests. ([\#5843](#5843))
This adds the room complexity client code to Synapse.