Ability to disable End-To-End Encryption via Config

[neilisfragile]

Description:
Originally proposed in element-hq/element-web#4367 - for the case of element-hq/element-web#4367 is was sufficient to address via power level settings. Creating a new issue to track doing this via a config setting.

The original ask:-
"I would like the ability to disable end-to-end encryption in my self hosted Synapse instance. I have a legal requirement to provide audit-able chat logs, which is obviously impossible if there's nothing preventing my end-users from encrypting any room they create."

[gordon-quad]

Maybe better approach could be to set default power level settings through config?

[Bun-Bun]

I also would like this option. The way encryption is handles is far too confusing for my users and I do not want to deal with them getting locked out of their messages.

[cnvandijk]

Related: element-hq/element-web#6660.

[mlaily]

This feature is becoming more urgent now that Riot has enabled encryption by default.

I have a home server for family and friends, and I'm glad I can self host a chat server matching commercial instant-messaging in quality, but I really can't justify to them the need for end to end encryption, given the hassle and risk (of losing data) it adds!

IMO, https is largely sufficient for my security needs, given I can trust the server (because I own it).

[Bun-Bun]

I agree with @mlaily. As it stands I am scared to update any of my infrastructure in fear of rooms getting automatically encrypted and my users losing access to business data.

[MurzNN]

What happens if we simply disable m.key.*, m.room.encrypted and m.room.encryption endpoints for local Synapse instance via nginx proxy? This will disable creating keys and E2EE rooms on server, or not? If not, which other endpoints can we block for disable all E2EE stuff?

[erikjohnston]

You can use the spam checker or third party rules plugin modules to disable/filter out events related to encryption, which is probably easier than relying on rules in nginx

[MurzNN]

Thanks for idea, I fill feature request about this in devture/matrix-corporal#8

[anilk9096]

hi ,
we had blocker with this . when can we expect this or please suggest any alternative way to disable this .

[anilk9096]

user are confusing to verify with lot of process

[mlaily]

Same. I don't want to upgrade until e2ee is easier, or until it is possible to make it so clients (Riot) don't try to force me to use it.

[anilk9096]

Even using old version . Even skipping encryption setup keys first time login for some user getting default enabled encryption why ?

[clokep]

It does not fully solve this, but element-hq/element-web#7639 added a config options for whether new rooms default to E2EE or not. Note that this will be available in the next version of Synapse.

[Bun-Bun]

element-hq/element-web#7639 doesn't fix it at all.

The client can still default/force the room to be encrypted. We need an option to fully disable/block e2ee on the server.

[mlaily]

It does not fully solve this, but element-hq/element-web#7639 added a config options for whether new rooms default to E2EE or not. Note that this will be available in the next version of Synapse.

That's good, but what about not bothering users to setup recovery keys and verifications upon login? (I'm more interested in not bothering clueless users, compared to preventing them from enabling e2ee if they want to)

[Bun-Bun]

It does not fully solve this, but element-hq/element-web#7639 added a config options for whether new rooms default to E2EE or not. Note that this will be available in the next version of Synapse.

That's good, but what about not bothering users to setup recovery keys and verifications upon login? (I'm more interested in not bothering clueless users, compared to preventing them from enabling e2ee if they want to)

That specifically is a client issue, aka Riot. However that ties into the Synapse config variable which the devs completely ignored the option to disable e2ee

[richvdh]

@Bun-Bun complaining in 15 places that we haven't implemented your favourite feature is unlikely to make us consider your request favourably.

As far as I can tell what you are demanding is https://github.com/vector-im/riot-web/issues/8067, which is only tangentially related to most of the PRs you have commented on.

I realise you are frustrated, but that does not excuse your rudeness.

[anilk9096]

hi clokep. thank you for response. Tried building dev branch code with element-hq/element-web#7639. I thought off will work :) . but it was not use full at all. kindly try to give option to disable encryption in private or direct chat like when creating in group . thank you very much

[Bun-Bun]

@Bun-Bun complaining in 15 places that we haven't implemented your favourite feature is unlikely to make us consider your request favourably.

As far as I can tell what you are demanding is vector-im/riot-web#8067, which is only tangentially related to most of the PRs you have commented on.

I realise you are frustrated, but that does not excuse your rudeness.

I apologize for complaining in the matrix-doc issue and in the Synapse channel. I am not a developer and exactly how PR's and issues are supposed to work is unclear to me.

Please try to understand my frustration. When I first indicated my support for the option to disable/block e2ee it was more a want as at that time I was satisfied with the workaround patch to rooms.py as discussed here #4367 However that changed when riot-web defaulted to encryption on, which given the way regular users work is effectively forcing encryption on. As indicated in #4367 that change broke the rooms.py workaround as it prevented direct chats from even being created. I see another user figured out an additional patch to workaround it again however that doesn't change the larger issue. The concept of default on e2ee is breaking for admins of homeservers like mine where company policy or legal obligations dictate that all data be auditable. Further on that point e2ee is complicated and confusing for regular users and can very easily lead to employees being locked out of business data with no way for the server admin to correct it.

At that time I raised the concern here element-hq/element-web#6779 and had discussion with t3chguy as well as in the various support rooms on matrix.org. t3chguy (a riot dev?) raised a very good point and I 100% agree with him. The switch controlling e2ee behavior needs to be on the server side so that the behavior is enforced at the homeserver level and clients can be configured appropriately. After further discussions the general consensus was this is a option that should exist but it really involves the matrix spec and I was pointed to that repository. I created an issue there and revised it to add options for other use cases that other users raised concerns over. It appears to me and other users that the issue I created has gone completely unnoticed and/or not considered. In my opinion #7639 and #2431 were directly related, as they are adding the option to control e2ee behavior, much to how I described the options in my matrix-doc issue, and were the perfect opportunity to explore and add this option to the synapse server. Which is why I asked there why the disable option was not considered.

Then I joined the synapse channel and asked how we can get this issue escalated and yes that conversation quickly degraded and after the link to the vector.im contact page I left as nothing productive was happening in that conversation.

Now faced with riot dev's telling me it's a synapse/spec issue and the spec guys saying there isn't any resources to do anything about it and synapse guys pointing it to be strictly a client issue I can't help but feel everyone is just passing the buck and I'm in that annoying support phone call loop of being transferred from department to department with no resolution. One thing I did get out of the synapse conversation is this was done https://github.com/vector-im/riot-web/blob/develop/docs/e2ee.md which is a great step in the right direction and I started exploring it's discussions. Even though the riot guys (or maybe only t3chguy?) think this is a server side issue they are the only ones discussing this or at least the riot repository is the only place I've seen said discussion. Specifically this post element-hq/element-web#13539 (comment) which led me to the related PR and issue matrix-org/matrix-react-sdk#4605 element-hq/element-web#13705 which had great work done on not only adding a config switch but how that change impacts the UI and how the user experience is affected. In my opinion disabling e2ee again is directly related as it has the same considerations with UI and user experience as well fits in with that .well-known config switch. Should it be handled at the homeserver level? Yes absolutely but the only place that any devs seem to be discussing these issues is on the riot repository hence why I asked the question there. I think at least for short term viability a riot-web config will help admin/users and spur further discussion to implement it properly.

I wish more than anything I had the capability and capacity to contribute directly to the development of these projects, but I am not a developer and I have my own team to manage. My use-case is for business and now that I know about vector.im I do plan to reach out and learn more about how I can support these projects and sponsor the features I and others need. These issues date back more than a year with no indication of any progress or consideration other than the mentioned riot-web issue/pr. That said, do you understand how frustrating it is from an admin/user perspective to have these independent yet closely intertwined projects pass the buck and have questions about how to escalate issues result in arguments about semantics? I am not demanding https://github.com/vector-im/riot-web/issues/8067 (especially since that is on the riot-web repository but it should be added to synapse) but rather asking how do I get this issue to the attention of the appropriate people? How can I help as a non developer?

[mlaily]

Hum. That was a long rant. :)

The hard truth is that this is a free and open source project and the devs don't owe us users anything.

I wish the issues I care about (this one in particular) would be assigned more importance and priority, but ultimately, this is a free an open source project, and even if it is frustrating, I think we should understand that if our priorities don't match the devs' ones, this might not be the best project for us to use. (Unfortunately for me and maybe you, this project is still the closest to my list of prerogatives for a self hosted IM...)

[LavTeamProject]

I decided not wait until E2EE disabling is implemented and wrote my own plugin to fix this - https://github.com/digitalentity/matrix_encryption_disabler

Feel free to use if it fits your use-cases.

@marsianer it's also easily modifyable to force-enable the encryption.

how to install this plugin?

[demlak]

jes..

this thread is full of people who also want to get rid of annoying "validation popups".. but your kind of communication is annoying, too.

calm down. your rant does not help anyone.

[mhtvsSFrpHdE]

Another sample here: my friend (I ask them to move from another software and register on my home server)
bought a new phone and factory reset the old one, then he is locked out.

Matrix devs optimized E2EE too good, a user know nothing, read nothing, just click-click-click can also "enjoy the service".
The one havo no knowledge about encryption, he just complaint why chat history is gone.
Says "but I remember the password! I already logged in!"

After some investigation, I find that in the latest clients,
start direct chat will create a new room with encryption enabled.
Most clients provide no option to disable encryption,
nheko have a switch to turn off, but server won't response.

Manually created room without encryption won't treat as direct chat by clients,
therefore some of them have display bug.

So my request is simple, E2EE off direct chat.
Now when I think of my friends,
I start to considering, between Matrix standard, me and my friend,
There must be one or more foolish to end up with this situation.

[grantm]

Since the merge of element-hq/element-web#12618, I have been able to disable end-to-end-encryption on our server by adding this to the homeserver.yaml (in particular "m.room.encryption": 999 requires a power level higher than any user has):

default_power_level_content_override:
  private_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 1

Can someone confirm that the requested functionality has now been implemented in element-hq/element-web#12618 and this issue can be closed? Or is there more to it and some other aspect that I'm missing?

EDIT: I'm not sure what happened with the issue links quoted above and how they ended up pointing at the element-web repo. They were meant to pointing to PR 12618 in this repo

[mhtvsSFrpHdE]

@grantm I can plan a test on my server, currently I use synapse docker image and have no knowledge about compile "nightly code", when will this update upload to docker?

[grantm]

@grantm I can plan a test on my server, currently I use synapse docker image and have no knowledge about compile "nightly code", when will this update upload to docker?

I believe this commit has been included in all releases since v1.60.0

[mhtvsSFrpHdE]

@grantm I use these steps to update my server

• sudo docker stop synapse
• sudo docker pull sudo docker pull matrixdotorg/synapse
• sudo nano /var/lib/docker/volumes/synapse-data/_data/homeserver.yaml
• Remove disable E2EE module
• Append to config

default_power_level_content_override:
  private_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 1

sudo docker start synapse

After I can log in to server, I try to start direct message use element Android,
but still result in encrypted room.

On nheko PC client, I can see the "<room creator> enabled encryption" message.
This message won't appear in rooms created when "disable E2EE module" installed.

Meanwhile, a shield icon appears after the message.
If I log out without backup keys, login again, these message will be unreadable,
clarify they are indeed encrypted message.

Am I missed something?

[grantm]

@mhtvsSFrpHdE - thank you for sanity-testing my earlier comments.

Not sure what happened when I tried to link to the relevant commit in my original comment. This is the link I intended.

Also my sample config had "events_default": 1 when "events_default": 0 would have made more sense.

Having gone back and retested I find that it doesn't work quite as I'd hoped.

If I attempt to create a room in Element with the "Enable end-to-end encryption" option turned off, then the room is created and even a room admin does not have sufficient power to enable encryption, so that option is disabled in the Element UI. This is what I had tested successfully previously.

On the other hand, if I attempt to create a room in Element with the "Enable end-to-end encryption" option turned on, then Element displays an error after the server responds with 403 Forbidden and this response body:

{
  "errcode":"M_FORBIDDEN",
  "error":"You don't have permission to post that to the room. user_level (100) < send_level (999)"
}

Unfortunately the room does get created before the error condition is encountered. It is not fully initialized however and the overridden power levels have not been applied. So the room admin can go into room settings to complete the remaining steps of configuring the room, including enabling encryption.

A second failure mode is if you open a direct chat with another user and send the first message to create the room, then the power level overrides don't get applied, so you can enable encryption via room settings (depending on server config it may have been enabled by default).

[mhtvsSFrpHdE]

@grantm digitalentity/matrix_encryption_disabler#10 (comment)

We have a case called "Non-standard usage".

According to collected information, start direct chat is assumed to force E2EE enabled,
therefore, clients like nheko or element Android can't perfectly handle non-E2EE direct chat.

You will start direct chat on element Android, it will prompt encryption is enabled anyway,
ignore this and send your first message, the app will hang for a while,
later will say failed to send message, but invite is actually sent,
just ignore this, back to main interface, and ask your friend to accept the invite.
During your waiting, do not send the message again, or delete the failed message,
or leave created empty room.

Once the friend joins room, click on room to open chat interface,
every function will work as intended, and you have a direct chat without encryption.
If you have manually created 2 user non encrypted room with same user before, leave the room,
(these room won't be recognized as direct chat on nheko)
otherwise, you can't use function like change display name in room locally like
/myroomnick <display-name> on element Android.

This information above is based on experiments and statistics,
I have zero knowledge about how code is actually run and what happened on the server.

E2EE module's implementation may more aggressive to block certain type of request completely other than just default level...?

If any progress on this topic, send me the steps to execute at any time

[Mogaba]

So 4,5 years have passed and it's still an issue. I don't understand: is it so difficult to add an option to completely disable encryption? Or there's another reason?

[dklimpel]

So 4,5 years have passed and it's still an issue. I don't understand: is it so difficult to add an option to completely disable encryption? Or there's another reason?

IMHO this is Open Source. If it is needed PR and contributions are welcome. Alternatively, it can certainly be sponsored financially.

[mhtvsSFrpHdE]

@grantm Hi, my docker command is not updating my server, so I'm still using 1.75.0

This can take a while.

To properly update a docker container instance:

#4401 (comment)

Plus:

sudo docker stop synapse
sudo docker rm synapse
sudo docker run -d --name synapse --mount type=volume,src=synapse-data,dst=/data -p 8008:8008 -p 8448:8448 matrixdotorg/synapse:latest

The last command will create and start synapse, but won't delete exist data.
8008 and 8448 is port configuration.

[mhtvsSFrpHdE]

@grantm By updating to synapse server 1.84.0 and use Element Android 1.5.32,
Remove disable E2EE module, append to config,
if try start direct chat, Element Android will say, "Your server admin has disabled end-to-end encryption by default in private rooms & Direct Messages."

The whole invite process no longer hang or fail, GUI logo assume this is an encrypted room,
all messages are still encrypted (with shield icon) and no longer readable if sign out without backup keys.

[theslash]

So 4,5 years have passed and it's still an issue. I don't understand: is it so difficult to add an option to completely disable encryption? Or there's another reason?

IMHO this is Open Source. If it is needed PR and contributions are welcome. Alternatively, it can certainly be sponsored financially.

This is of course right. Although so many requests for this get completely shut down or ingored.
Is this somewhere on the list of future Features or not?

[mhtvsSFrpHdE]

@theslash I think @grantm is trying to solve this once before, but it seems something still not right. In the meanwhile, you can try E2EE plugin until "official support" digitalentity/matrix_encryption_disabler#10 (comment)

[grantm]

With the following in my homeserver config, any attempt to create an encrypted room will be refused with a 401 error:

default_power_level_content_override:
  private_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 0
  trusted_private_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 0
  public_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 0

The "m.room.encryption": 999 lines are the ones that do the job but all values need to be specified. The values for the other events might need to be different in your use case.

When I attempted this earlier, I failed to set the overrides for trusted_private_chat and public_chat. I also ran into a bug where the permission check was happening too late so a partially initialised room got created with encryption disabled but the permission overrides had not been applied so the user could then change the config to enable encryption. That bug has now been fixed.

Another part of the solution is publishing a client config to disable encryption by default and to disable the option for the user to enable it. This means that users are less likely to ever see the 401 error. Bear in mind that the linked setting only applies to the element clients specifically and clients may ignore the contents of the /.well-known/matrix/client file anyway.

So it seems to me that Synapse does now provide the "Ability to disabled End-To-End Encryption via Config" and this issue could be closed..

[dolphinscorp]

With the following in my homeserver config, any attempt to create an encrypted room will be refused with a 401 error:

default_power_level_content_override:
  private_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 0
  trusted_private_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 0
  public_chat:
    "events":
      "m.room.avatar": 50
      "m.room.canonical_alias": 50
      "m.room.encryption": 999
      "m.room.history_visibility": 100
      "m.room.name": 50
      "m.room.power_levels": 100
      "m.room.server_acl": 100
      "m.room.tombstone": 100
    "events_default": 0

The "m.room.encryption": 999 lines are the ones that do the job but all values need to be specified. The values for the other events might need to be different in your use case.

When I attempted this earlier, I failed to set the overrides for trusted_private_chat and public_chat. I also ran into a bug where the permission check was happening too late so a partially initialised room got created with encryption disabled but the permission overrides had not been applied so the user could then change the config to enable encryption. That bug has now been fixed.

Another part of the solution is publishing a client config to disable encryption by default and to disable the option for the user to enable it. This means that users are less likely to ever see the 401 error. Bear in mind that the linked setting only applies to the element clients specifically and clients may ignore the contents of the /.well-known/matrix/client file anyway.

So it seems to me that Synapse does now provide the "Ability to disabled End-To-End Encryption via Config" and this issue could be closed..

Hi it seems working. Enable encryption option is grayed out. that's fine i think.

Now issue is that when we start a direct chat/message it by default encrypted. But since encryption is disabled, message is unable to send. How we can disable by default encryption. For direct message it does not prompt for option to encrypt or not. it just enable e2ee.

Is there some option can be set in homeserver.yaml to disable default e2ee enabled?

I have tried this encryption_enabled_by_default_for_room_type: off in homeserver.yaml but have no luck

[grantm]

Now issue is that when we start a direct chat/message it by default encrypted. But since encryption is disabled, message is unable to send. How we can disable by default encryption. For direct message it does not prompt for option to encrypt or not. it just enable e2ee.

You'll want to ensure that your server has encryption_enabled_by_default_for_room_type set to off.

Then you're left with a client issue. You need to configure the client to either default to E2EE disabled, or to not ask and to force E2EE to be disabled. That's what I was referring to with the client config link, which describes some entries that you'll want to add to your https://your.domain/.well-known/matrix/client file. This is a JSON file that clients look to for configuration information.

[dolphinscorp]

Hi Thank you very much, yesterday I tried to find this .well-known directory and client file, Matrix Server. But could not find it on matrix server. Also could not find it in element-web (root directory)

Can you please let me know where is that client json file, exact linux path please?

[grantm]

Can you please let me know where is that client json file, exact linux path please?

You might not have one currently - it's optional. If you look in the browser devtools network tab when loading your web client, you might see a request for the file and a 404 response.

One possible implementation is to simply put a static file on the web server for the domain used in your matrix user ids. So for example, if someone fires up a web client (e.g.: https://app.element.io/) and says their matrix user is @myuser:example.com, then the client will attempt to request the URL https://example.com/.well-known/matrix/client and that file (if it exists) would typically include the address of the matrix homeserver for the domain, as well as client settings. Clients may not be able to access the file if it's not served up with appropriate CORS headers.

More information at: https://spec.matrix.org/v1.3/client-server-api/#well-known-uri

We've already strayed far beyond the bounds of what's reasonable within an issue thread. You're now asking questions that would be better addressed in a support forum like: https://matrix.to/#/#synapse:matrix.org