Unable to configure SMTP to use TLS without STARTTLS

[Toniob]

Description

Hi. I'm the sysadmin operating converser.eu homeserver. It seems that my users can't register email in their accounts. New users can't register when they put an email.

When I try to add an email, there is a POST request to https://converser.eu/_matrix/client/r0/account/3pid/email/requestToken but it timeouts. On app.element.io, I've got the error request failed: ETIMEDOUT .

In synapse configuration, the identity server is defined. The mailserver is configured too, and the connection to it is ok.

I've got nginx as a proxy. I've got a timeout :
2020/08/06 15:32:09 [error] 1067425#1067425: *1948 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 2a0e:xxxxxx, server: converser.eu, request: "POST /_matrix/client/r0/account/3pid/email/requestToken HTTP/2.0", upstream: "http://127.0.0.1:8008/_matrix/client/r0/account/3pid/email/requestToken", host: "converser.eu"
But no logs related to this in homeserver.log.

Am I missing something ?

Version information

• Homeserver: converser.eu

• Version: 1.17.0

• Install method: debian package

• Platform: debian buster + backports

[Hyask]

I'm one of these users, and in my account, which already has a registered email, I get the ETIMEDOUT error when trying to perform a password reset. I don't have access to the server, but the end-user symptom is a POST request on https://converser.eu/_matrix/client/r0/account/password/email/requestToken that never gets any answer.

[clokep]

But no logs related to this in homeserver.log.

The first thing to try is upping your log level to INFO or DEBUG and seeing if that gives any additional information.

If the request isn't making it to the homeserver this sounds like an issue with the nginx config.

[Toniob]

Here are the logs. On synapse :

2020-08-08 10:00:43,710 - synapse.access.http.8008 - 249 - DEBUG - POST-615- 2a0e:xxxx - 8008 - Received request: POST /_matrix/client/r0/account/3pid/email/requestToken
2020-08-08 10:00:43,711 - synapse.storage.txn - 402 - DEBUG - POST-615- [TXN START] {get_user_id_by_threepid-1643}
2020-08-08 10:00:43,712 - synapse.storage.SQL - 221 - DEBUG - POST-615- [SQL] {get_user_id_by_threepid-1643} SELECT user_id FROM user_threepids WHERE medium = ? AND address = ?
2020-08-08 10:00:43,713 - synapse.storage.SQL - 226 - DEBUG - POST-615- [SQL values] {get_user_id_by_threepid-1643} ['email', 'xxxx']
2020-08-08 10:00:43,715 - synapse.storage.SQL - 240 - DEBUG - POST-615- [SQL time] {get_user_id_by_threepid-1643} 0.001488 sec
2020-08-08 10:00:43,717 - synapse.storage.txn - 489 - DEBUG - POST-615- [TXN END] {get_user_id_by_threepid-1643} 0.005449 sec
2020-08-08 10:00:43,807 - synapse.storage.txn - 402 - DEBUG - POST-615- [TXN START] {get_threepid_validation_session-1647}
2020-08-08 10:00:43,809 - synapse.storage.SQL - 221 - DEBUG - POST-615- [SQL] {get_threepid_validation_session-1647} SELECT address, session_id, medium, client_secret, last_send_attempt, validated_at FROM threepid_validation_session WHERE client_secret = ? AND medium = ? AND address = ? AND validated_at IS NULL LIMIT 1
2020-08-08 10:00:43,811 - synapse.storage.SQL - 226 - DEBUG - POST-615- [SQL values] {get_threepid_validation_session-1647} ['xxxx', 'email', 'xxxx']
2020-08-08 10:00:43,816 - synapse.storage.SQL - 240 - DEBUG - POST-615- [SQL time] {get_threepid_validation_session-1647} 0.002463 sec
2020-08-08 10:00:43,818 - synapse.storage.txn - 489 - DEBUG - POST-615- [TXN END] {get_threepid_validation_session-1647} 0.011665 sec
2020-08-08 10:00:43,900 - synapse.push.mailer - 309 - INFO - POST-615- Sending email to xxxx
2020-08-08 10:06:05,959 - synapse.http.site - 219 - WARNING - POST-615- Error processing request <XForwardedForRequest at 0x7fd4ab7bfba8 method='POST' uri='/_matrix/client/r0/account/3pid/email/requestToken' clientproto='HTTP/1.0' site=8008>: <class 'twisted.internet.error.ConnectionLost'> Connection to the other side was lost in a non-clean fashion: Connection lost.

And on my mailserver :

Aug  7 10:04:40 mail1 postfix/submissions/smtpd[26520]: connect from matrix.6clones.net[51.15.191.9]
Aug  7 10:07:49 mail1 postfix/submissions/smtpd[30015]: SSL_accept error from matrix.6clones.net[51.15.191.9]: Connection timed out
Aug  7 10:07:49 mail1 postfix/submissions/smtpd[30015]: lost connection after CONNECT from matrix.6clones.net[51.15.191.9]
Aug  7 10:07:49 mail1 postfix/submissions/smtpd[30015]: disconnect from matrix.6clones.net[51.15.191.9] commands=0/0

So synapse is connecting to the mailserver, but the connection fails. I wondered if that was because of the TLS. I've configured Synapse to use 465 port, which is direct TLS, not STARTTLS method. And it seems Synapse doesn't handle it. I've changed the port to 587 and now it works.

So, new question, why Synapse can't connect to a mailserver with direct TLS connection ?

[clokep]

@Toniob What's the value of email.require_transport_security in your config?

[Toniob]

require_transport_security: True

With port 465, it fails, with 587, it works.

[clokep]

The require_transport_security configuration option seems to be a bit poorly named (but is inherited from Twisted and has a decent comment around):

Uncomment the following to require TLS transport security for SMTP. By default, Synapse will connect over plain text, and will then switch to TLS via STARTTLS if the SMTP server supports it. If this option is set, Synapse will refuse to connect unless the server supports STARTTLS.

require_transport_security means that STARTTLS is still used, but it requires it to succeed.

I do not believe there is a way to configure the initial connection to the SMTP server use TLS. (Note that Synapse uses Twisted's sendmail method under-the-hood, which also doesn't seem to support this configuration.)

I'll update this issue as a request for just using TLS without STARTTLS.

[jacksitlab]

Sorry to ask here, but I am not able to get this running with synapse v1.41.0. What is now the correct configuration? The only log I see is the info log for "Sending Mail". No warnings or errors.
My current config is:

registrations_require_3pid:
    - email
allowed_local_3pids:
    - medium: email
      pattern: '^[^@]+@<my-domain>$'
email:
    smtp_host: smtp.***
    smtp_port: 465
    smtp_user: '****'
    smtp_pass: '****'
    require_transport_security: False
    enable_tls: True 
    notif_from: "Your Friendly %(app)s homeserver <no-reply@<my-domain>>"
    subjects:
        email_validation: "[%(server_name)s] Validate your email"

Sadly I am not able to get logs from the email-server. Can someone please verify that this is the correct configuration. Or do you have a hint how I can enable more logs for the twisted library to find out what the problem on my side is?

[Jieiku]

Same problem here port 587 works, port 465 does not.

The mail server itself supports 465 without issue, I have other things configured to use it over smtp via port 465 such as nextcloud, pfsense notification, grafana, etc.

Port 465 is always encrypted under SSL, why is it not supported?

Why do these settings even exist, couldn't they be inferred from the configured port?

  enable_tls: true
  require_transport_security: true

This is configured in the homeserver.yaml

because I am on debian mine is here:
sudo nano /etc/matrix-synapse/conf.d/config.yaml

but non debian users might be here:
sudo nano /etc/matrix-synapse/homeserver.yaml

This is 587 which works:

email:
  smtp_host: "smtp.example.com"
  smtp_port: 587
  smtp_user: "[email protected]"
  smtp_pass: "ExampleExample"
  notif_from: "%(app)s Homeserver <[email protected]>"
  client_base_url: "https://matrix.example.com"
  app_name: "My Matrix"
  enable_tls: true
  require_transport_security: true

This is 465, does not work:

email:
  smtp_host: "smtp.example.com"
  smtp_port: 465
  smtp_user: "[email protected]"
  smtp_pass: "ExampleExample"
  notif_from: "%(app)s Homeserver <[email protected]>"
  client_base_url: "https://matrix.example.com"
  app_name: "My Matrix"
#  enable_tls: true
#  require_transport_security: true

Also tried this for 465, does not work:

email:
  smtp_host: "smtp.example.com"
  smtp_port: 465
  smtp_user: "[email protected]"
  smtp_pass: "ExampleExample"
  notif_from: "%(app)s Homeserver <[email protected]>"
  client_base_url: "https://matrix.example.com"
  app_name: "My Matrix"
  enable_tls: true
#  require_transport_security: true

[richvdh]

#10546 did not fix this.