From d9a50b9f31761cfe1f2c5b7f7c90cf818049e811 Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 19:35:33 +0100 Subject: [PATCH 1/8] Added a new PEP as draft: Enabling certificate verification by default for stdlib mail modules --- peps/pep-8106.rst | 90 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 peps/pep-8106.rst diff --git a/peps/pep-8106.rst b/peps/pep-8106.rst new file mode 100644 index 00000000000..2db1d4ddcd2 --- /dev/null +++ b/peps/pep-8106.rst @@ -0,0 +1,90 @@ +PEP: 8106 +Title: Enabling certificate verification by default for stdlib mail modules +Version: $Revision$ +Last-Modified: $Date$ +Author: Martin Schobert +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 17-Dec-2023 + +Abstract +======== + +The Python mail libraries such as ``smtplib``, ``imaplib`` and ``pop3lib`` +do not verify server certificates per default, when a client based on these +mail libraries connects to a server via TLS. Any server certificate is +accepted per default. This means, a client can't ensure that it connects +to the server to which the connection was intended. + +This allows an active attacker in a machine-in-the-middle postion to intercept +communication, read mail contents, credentials or may abuse an SMTP server +for spaming. + +This PEP proposes to enable verification of X509 certificates for Python's +mail clients by default, subject to opt-out on a per-call basis. This change +would be applied to all maintained Python versions. + +Rationale +========= + +It can be expected that a Python mail client, written in the most simple way, is +secure by default. If progam code wants to insecurely connect to a server, this +action should require and explicit confirmation. Not verifying a server certificate +and accepting it violates PEP 20's principle "errors should never pass silently." + +It can also be expected that Python standard libraries behave in a consitent way. +While Python's HTTP stdlib modules verify server certificates per default after +implementing the change request from PEP 476, developers may expect Python mail +libraries to behave similar. + +The failure of various applications to note Python's negligence in this matter +is a source of vulnerabilities [#]_ [#]_ [#]_ [#]_. + +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-39441 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-38686 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-39441 +.. [#] https://www.pentagrid.ch/en/blog/python-mail-libraries-certificate-verification/ + +Technical Details +================= + +Python would use the system provided certificate database on all platforms. +Failure to locate such a database would be an error, and users would need to +explicitly specify a location to fix it. + +Python will change the ``_create_stdlib_context`` as default context in the +mail libraries with ``ssl.create_default_context``. + +If a developer wants to disable certificate verification, an explicitly created +SSL/TLS context may be passed that has been created with ``_create_unverified_context``. + +Trust database +-------------- + +This PEP proposes using the system-provided certificate database. + + +Opting out +---------- + +For users who wish to opt out of certificate verification on a single +connection, they can achieve this by providing the ``ssl_context`` +for ``imaplib.IMAP4_SSL`` respectively ``context`` for ``smtplib.SMTP_SSL`` and +``poplib.POP3_SSL``. + + + +Copyright +========= + +This document has been placed into the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 From bb2bcc4b437ee404956bb5491cf50666c9f63f15 Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 19:42:29 +0100 Subject: [PATCH 2/8] Added @vstinner as sponsor for PEP 8106. --- .github/CODEOWNERS | 1 + peps/pep-8106.rst | 1 + 2 files changed, 2 insertions(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 03d9c709e9f..5293b48149e 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -698,3 +698,4 @@ peps/pep-8016.rst @njsmith @dstufft peps/pep-8100.rst @njsmith # peps/pep-8101.rst # peps/pep-8102.rst +peps/pep-8106.rst @vstinner diff --git a/peps/pep-8106.rst b/peps/pep-8106.rst index 2db1d4ddcd2..260f7dcd175 100644 --- a/peps/pep-8106.rst +++ b/peps/pep-8106.rst @@ -7,6 +7,7 @@ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 17-Dec-2023 +Sponsor: @vstinner Abstract ======== From 0f97851f3a1cd4fcecc3cda4a672696af246d6f7 Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 19:49:56 +0100 Subject: [PATCH 3/8] Created date fixed. --- peps/pep-8106.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-8106.rst b/peps/pep-8106.rst index 260f7dcd175..c26b9713ef2 100644 --- a/peps/pep-8106.rst +++ b/peps/pep-8106.rst @@ -6,7 +6,7 @@ Author: Martin Schobert Status: Draft Type: Standards Track Content-Type: text/x-rst -Created: 17-Dec-2023 +Created: 17-Nov-2023 Sponsor: @vstinner Abstract From 3c86a13d5d9de2fa7513697bdd32744dd8bf0540 Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 20:18:35 +0100 Subject: [PATCH 4/8] Fixed sponsor format to hopefully pass PEP-12 commit check, changed section names, added backward compatiblity section. --- peps/pep-8106.rst | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/peps/pep-8106.rst b/peps/pep-8106.rst index c26b9713ef2..335254299cd 100644 --- a/peps/pep-8106.rst +++ b/peps/pep-8106.rst @@ -7,10 +7,11 @@ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 17-Nov-2023 -Sponsor: @vstinner +Sponsor: Victor Stinner +Python-Version: 3.x -Abstract -======== +Motivaton +========= The Python mail libraries such as ``smtplib``, ``imaplib`` and ``pop3lib`` do not verify server certificates per default, when a client based on these @@ -47,7 +48,7 @@ is a source of vulnerabilities [#]_ [#]_ [#]_ [#]_. .. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-39441 .. [#] https://www.pentagrid.ch/en/blog/python-mail-libraries-certificate-verification/ -Technical Details +Specification ================= Python would use the system provided certificate database on all platforms. @@ -65,6 +66,21 @@ Trust database This PEP proposes using the system-provided certificate database. +Backwards compatibility +----------------------- + +This change may result in failed connections, when servers do not use +valid and trusted certificates. With invalid or untrusted certificates, the Python +TLS library will raise an exception during TLS handshake. + +Nevertheless, this will only happen with invalid mail server certificates or +server certificates that do not rely on a trust anchor known to Python's +runtime environment. In these cases it is necessary that the system admin +either installs a valid mail server certificate or the client ensures +there is a trust anchor defined (a valid certificate chain leading from a CA +file to the server certificate). As an unrecommended alternative, the client +code may use an SSL context that does not verify certificates as descibed +in the next section. Opting out ---------- @@ -75,7 +91,6 @@ for ``imaplib.IMAP4_SSL`` respectively ``context`` for ``smtplib.SMTP_SSL`` and ``poplib.POP3_SSL``. - Copyright ========= From 58dfba0810f38a6419e3f20833c3ea28c144f69d Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 20:25:40 +0100 Subject: [PATCH 5/8] Changed PEP version to 734. --- .github/CODEOWNERS | 3 ++- peps/{pep-8106.rst => pep-0734.rst} | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) rename peps/{pep-8106.rst => pep-0734.rst} (99%) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 5293b48149e..95ab36a1a8c 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -611,6 +611,8 @@ peps/pep-0730.rst @ned-deily peps/pep-0731.rst @gvanrossum @encukou @vstinner @zooba @iritkatriel peps/pep-0732.rst @Mariatta peps/pep-0733.rst @encukou @vstinner @zooba @iritkatriel +peps/pep-0734.rst @vstinner + # ... # peps/pep-0754.rst # ... @@ -698,4 +700,3 @@ peps/pep-8016.rst @njsmith @dstufft peps/pep-8100.rst @njsmith # peps/pep-8101.rst # peps/pep-8102.rst -peps/pep-8106.rst @vstinner diff --git a/peps/pep-8106.rst b/peps/pep-0734.rst similarity index 99% rename from peps/pep-8106.rst rename to peps/pep-0734.rst index 335254299cd..8a91aeab8f6 100644 --- a/peps/pep-8106.rst +++ b/peps/pep-0734.rst @@ -1,4 +1,4 @@ -PEP: 8106 +PEP: 734 Title: Enabling certificate verification by default for stdlib mail modules Version: $Revision$ Last-Modified: $Date$ From 50424a3f0be8f5fecb4c09d92a41afbccddc4e9c Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 20:37:45 +0100 Subject: [PATCH 6/8] Changed header order to pass the PEP-12 check. --- .github/CODEOWNERS | 2 +- peps/{pep-0734.rst => pep-0735.rst} | 14 +++++++------- 2 files changed, 8 insertions(+), 8 deletions(-) rename peps/{pep-0734.rst => pep-0735.rst} (97%) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 95ab36a1a8c..ad740e527c7 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -611,7 +611,7 @@ peps/pep-0730.rst @ned-deily peps/pep-0731.rst @gvanrossum @encukou @vstinner @zooba @iritkatriel peps/pep-0732.rst @Mariatta peps/pep-0733.rst @encukou @vstinner @zooba @iritkatriel -peps/pep-0734.rst @vstinner +peps/pep-0735.rst @vstinner # ... # peps/pep-0754.rst diff --git a/peps/pep-0734.rst b/peps/pep-0735.rst similarity index 97% rename from peps/pep-0734.rst rename to peps/pep-0735.rst index 8a91aeab8f6..6bdb005118e 100644 --- a/peps/pep-0734.rst +++ b/peps/pep-0735.rst @@ -1,16 +1,13 @@ -PEP: 734 +PEP: 735 Title: Enabling certificate verification by default for stdlib mail modules -Version: $Revision$ -Last-Modified: $Date$ Author: Martin Schobert +Sponsor: Victor Stinner Status: Draft Type: Standards Track -Content-Type: text/x-rst Created: 17-Nov-2023 -Sponsor: Victor Stinner -Python-Version: 3.x +Resolution: -Motivaton +Abstract ========= The Python mail libraries such as ``smtplib``, ``imaplib`` and ``pop3lib`` @@ -23,6 +20,9 @@ This allows an active attacker in a machine-in-the-middle postion to intercept communication, read mail contents, credentials or may abuse an SMTP server for spaming. +Motivation +=========== + This PEP proposes to enable verification of X509 certificates for Python's mail clients by default, subject to opt-out on a per-call basis. This change would be applied to all maintained Python versions. From 2e41cb470bb709d4cc63ba6f26c5fcd0f9cc42ab Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 20:41:45 +0100 Subject: [PATCH 7/8] Added URL to Cpython ticket #91826 to reference discussion. --- peps/pep-0735.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0735.rst b/peps/pep-0735.rst index 6bdb005118e..99ced72a0df 100644 --- a/peps/pep-0735.rst +++ b/peps/pep-0735.rst @@ -5,7 +5,7 @@ Sponsor: Victor Stinner Status: Draft Type: Standards Track Created: 17-Nov-2023 -Resolution: +Resolution: https://github.com/python/cpython/issues/91826 Abstract ========= From bd13e6694566552d85c408018f9eaa2f3c5c787e Mon Sep 17 00:00:00 2001 From: Martin Schobert Date: Fri, 17 Nov 2023 20:52:19 +0100 Subject: [PATCH 8/8] Resolution header seems to be optional. Therefore removed. --- peps/pep-0735.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/peps/pep-0735.rst b/peps/pep-0735.rst index 99ced72a0df..aa680e3a6cb 100644 --- a/peps/pep-0735.rst +++ b/peps/pep-0735.rst @@ -5,7 +5,6 @@ Sponsor: Victor Stinner Status: Draft Type: Standards Track Created: 17-Nov-2023 -Resolution: https://github.com/python/cpython/issues/91826 Abstract =========