add Oauth2 support

[drakkan]

fixes #129

The Microsoft variant is untested. Do no merge.

If in the meantime you have suggestions to improve the code, they are welcome.
If anyone can share a Microsoft Exchange Online account for testing and is interested in this feature, please contact me privately

[wneessen]

Thanks again for the work you're putting into this @drakkan. I do have a private O365 account for Office, so I have access to outlook.com. Not sure if this also supports MS Exchange Online or if that's only for business customers. I'm happy to test with my credentials if this works.

[drakkan]

@wneessen I think you are right. I can register an application targeting personal accounts. I think I can also test it myself this way. I will try it during the next week

[james-d-elliott]

Worst case I have access to both providers enterprise offerings and I have access to the Microsoft Sponsored Azure Program.

[drakkan]

uhmm, the protocol implementation looks correct, maybe I have to fix something with my app registration

2023/05/29 11:17:10 DEBUG: C <-- S: 250 MR1P264CA0133.outlook.office365.com Hello [82.84.55.116]
SIZE 157286400
PIPELINING
DSN
ENHANCEDSTATUSCODES
AUTH LOGIN XOAUTH2
8BITMIME
BINARYMIME
CHUNKING
SMTPUTF8
2023/05/29 11:17:10 DEBUG: C --> S: AUTH XOAUTH2
2023/05/29 11:17:10 DEBUG: C <-- S: 334 
2023/05/29 11:17:10 DEBUG: C --> S: dXNlcj1vdXRsb29rX0I0QUREOUQ4QjJEMkZCN0VAb3V0bG9vay5jb20BYXV0aD1CZWFyZXIgZXlKMGVYQWlPaUpLVjFRaUxDSnViMjVqWlNJNkltcEdaWFJhV1ZoeWVWTk5NMmRzY1ZBMk9WWk1UMGQ2ZEU1S1owUkpaWFZrVUMxTlNsSlZUMFZJZVVVaUxDSmhiR2NpT2lKU1V6STFOaUlzSW5nMWRDSTZJaTFMU1ROUk9XNU9VamRpVW05bWVHMWxXbTlZY1dKSVdrZGxkeUlzSW10cFpDSTZJaTFMU1ROUk9XNU9VamRpVW05bWVHMWxXbTlZY1dKSVdrZGxkeUo5LmV5SmhkV1FpT2lKb2RIUndjem92TDI5MWRHeHZiMnN1YjJabWFXTmxMbU52YlNJc0ltbHpjeUk2SW1oMGRIQnpPaTh2YzNSekxuZHBibVJ2ZDNNdWJtVjBMemRtTmpCbE5ETTBMV0V4TlRndE5EZzNOeTA0WXpBeUxUa3dZbVV5TlRFd1lqbGlOUzhpTENKcFlYUWlPakUyT0RVek5URTFNamtzSW01aVppSTZNVFk0TlRNMU1UVXlPU3dpWlhod0lqb3hOamcxTXpVMk5ERTFMQ0poWTJOMElqb3hMQ0poWTNJaU9pSXhJaXdpWVdsdklqb2lRVmRSUVcwdk9GUkJRVUZCUkVJdk1saGhTWFJPY0hWSlozWlVjV0kxUm10RFV6aGpWRTA1YnpGNGFtTmpLM05qUTBKbWRqQTBUREJWYWxaUmRqRndjVzFHVFRGNmJsZHpkRk5hY1VwU2VUa3pNSEpTWm5kUVZXSk1NVGRIYnpJNVVtaGthMk0wY1doRVNHNVBiMnBtVXpGbmIyZ3daMlZqTWxoWkwzQldORmg2UTJkUGRFOHlRMEowVVRBaUxDSmhiSFJ6WldOcFpDSTZJakU2YkdsMlpTNWpiMjA2TURBd016ZEdSa1ZETkRRMU1EQTFPQ0lzSW1GdGNpSTZXeUp3ZDJRaVhTd2lZWEJ3WDJScGMzQnNZWGx1WVcxbElqb2lVMFpVVUVkdklpd2lZWEJ3YVdRaU9pSTVOR1ZqTnpBNE5DMHpNbVE1TFRRNE9XSXRPR1F6TXkwek1UWTJObUpqWWpKallqRWlMQ0poY0hCcFpHRmpjaUk2SWpFaUxDSmxiV0ZwYkNJNklteHBjM1J6UUhOMmNtbHVabTl5YldGMGFXTmhMbWwwSWl3aVpXNW1jRzlzYVdSeklqcGJYU3dpWm1GdGFXeDVYMjVoYldVaU9pSlRWbElpTENKbmFYWmxibDl1WVcxbElqb2lUR2x6ZEhNaUxDSnBaSEFpT2lKc2FYWmxMbU52YlNJc0ltbHdZV1JrY2lJNklqZ3lMamcwTGpVMUxqRXhOaUlzSW01aGJXVWlPaUpzYVhOMGN5SXNJbTlwWkNJNklqY3hNVEF4Wm1VMExUUmxPR1V0TkdGaE55MDVNR1JrTFdZM05EYzNNVEF4WldRMVlTSXNJbkIxYVdRaU9pSXhNREF6TWpBd01rRTVNMEU0TmtJNElpd2ljbWdpT2lJd0xrRlZORUZPVDFKblpqRnBhR1F3YVUxQmNFTXRTbEpETlhSUlNVRkJRVUZCUVZCRlVIcG5RVUZCUVVGQlFVRkNUMEZLWnk0aUxDSnpZM0FpT2lKVFRWUlFMbE5sYm1RaUxDSnphV1FpT2lJM09XVm1aalk1WlMxaVpEazBMVFE1WXpZdFlqYzVOaTAzWVRrM09EWmhObVpqWVRFaUxDSnpkV0lpT2lJMlN6aFZka3RLTlZodWRHZ3hjbVpUU0d0NVVGRlhhVkJGVDI1MFJIa3RaSHBDZVc5TmJVbG9TR2xWSWl3aWRHbGtJam9pTjJZMk1HVTBNelF0WVRFMU9DMDBPRGMzTFRoak1ESXRPVEJpWlRJMU1UQmlPV0kxSWl3aWRXNXBjWFZsWDI1aGJXVWlPaUpzYVhabExtTnZiU05zYVhOMGMwQnpkbkpwYm1admNtMWhkR2xqWVM1cGRDSXNJblZ3YmlJNklteHBjM1J6WDNOMmNtbHVabTl5YldGMGFXTmhMbWwwSTBWWVZDTkFjMlowY0dkdkxtOXViV2xqY205emIyWjBMbU52YlNJc0luVjBhU0k2SW5sdVNERndTMlIyU2pCeFJsWlVSbDk0WWxJMlFVRWlMQ0oyWlhJaU9pSXhMakFpTENKM2FXUnpJanBiSWpFelltUXhZemN5TFRabU5HRXROR1JqWmkwNU9EVm1MVEU0WkROaU9EQm1NakE0WVNKZGZRLmgxLUp4SUVGQ3cyZmFyM1VJcnFsT0xYcm0xT2pJOUY4YjRkZGVZbkJaQkpqZ2lKby01QWs0TElYbTB2TVhRTmUySUNObl9INnRaNUtsQzNUcGc2RDBPdWpHYTZfOWpxdlFBb1VjbFc0cnFXWWM0WjhyNkZvWWctdDh6WTlhNjZhR2JzYXcxRm9mTk02Q0hDVlpaNHlGSTh1YTVvLU9taVcwUE5PSXdralhiazZiclpIdEttclhJNnhPa1NDNl94ZnB3WmFqai1aN2FuVmFRbDdORFNXX1l3a2dfdm1zUm5pNHMwS2JLSWJ3TlVqajE0WnhRaFVSd3pxR2ZHekxfRmhERXJZTEc1YzNGUEQyVGZ3S1U3NWh0aWZ0R1dxOHA0X1o5bkdHMTdFMVphS3VkOXZHaEEtektPNUZIZ0YzUVlxeHdlWExvWlZzeERJaWdGLXp4bllRUQEB
2023/05/29 11:17:15 DEBUG: C <-- S: 535 5.7.3 Authentication unsuccessful [MR1P264CA0133.FRAP264.PROD.OUTLOOK.COM 2023-05-29T09:17:15.530Z 08DB5FF35BA9DFD7]
2023/05/29 11:17:15 DEBUG: C --> S: *
2023/05/29 11:17:20 DEBUG: C <-- S: 500 5.3.3 Unrecognized command '*' [MR1P264CA0133.FRAP264.PROD.OUTLOOK.COM 2023-05-29T09:17:20.561Z 08DB5FF35BA9DFD7]
2023/05/29 11:17:20 DEBUG: C --> S: QUIT
2023/05/29 11:17:20 DEBUG: C <-- S: 221 2.0.0 Service closing transmission channel

also note the 5 seconds delay after the * command (unrelated to this PR I think)

[james-d-elliott]

[connection begins]
C: C01 CAPABILITY
S: * CAPABILITY … AUTH=XOAUTH2
S: C01 OK Completed
C: A01 AUTHENTICATE XOAUTH2 dXNlcj1zb21ldXNlckBleGFtcGxlLmNvbQFhdXRoPUJlYXJlciB5YTI5LnZGOWRmdDRxbVRjMk52YjNSbGNrQmhkSFJoZG1semRHRXVZMjl0Q2cBAQ==
S: A01 OK AUTHENTICATE completed.

This reads to me like the server (S) sends * CAPABILITY .. AUTH=XOAUTH2, but you're sending the * in the logs. Looks to me the client only sends one important command. Maybe it's the same as Google?

[james-d-elliott]

This is probably more relevant (pretty sure on closer inspection it is indeed the same):

https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth#smtp-protocol-exchange

AUTH XOAUTH2 <base64 string in XOAUTH2 format>

[connection begins]
C: auth xoauth2
S: 334
C: dXNlcj1zb21ldXNlckBleGFtcGxlLmNvbQFhdXRoPUJlY
XJlciB5YTI5LnZGOWRmdDRxbVRjMk52YjNSbGNrQmhkSFJoZG1semRHRXVZMj
l0Q2cBAQ==
S: 235 2.7.0 Authentication successful
[connection continues...]

[drakkan]

It works! I missed a permission while registering my app

2023/05/29 11:59:14 DEBUG: C <-- S: 250 ZR2P278CA0019.outlook.office365.com Hello [82.84.55.116]
SIZE 157286400
PIPELINING
DSN
ENHANCEDSTATUSCODES
STARTTLS
8BITMIME
BINARYMIME
CHUNKING
SMTPUTF8
2023/05/29 11:59:14 DEBUG: C --> S: STARTTLS
2023/05/29 11:59:14 DEBUG: C <-- S: 220 2.0.0 SMTP server ready
2023/05/29 11:59:14 DEBUG: C --> S: EHLO p1
2023/05/29 11:59:14 DEBUG: C <-- S: 250 ZR2P278CA0019.outlook.office365.com Hello [82.84.55.116]
SIZE 157286400
PIPELINING
DSN
ENHANCEDSTATUSCODES
AUTH LOGIN XOAUTH2
8BITMIME
BINARYMIME
CHUNKING
SMTPUTF8
2023/05/29 11:59:14 DEBUG: C --> S: AUTH XOAUTH2
2023/05/29 11:59:14 DEBUG: C <-- S: 334 
2023/05/29 11:59:14 DEBUG: C --> S: dXNlcj1kcmFra2FuMTAwMEBvdXRsb29rLmNvbQFhdXRoPUJlYXJlciBFd0JBQStsM0JBQVVzaG5sS1VhY3lRZFFLT3FjVUdSMDBlR0xkbndBQWF3b3BJenZGeFVDK1Bqc0E3Q2dLaGZLSFFsUmZVTXdzRTY4VTA5L2NZU2d0M1gycjNKUUFpbmNSZGpaL1JScmtoSi9PL2JudkI5anN6S1h3UFFyTHlFNldseU0xMDZpNWVRRElMV0ZvaGQ3M3hsZmpVSEt5S1hBS0pmQ1ZRelpZeE16ak9BY0JZbkdDK0tOQXBCOWFEWkRuTW9HUFZRRXVrektVVXZCQXl6UHl3TDN6b00vMmlybDhHS3ZOcXNxTjdjOG5kV1JGcjQ1ajFNNkYzM2FQeXlFRUEvbCtTWC9CQklGTm9md0lYTXVGTDBkelEwaEdBTGFCR1oyU3RuazdMYXF2TnBHNTE0R0sxZ0ZQTFZkdTNzdE5iekZUeDZjMmtGTERycEQvazAxTWN2SE1hSTN0a0xxblREWnIwMjJ1bTBNQzZkV3pIdXl5NnhidTNHSy9tc0RaZ0FBQ04rRktIdWgwU2tDRUFLYWhiV3RnUlFnT1F4WVBKbWFmUFlvLzlVOHBpVS9WNm4zejg1dHJvYjR6bjlPUHcyRlllclRqWmlKSndrSlNKUzIrOE1WNEFyeElNWXJ0YjFPT0hZdW9WMmJpeEtPc1RnMC9JMXF5VFZDblVCZ01qNWVuRGRPVi9RYlJKU2wvZW5LN1JyTVBjNGZmNnlVanNBeEdUdENWV0daazY0L01BUlc1Tzc2MVQwY0REMUYyVkYwQktWVVl2WlBZajlxM3dKLzlwVzRQZUxCSEhOUlZQMFI0WGdXa1orRk14WDhlNlVIVlMxTGJKZWEzN3hSTzZDK2NnU3A1MEJZMkFBbW9HUHVYdjlHT3htSjFlL2tLNDU4VXlPRXpZM0t0Y0RKLy8rT2dCTXZiU3MxMDh0M3dMWE5OWWp4K3FlMVVSM0ZxSHBBd3FTd0dSaWJlcGVuZUFZS09GTUZIdGp0dHNJK1IvYVNzaldiMENhVm9rQyt0TmNrb043cTcrbUhsYzdXK0FuV3hNSFkrTzU3aGF3T1hHZ2hxTk91OGdEVkovSTZqNlRRVmwvNS9LclE5cXpaZVlHeXJ3cUxwSHV0Q2NJbGZrV3RyL0VhNnNlWDEzMmUwM2V0dXdJbFZaYVU0ZkJlV3RPSFlCSnVPcVlUSkZUQjhtd0Yxa1NPazYzMEl5SXZaR0dTdTdoWCswVjByZEtrMktlbVJnQk5PelZ0cXBiWVpwckw0QU9ZZjljVmIycSt4TlhOQVRGaXlQTWJZVElKSzZjZ3IzenBQWFFhQWwvMVdzbHFURWcxRk4ralNMeGhJWHRwdEFkSEN5MmVxV2pKbkVjcVMyc2VoRlR4eGlzVENYUkpqM1NsU3ljRDBDMmZOUHMyQzJnZ0RYRnJ6WFdTYmhHK3dpTUlRWUhXV2hDaGgwdGZyRm9MR2RGY0haRVh3RkZGQWc9PQEB
2023/05/29 11:59:14 DEBUG: C <-- S: 235 2.7.0 Authentication successful
2023/05/29 11:59:14 DEBUG: C --> S: MAIL FROM:<[email protected]> BODY=8BITMIME SMTPUTF8
2023/05/29 11:59:15 DEBUG: C <-- S: 250 2.1.0 Sender OK
2023/05/29 11:59:15 DEBUG: C --> S: RCPT TO:<[email protected]>
2023/05/29 11:59:15 DEBUG: C <-- S: 250 2.1.5 Recipient OK
2023/05/29 11:59:15 DEBUG: C --> S: DATA
2023/05/29 11:59:15 DEBUG: C <-- S: 354 Start mail input; end with <CRLF>.<CRLF>
2023/05/29 11:59:15 DEBUG: C --> S: RSET
2023/05/29 11:59:15 DEBUG: C <-- S: 250 2.0.0 Resetting
2023/05/29 11:59:15 DEBUG: C --> S: QUIT
2023/05/29 11:59:15 DEBUG: C <-- S: 221 2.0.0 Service closing transmission channel

[james-d-elliott]

Does it happen to work with the default implementation too? I suspect it may

[drakkan]

Does it happen to work with the default implementation too? I suspect it may

yes it works

2023/05/29 12:06:08 DEBUG: C <-- S: 250 MR1P264CA0147.outlook.office365.com Hello [82.84.55.116]
SIZE 157286400
PIPELINING
DSN
ENHANCEDSTATUSCODES
AUTH LOGIN XOAUTH2
8BITMIME
BINARYMIME
CHUNKING
SMTPUTF8
2023/05/29 12:06:08 DEBUG: C --> S: AUTH XOAUTH2 dXNlcj1kcmFra2FuMTAwMEBvdXRsb29rLmNvbQFhdXRoPUJlYXJlciBFd0JBQStsM0JBQVVzaG5sS1VhY3lRZFFLT3FjVUdSMDBlR0xkbndBQVhBa21WQ1F5My9odTQvN04yZVlRWmNwM3pidVRpTmFyTXJnSHhzd2k0K3Zvak9DbkFDQjhZSnllWndpY09aUDVQUHE1cUlFTGh2R1V0dHExV0VmT2tJU2I3aFV4U29mVy9YUjBPdkdQQkhxYVVPbHNLWktCb2VaYWFCc0gydXdGK1g4TlF0Yng2NWJQSkN5TTNyZUNGOFBwSE9KZ1p5eEZZTkxIYzZybjlyZUZuSUdVU0FhY2kzMzNPMEdnbnRlem52ZGhITHY5WU9DQ3VDdmk0cjU3NnJJWUw2bFdDUlJ5S3RYY1hnTnZqOUtFQmFzYjJ6L1RETGZxRnRqbC9yN3IvTGVYYTdUN0lxdlk5c3U1ZmVUQXlhM3RpZUdSZkVHZ3o0eUFMRVlMUFdJWkE3OVVkWDNQa0t0WUQzZXFLNmxPazVuQlQyL2JCQVZxNWx3S1NWWkFvSURaZ0FBQ0piMWxJRThDK05iRUFKbnZEZlM3NE94UUpUcXZvK29WZWVIWXFpWnd4dmQ3aGpnZEJzYjZNWXJvTU5NNXhWYk5yRGMvTElzYWcrTGhLNktZYm5JbjNOZmFlOFllN3VneVJsWXRhS3VJaHNQN0dvdU5lQ0VsMWlDaHlMYnp5bWwwN0VQUkVRRFFVaGZhVkFudVptRUsxUTJXM3RNS2oyNWdvWFEzS0RDWVRpUlFBaW5iVThFN2xtN3YxMnU2YjdHSVZlTC9oY3NuQ2JmWUhPWkVneWo4REVoMERBdHNwa0hpdTh5NUg2LzdreFFRYmpIQmFidU11aGVHUGtON0dYQkdqTm9WblJyQU5IWjFTaXdGRytzUUZ3TlRLZk5idXZqL0N2a0hTMXF4Vk1EL05XTWh1UGp5SFVkWkE1VllPS2NxYVZnNnBrR2t5SkdQOHFpcHMwOUtGbzFkeXA0ZlhMUm8zdHpWZFR6TmFhblk5WlRQK3FkeVJSS0xvV0lhZHBJNHkza096Qk4rSjJZdWcrV2JGVWxQMHlIdTVtQklhS0pFU3lXOU5uRnF3QTRSYmRhSDlBdWd6bkQ2NW9RNFc4Q3pPOWs3RTA1YTlUbnpYdkRMVzJKOUcxbFR3ZXFpYUxHT2VTcTZyTGZ1QVNURzVkay9YNW9EaGMvS1FWU1Zja1ZnTkNZTEVxcUFRamYwSVJBRisralY5Z09uYmQ3SHhrUkUrZFYzN0dRRi96YlR2YWRzbmVnalY2bmlnbEZvT28zcEg2U1FDWU96NXpobTgxSWZId2EvdjdoMzhzRm1vODNWRDVEdlg4alJDVVBHaWd2NWMvMVBtWVkrU2NVejdncFBMVXJ1NkZET2x0K3dVa09Kd2xOcEpnYm12enUvaStSSlg5MXFOamxxZEV3MU01L1NyVmlHZzhid29BRDZHRTE3K2h6dUppZ0VGTUFtcFJGQWc9PQEB
2023/05/29 12:06:09 DEBUG: C <-- S: 235 2.7.0 Authentication successful
2023/05/29 12:06:09 DEBUG: C --> S: MAIL FROM:<[email protected]> BODY=8BITMIME SMTPUTF8
2023/05/29 12:06:09 DEBUG: C <-- S: 250 2.1.0 Sender OK
2023/05/29 12:06:09 DEBUG: C --> S: RCPT TO:<[email protected]>
2023/05/29 12:06:09 DEBUG: C <-- S: 250 2.1.5 Recipient OK
2023/05/29 12:06:09 DEBUG: C --> S: DATA
2023/05/29 12:06:09 DEBUG: C <-- S: 354 Start mail input; end with <CRLF>.<CRLF>
2023/05/29 12:06:10 DEBUG: C --> S: RSET
2023/05/29 12:06:10 DEBUG: C <-- S: 250 2.0.0 Resetting
2023/05/29 12:06:10 DEBUG: C --> S: QUIT
2023/05/29 12:06:10 DEBUG: C <-- S: 221 2.0.0 Service closing transmission channel

[drakkan]

and here is the output for an auth error

2023/05/29 12:11:35 DEBUG: C <-- S: 250 ZR0P278CA0163.outlook.office365.com Hello [82.84.55.116]
SIZE 157286400
PIPELINING
DSN
ENHANCEDSTATUSCODES
AUTH LOGIN XOAUTH2
8BITMIME
BINARYMIME
CHUNKING
SMTPUTF8
2023/05/29 12:11:35 DEBUG: C --> S: AUTH XOAUTH2 dXNlcj1kcmFra2FuMTAwMEBvdXRsb29rLmNvbQFhdXRoPUJlYXJlciAxMTExMTExMTExMTExMTExMTEBAQ==
2023/05/29 12:11:42 DEBUG: C <-- S: 535 5.7.3 Authentication unsuccessful [ZR0P278CA0163.CHEP278.PROD.OUTLOOK.COM 2023-05-29T10:11:42.622Z 08DB5F9615773452]
2023/05/29 12:11:42 DEBUG: C --> S: *
2023/05/29 12:11:47 DEBUG: C <-- S: 500 5.3.3 Unrecognized command '*' [ZR0P278CA0163.CHEP278.PROD.OUTLOOK.COM 2023-05-29T10:11:47.652Z 08DB5F9615773452]
2023/05/29 12:11:47 DEBUG: C --> S: QUIT

[james-d-elliott]

Nice work! Probably want to delete those credentials. Looks like you can probably just remove variants all together for now probably?

[wneessen]

Great work @drakkan and thanks as always for the helpful input as well @james-d-elliott! If you give me a hint on how get the tokens created, I can set up a github secret for the GH test environment, so that they don't fail.

[drakkan]

Nice work! Probably want to delete those credentials. Looks like you can probably just remove variants all together for now probably?

yes, no need for variants, I'll update the PR later.
Outputs with Google

2023/05/29 12:28:14 DEBUG: C <-- S: 250 smtp.gmail.com at your service, [82.84.55.116]
SIZE 35882577
8BITMIME
AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
ENHANCEDSTATUSCODES
PIPELINING
CHUNKING
SMTPUTF8
2023/05/29 12:28:14 DEBUG: C --> S: AUTH XOAUTH2 dXNlcj1tYXJ6aWFlbGlhNzhAZ21haWwuY29tAWF1dGg9QmVhcmVyIHlhMjkuYTBBV1k3Q2tuTmJRX1YwYm5WVVl2dW5XZlBjV1BBVG1fTHF2elZyYUVrNDNWQVJMSmtzQUNqQVE2cF9sV1ZUUzJMN2liVEJIUXRCU0VxRXZSVDdUV2FkeS15U3l0cGh6Vk1rQ3J3M2pINWtucndyUE1Fd3VzYUtDN2h3bV9CaGlFOU4xd2stdHA3Wm42UzE2Q051czVXMk5XU3ROX1N6X3B3YUNnWUtBYzRTQVJNU0ZRRzF0RHJwTlhBTWdsWDBMS3VUdDAwbFdEM0pLQTAxNjcBAQ==
2023/05/29 12:28:14 DEBUG: C <-- S: 235 2.7.0 Accepted
2023/05/29 12:28:14 DEBUG: C --> S: MAIL FROM:<[email protected]> BODY=8BITMIME SMTPUTF8
2023/05/29 12:28:14 DEBUG: C <-- S: 250 2.1.0 OK j17-20020a170906279100b00969f44bbef3sm5794725ejc.11 - gsmtp
2023/05/29 12:28:14 DEBUG: C --> S: RCPT TO:<[email protected]>
2023/05/29 12:28:14 DEBUG: C <-- S: 250 2.1.5 OK j17-20020a170906279100b00969f44bbef3sm5794725ejc.11 - gsmtp
2023/05/29 12:28:14 DEBUG: C --> S: DATA
2023/05/29 12:28:15 DEBUG: C <-- S: 354  Go ahead j17-20020a170906279100b00969f44bbef3sm5794725ejc.11 - gsmtp
2023/05/29 12:28:15 DEBUG: C --> S: RSET
2023/05/29 12:28:15 DEBUG: C <-- S: 250 2.1.5 Flushed j17-20020a170906279100b00969f44bbef3sm5794725ejc.11 - gsmtp
2023/05/29 12:28:15 DEBUG: C --> S: QUIT
2023/05/29 12:28:15 DEBUG: C <-- S: 221 2.0.0 closing connection j17-20020a170906279100b00969f44bbef3sm5794725ejc.11 - gsmtp

Bad token

2023/05/29 12:30:38 DEBUG: C <-- S: 250 smtp.gmail.com at your service, [82.84.55.116]
SIZE 35882577
8BITMIME
STARTTLS
ENHANCEDSTATUSCODES
PIPELINING
CHUNKING
SMTPUTF8
2023/05/29 12:30:38 DEBUG: C --> S: STARTTLS
2023/05/29 12:30:38 DEBUG: C <-- S: 220 2.0.0 Ready to start TLS
2023/05/29 12:30:38 DEBUG: C --> S: EHLO p1
2023/05/29 12:30:38 DEBUG: C <-- S: 250 smtp.gmail.com at your service, [82.84.55.116]
SIZE 35882577
8BITMIME
AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
ENHANCEDSTATUSCODES
PIPELINING
CHUNKING
SMTPUTF8
2023/05/29 12:30:38 DEBUG: C --> S: AUTH XOAUTH2 dXNlcj1tYXJ6aWFlbGlhNzhAZ21haWwuY29tAWF1dGg9QmVhcmVyIDExMTExMTExMTExMTExAQE=
2023/05/29 12:30:38 DEBUG: C <-- S: 334 eyJzdGF0dXMiOiI0MDAiLCJzY2hlbWVzIjoiQmVhcmVyIiwic2NvcGUiOiJodHRwczovL21haWwuZ29vZ2xlLmNvbS8ifQ==
2023/05/29 12:30:38 DEBUG: C --> S: 
2023/05/29 12:30:38 DEBUG: C <-- S: 535 5.7.8 Username and Password not accepted. Learn more at
5.7.8  https://support.google.com/mail/?p=BadCredentials d7-20020a170906c20700b0096f55247570sm5710307ejz.0 - gsmtp
2023/05/29 12:30:38 DEBUG: C --> S: *
2023/05/29 12:30:38 DEBUG: C <-- S: 502 5.5.1 Unrecognized command. d7-20020a170906c20700b0096f55247570sm5710307ejz.0 - gsmtp
2023/05/29 12:30:38 DEBUG: C --> S: QUIT
2023/05/29 12:30:38 DEBUG: C <-- S: 221 2.0.0 closing connection d7-20020a170906c20700b0096f55247570sm5710307ejz.0 - gsmtp

[drakkan]

Great work @drakkan and thanks as always for the helpful input as well @james-d-elliott! If you give me a hint on how get the tokens created, I can set up a github secret for the GH test environment, so that they don't fail.

To create the tokens you need first to register your app on Google/Microsoft and then you can use the oauth2 library. To complete the oauth exchange and get a refresh token. I don't think this can be done in go-mail, you need an http server to get the auth callback. This is a one time operation, the refresh tokens never expire or expire after a long time

[drakkan]

@wneessen for Microsoft for example, I registered an app in the Azure portal like this

You need to set credentials, redirect uri, permissions etc. Not sure if you can automate this in GitHub. Using the refresh token you can create access tokens to use for sending emails

[wneessen]

Ok, that sounds more complex than expected. Not sure if the GH workflow will let us accomplish this easily.

[drakkan]

@wneessen we need to better investigate this:

2023/05/29 12:30:38 DEBUG: C <-- S: 535 5.7.8 Username and Password not accepted. Learn more at
5.7.8  https://support.google.com/mail/?p=BadCredentials d7-20020a170906c20700b0096f55247570sm5710307ejz.0 - gsmtp
2023/05/29 12:30:38 DEBUG: C --> S: *

I don't thing my PR sends the *, do you have ideas? I have no more time for now. I can take a look after my working hours

[wneessen]

2023/05/29 12:30:38 DEBUG: C <-- S: 535 5.7.8 Username and Password not accepted. Learn more at
5.7.8  https://support.google.com/mail/?p=BadCredentials d7-20020a170906c20700b0096f55247570sm5710307ejz.0 - gsmtp
2023/05/29 12:30:38 DEBUG: C --> S: *

I don't thing my PR sends the *, do you have ideas? I have no more time for now. I can take a look after my working hours

@drakkan I'm pretty sure that comes from the smtp auth method:

go-mail/smtp/smtp.go

Line 238 in 13c8d0a

_, _, _ = c.cmd(501, "*")

Reason behind this is the SMTP AUTH RFC where it states:

If the client wishes to cancel the authentication exchange, it issues a line with a single "*". If the server receives such a response, it MUST reject the AUTH command by sending a 501 reply.

I assume that the OAUTH implementations of MS and Google did not implement this behaviour. Not sure if it's better to extend Client.Auth() to catch this behaviour or maybe catch it in the OAUTH implementation instead. Since we have our own smtp client, we should be free to do either way.

[drakkan]

Ok, that sounds more complex than expected. Not sure if the GH workflow will let us accomplish this easily.

yes it is not so easy to automate, additional the first time the user must be redirected to its account and grant the required permission. Take a look here for an overview

[drakkan]

2023/05/29 12:30:38 DEBUG: C <-- S: 535 5.7.8 Username and Password not accepted. Learn more at
5.7.8  https://support.google.com/mail/?p=BadCredentials d7-20020a170906c20700b0096f55247570sm5710307ejz.0 - gsmtp
2023/05/29 12:30:38 DEBUG: C --> S: *

I don't thing my PR sends the *, do you have ideas? I have no more time for now. I can take a look after my working hours

@drakkan I'm pretty sure that comes from the smtp auth method:

go-mail/smtp/smtp.go

Line 238 in 13c8d0a

_, _, _ = c.cmd(501, "*")

Reason behind this is the SMTP AUTH RFC where it states:

If the client wishes to cancel the authentication exchange, it issues a line with a single "*". If the server receives such a response, it MUST reject the AUTH command by sending a 501 reply.

I assume that the OAUTH implementations of MS and Google did not implement this behaviour. Not sure if it's better to extend Client.Auth() to catch this behaviour or maybe catch it in the OAUTH implementation instead. Since we have our own smtp client, we should be free to do either way.

Thanks, I'll take a look later today and/or in the next few days

[wneessen]

@drakkan Is the PR ready to review/merge or are you still working on it?

[drakkan]

@drakkan Is the PR ready to review/merge or are you still working on it?

It should be ready. Both success and auth error cases seem to work as expected

[wneessen]

Perfect. I'll review and get a new release ready in the next days.

[drakkan]

@drakkan Is the PR ready to review/merge or are you still working on it?

It should be ready. Both success and auth error cases seem to work as expected

Thank you.
No hurry for the new release. I think I need about 2 weeks to integrate the oauth part into SFTPGo (get refresh token, UI etc) and it's not a problem for me to use an untagged version or even temporary replace go-mail with my branch.
If you prefer to have more real tests before tagging a new release, you can wait a few weeks after the feature is released in SFTPGo.

[wneessen]

That's good to know. Might actually be a good idea to get some "real life" data first.

[wneessen]

Everything looks good. Ready to merge

[drakkan]

Worst case I have access to both providers enterprise offerings and I have access to the Microsoft Sponsored Azure Program.

@james-d-elliott this feature has been included in the development version of SFTPGo since last week but unfortunately I still haven't received any feedback from the user who requested it.
If you have time/motivation to test it, please contact me at nicola dot murino at gmail dot com. Thanks in advance

[wneessen]

@drakkan Were you able to get in some more real-life testing from your project? Do you think we are ready for an official release?

[drakkan]

@drakkan Were you able to get in some more real-life testing from your project? Do you think we are ready for an official release?

not yet, sorry. I'll tag SFTPGo v2.5.2 with this feature included the next weekend anyway.

[james-d-elliott]

I did email you on the 9th, figured you were busy because I didn't see a response.

[james-d-elliott]

I have sent the email, you can remove them from this post. I mistyped "gmail" incidentally it looks like.

[drakkan]

I have sent the email, you can remove them from this post. I mistyped "gmail" incidentally it looks like.

sent some instructions via email, check your spam folder if you dont't see any reply 😄 Thank you!!!

[james-d-elliott]

I see them, will take a look this weekend.

[drakkan]

I see them, will take a look this weekend.

no hurry, thank you!