[Linagora] Sent mail appears in base64

[simonelbaz]

Hi James team,

A sent email appears in base64 in Thunderbird 128.4.0

Kind regards

[Arsnael]

Hi @simonelbaz

Can you provide us with the full eml of the message if possible? Thanks !

[simonelbaz]

Here it is:

mail.zip

PS: sorry i have mistakenly closed/reopened the issue

[Arsnael]

PS: sorry i have mistakenly closed/reopened the issue

No worries :)

Was the eml like that? it is clearly incomplete. Can we get the full version of it instead? If info are sensitive send it by mail instead!

On which platform? linagora.com on your personal account?

[simonelbaz]

Getting the eml from TB, truncates it.

From Tmail web, the mail is correctly displayed and full EML available:

mail_eml_tmail.zip

[chibenwa]

Hello.

Could it be that IMAP FETCH could ommit part of the content?

We would need to further test S3BlobStoreDAO::readBytes that is exercised upon IMAP FETCH.

We could try to reproduce with and intergation test in the distributed tests as well...

(looks like a 1 time in a million concurrency bug...)

[chibenwa]

For the record related CNB defect https://smartsla.08000linux.com/requests/3828

Email written correctly in DB.

Likely EmailSubmission/set fucked up the read and did thus send partial data.

I'd thus tend to blame the blob store. Could it be that it misses bytes parts when reading data from the Object store?

Context: AESBlobStore + S3

[Arsnael]

@chibenwa : from what I can read again on smartsla ticket that you linked, was the issue the images with GMail and was the fix setting up public assets for cnb perhaps?

I don't think it's related to this issue though, as Simon seems to get a truncated base64 email on Thunderbird

[Arsnael]

@simonelbaz sorry to come back to it so late...

Checking the full eml again, and trying to navigate the logs a bit from that day, it looks like you are sending email with Thunderbird. Can you confirm this is correct?

Then there is nothing to do with JMAP and EmailSubmission/set then.

Can we ask as well to Yassin Benhaddou that you sent email to and is part of Linagora, if he can read the mail fully with Thunderbird, or if it's the same issue as you? Just to make sure

[Arsnael]

If it's thunderbird as I suspect I would tend to think something went wrong with the APPEND IMAP command.

I dont see much anything wrong around that APPEND in the logs, except that... there was an IMAP LOGOUT command issued right after. Maybe that affected something?

If can read the message vi TMail though, it would tend to mean that server-side the message has been stored successfully. Maybe Thunderbird messed up his sync here?

@simonelbaz did you try to right click on your Sent folder on Thunderbird, select properties, and on the General Information tab you will see a button called 'Repair Folder'. If you didn't can you try to click on it? That normally should rebuild the folder index and force TB to refetch the message metadat and content from the tmail server.

If after that you can read that mail normally on TB, then it means TB messed up somehow on its side when appending the message to sent mailbox. If not... we need to dig more.

[chibenwa]

@chibenwa : from what I can read again on smartsla ticket that you linked, was the issue the images with GMail and was the fix setting up public assets for cnb perhaps?

Yes unrelated

[chibenwa]

If it's thunderbird as I suspect I would tend to think something went wrong with the APPEND IMAP command.

No. If it was the case data would be stored screwed, and displayed screwed in TMail.

=> 100% it is FETCH.

[chibenwa]

If after that you can read that mail normally on TB, then it means TB messed up somehow on its side when appending the message to sent mailbox. If not... we need to dig more.

I'd like to be sure FETCH is not screwed, thus I would like the team to do exploratory tests on the topic just in case.

[Arsnael]

Yes sorry the fetch after the append, yes, didn't express myself correctly here.

The fetch could have messed up on TMail side but also TB could have messed up or maybe there was a connection issue.

Concerned fetch on the logs:

I'd like to be sure FETCH is not screwed, thus I would like the team to do exploratory tests on the topic just in case.

Well I do understand the concern and will try to check it out

[chibenwa]

We shall do size verifications in SimpleMailboxMessage

(we have the expected message size as saved in the DB as well as the actual content: we could easily checked that the two are alined and WARN if it is not the case. I could help rouling out the eventuality of an issue within the S3BlobStoreDAO)

[chibenwa]

My feeling is that it is TB side.

Let's keep this in mind and investigate if reproduced.