Google Reports Double Attached
Closed this issue · 4 comments
So for unexplainable reasons, Google is sending DMARC reports to me as a double embedded attachment. The DMARC email I receive contains an attachment with another email that has a Zip file attached that contains the XML DMARC report.
The dmarcts-report-parser.pl fails to understand what to do with this and drops the email in the not processed folder.
Setup:
Office 365 IMAP using Modern Authentication (with this fix applied).
Script and Database Running on Ubuntu 20.04 LTS
Perl 5 v30
MSG: IMAP message with UID #224
Subject: Report domain: igstesting.com Submitter: google.com Report-ID:
10496786790271017863
MimeType: multipart/mixed
This is a multipart attachment
Skipped an unknown attachment (text/plain)
Skipped an unknown attachment (message/rfc822)
dmarcts-report-parser.pl: Subject: Report domain: igstesting.com Submitter: google.com Report-ID:
10496786790271017863
: at ./dmarcts-report-parser.pl line 706.
dmarcts-report-parser.pl: Could not find an embedded ZIP!
dmarcts-report-parser.pl: The IMAP message with UID #224 does not seem to contain a valid DMARC report. Skipped.
Moving (copy and delete) IMAP message file to IMAP folder: Inbox.notProcessed
Sending: 8 STATUS Inbox.notProcessed (MESSAGES)
Sent 40 bytes
Read: * STATUS Inbox.notProcessed (MESSAGES 23)
8 OK STATUS completed.
Sending: 9 UID COPY 224 Inbox.notProcessed
Sent 35 bytes
Read: 9 OK [COPYUID 290 224 26] COPY completed.
Sending: 10 UID STORE 224 +FLAGS.SILENT (\Deleted)
Sent 43 bytes
Read: 10 OK STORE completed.
Sending: 11 EXPUNGE
Sent 12 bytes
Read: * 1 EXPUNGE
* 0 EXISTS
Read: 11 OK EXPUNGE completed.
Sending: 12 CLOSE
Sent 10 bytes
Read: 12 OK CLOSE completed.
Sending: 13 LOGOUT
Sent 11 bytes
Read: * BYE Microsoft Exchange Server IMAP4 server signing off.
13 OK LOGOUT completed.
dmarcts-report-parser.pl: Processed 1 emails.
I've zipped and attached the message if anyone else is curious. Report domain_ igstesting_com Submitter_ google_com Report-ID_ 10496786790271017863.zip
As a workaround, I'm brushing off my perl skill (yes, just one) and looking to handle these messages specifically. Essentially I'd add a subroutine that will handle the rfc822 attachment type and then keep unpacking them until it finds a zip or xml file and then drop back in to the existing code flow.
Posting this here in case anyone has seen this and already fixed this (yep, I'm that lazy) or has a better way to approach this. Or you can point out the part of the error I missed that tells me what I need to fix. (it's been one of those days, apparently).
Thanks in advance!!
--Chad
Here it is in something more human-readable. Thankfully, there are converter programs for that.
Looks like it's something that Exchange is doing, wrapping the incoming message as an attachment and adding the words "External Sender". Easiest thing to do might be to turn that off for that address, if possible. With the MIME type being used for the attached message, there would be no way for the parser to tell what format it's in, which as it turns out is not MBOX format.
just noticed in the log that IMAP clients apparently do get the attachment in MBOX format. Microsoft's on-the-fly format conversion at work...
Hm. I should be able to get the External Sender flagged turned off for this account. I hadn't considered it being that since messages from other domains (microsoft and yahoo specifically for my current testing scenario) are attaching correctly with that flag in place.
If it is indeed that simple, I will gladly choose that route over writing my own perl. I am more of an engineer than I am a programmer.
Thanks for converting the headers into an easier format. I had reviewed and then discarded the headers as helpful, but I see the difference between Google's DMARC reports and the other reports is that the 'External Sender' tag was made part of the body seamlessly. I think it's the HTML assumption built into that tagging.
I'll make the change today and see if that resolves it for reports that come in overnight tonight.
Thanks for the quick assist and great find!
Following up on this, turns out it was that simple and removing the External Sender banner solved the entire problem.
Thanks again for spotting that.