Body being parsed as nil

Question

Body being parsed as nil

Closed this issue 8 years ago · 6 comments

Hi,

I've followed the setup in the blogs/readme but I seem to be having trouble getting Griddler to parse my emails:

Gem versions:

griddler (1.3.1)
griddler-sendgrid (0.0.1)

config/initializers/griddler.rb

require 'email_processor'

Griddler.configure do |config|
  config.processor_class = EmailProcessor # CommentViaEmail
  config.processor_method = :process # :create_comment (A method on CommentViaEmail)
  config.reply_delimiter = '-- REPLY ABOVE THIS LINE --'
  config.email_service = :sendgrid # :cloudmailin, :postmark, :mandrill, :mailgun
end

routes.rb

post '/email_processor' => 'griddler/emails#create'

end result (output of raise @email.inspect within EmailProcessor):

RuntimeError: #<Griddler::Email:0x007f5cc90e82b0 @params={"dkim"=>"{@example-co-uk.20150623.gappssmtp.com : pass}", "email"=>"Received: by mx0033p1mdw1.sendgrid.net with SMTP id c4tNs72Aee Tue, 20 Sep 2016 10:45:09 +0000 (UTC)\nReceived: from mail-wm0-f46.google.com (mail-wm0-f46.google.com [74.125.82.46]) by mx0033p1mdw1.sendgrid.net (Postfix) with ESMTPS id 3A82A781CB4 for <382@replies.example.com>; Tue, 20 Sep 2016 10:45:09 +0000 (UTC)\nReceived: by mail-wm0-f46.google.com with SMTP id l132so27591794wmf.1 for <382@replies.example.com>; Tue, 20 Sep 2016 03:45:07 -0700 (PDT)\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example-co-uk.20150623.gappssmtp.com; s=20150623; h=date:from:to:message-id:in-reply-to:references:subject:mime-version; bh=gh2deNT6LbUiDgo6wrJ7hN2uE6juqZ+pYffSO2eQ/G4=; b=dI5CtUPv+JklAqQ1ej1XFGdju2nt78vfe4PNfgyQMe42T9p7Bp3bqVBxRPljUvLqZA 8AaOQ44/nup6zJUlXdBRQhjLnMI+5gI26wObZlRFUkW9kVpsH+NjuPw4aoJr+nkL2OBk gjvP+E0tMZO6K/xFHlOuTVZFJ7Az6S+k7hjl2vcqG+yrCMseMww7PltzWyeu+r23P20M ZXPeuPy+dPANM4E9WrUorDxy4sLoC7NXWwM2gijzCmzVRDE26XZvwOlYgjtMlrAJmbzB FxRTxuawxqqvl15KXCDMclQUk8AWNwabmyY91h4NDDXgBaZ3gslyVRXNI58nYvhLx/UX vH/A==\nX-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version; bh=gh2deNT6LbUiDgo6wrJ7hN2uE6juqZ+pYffSO2eQ/G4=; b=BhHB11XYN0rIk1B/nr5SZFbUCtjwa1aAyYCl2BwqYDgT/4jnDPRB+za2kCDdKKsNtg gCS25IRlOkntyuytKQYj1YlPlqatCH5Zd1HLDyaZVJ0KM48CiuBph9XqUEektb6GifOg qdLwIqF6S0GAv+NHPKfWtfY/tbN5e+whS0H9X3PJih4FP8N8oNVaIwIwGrRhOd2S524+ B1L3VPAygTB1kKs1wxVb23MnnYrk+xxapd3IJRVqorucWIfJ06DgjoXFR7ZiTYihIs0k 0v/T1AZKGeycU6tJReXzn0cIe7ox3QR+8exAYWLsd7UQr92RsQRq4OG5IGeBqLV7cNO+ tKLw==\nX-Gm-Message-State: AE9vXwORQK9HMDJV6sgYRBIgmWlR9jEQ6rc1YeocSRZ5fF8esk8Rbb+vgu8l0R0suthCuQ==\nX-Received: by 10.28.187.70 with SMTP id l67mr2856736wmf.82.1474368306666; Tue, 20 Sep 2016 03:45:06 -0700 (PDT)\nReceived: from local.mail ([217.38.84.252]) by smtp.gmail.com with ESMTPSA id uw3sm27784261wjb.21.2016.09.20.03.45.06 for <382@replies.exampe.com> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Sep 2016 03:45:06 -0700 (PDT)\nDate: Tue, 20 Sep 2016 11:45:00 +0100\nFrom: Sam Cambridge <sam@example.co.uk>\nTo: 382@replies.example.com\nMessage-ID: <etPan.57e11331.1d683901.2cc9@example.co.uk>\nIn-Reply-To: <CANV7_b-1NZ8-cFd_PkGXKAyW_FY8NSb3QXZvWVFdWEHqKYys3w@mail.gmail.com>\nReferences: <57e110994e704_19953fc1f147bca433350@example.io.mail> <CANV7_b-1NZ8-cFd_PkGXKAyW_FY8NSb3QXZvWVFdWEHqKYys3w@mail.gmail.com>\nSubject: Re: Fwd: (Comments Test) Re: Testing comment system\nX-Mailer: Airmail (367)\nMIME-Version: 1.0\nContent-Type: multipart/alternative; boundary=\"57e11331_40bef2e3_2cc9\"\n\n--57e11331_40bef2e3_2cc9\nContent-Type: text/plain; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\nTesty mctest\n\n--=C2=A0\nSam Cambridge\nSent with Airmail\n\nOn 20 September 2016 at 11:43:31, Paul (paul@example.co.uk) wrote:\n\n\n---------- Forwarded message ----------\nFrom: <382@replies.example.com>\nDate: 20 September 2016 at 11:34\nSubject: (Comments Test) Re: Testing comment system\nTo: paul@example.co.uk\n\n\n-- REPLY ABOVE THIS LINE --\n\n(Comments Test) Re: Testing comment system\n\n10:34am 20/09/2016\n\nsam@example.co.uk commented on this item:\n\nstarting over.\n\nReply to this comment\n\n\n\n\n--\n--\n\nPaul\n\n=C2=A0\n\n--57e11331_40bef2e3_2cc9\nContent-Type: text/html; charset=\"utf-8\"\nContent-Transfer-Encoding: quoted-printable\nContent-Disposition: inline\n\n<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style>=\n</head><body style=3D\"word-wrap: break-word; -webkit-nbsp-mode: space; -web=\nkit-line-break: after-white-space;\"><div id=3D\"bloop_customfont\" style=3D\"f=\nont-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: =\n0px; line-height: auto;\">Testy mctest</div> <br> <div id=3D\"bloop_sign_1474=\n368285637946112\" class=3D\"bloop_sign\"><div style=3D\"font-family:helvetica,a=\nrial;font-size:13px\">--&nbsp;<br>Sam Cambridge<br>Sent with Airmail</div></=\ndiv> <br><p class=3D\"airmail_on\">On 20 September 2016 at 11:43:31, Paul 
 (<a href=3D\"mailto:paul@example.co.uk\">paul@example.co.uk</a>) wrote:</p>=\n <blockquote type=3D\"cite\" class=3D\"clean_bq\"><span><div><div></div><div>\n\n\n<title></title>\n\n\n<div dir=3D\"ltr\"><br>\n<div class=3D\"gmail_quote\">---------- Forwarded message\n----------<br>\nFrom: <span dir=3D\"ltr\">&lt;<a href=3D\"mailto:382@replies.example.io\">382=\n@replies.example.io</a>&gt;</span><br>\n\nDate: 20 September 2016 at 11:34<br>\nSubject: (Comments Test) Re: Testing comment system<br>\nTo: <a href=3D\"mailto:paul@example.co.uk\">paul@example.co.uk</a><br>\n<br>\n<br>\n<div style=3D\"font-family:Helvetica,Arial,sans-serif\">\n<div style=3D\"background:#fff;padding:20px\">\n<div>\n<p>-- REPLY ABOVE THIS LINE --</p>\n<h3>(Comments Test) Re: Testing comment system</h3>\n<p style=3D\"font-size:14px\">10:34am 20/09/2016</p>\n<p style=3D\"display:block;min-height:20px\"><img style=3D\"width:20px;min-hei=\nght:20px;max-height:20px;margin-right:5px\" src=3D\"http://gravatar.com/avata=\nr/82d0567ef38996eb934f5a3f768bea81\" alt=3D\"Gravatar\" width=3D\"80\" height=3D=\n\"80\"><strong><a href=3D\"mailto:sam@example.co.uk\" target=3D\"_blank\">sam@example.co.uk</a></strong> commented on this <a href=3D\"https://u2704591.ct.send=\ngrid.net/wf/click?upn=3DeOimKyuinAz9DAvIyW5D7f41SJ9aAjA2eSo472DKnHtC2Q5amlj=\nrI0nLWmHcUHmgH9PU27lcky0LYuAmzehMTw-3D-3D_US5yf2kX9aFp-2BReNjwZn1wyukcdQXft=\nK4YmtndkjZK1ByVaQ4XNtfwcUQnjevpsDMJy8G4i7v53rv3QklwK6mrftjjcI-2FZld8-2B-2FJ=\nIAz9orm05nUSnr9xfwtcGrnnaqd-2Bu4mpweo1dMKyd8iYFTcv3bKBRaFKaD7SCHWjWzegsUgyV=\n-2Ba-2BMW7o2gTLhf4bML4nyZ10EXg6YXinZVzpIKYPjA-3D-3D\" target=3D\"_blank\">item=\n</a>:</p>\n<hr style=3D\"background:#eee;min-height:1px;outline:none;border:none;margin=\n-bottom:20px\">\n\n<div style=3D\"font-size:18px;color:#333\">\n<p>starting over.</p>\n</div>\n<a style=3D\"color:#fff;background-color:#5cb85c;border-color:#4cae4c;displa=\ny:inline-block;padding:6px 12px;margin-bottom:0;font-size:14px;font-weight:=\n400;line-height:1.42857143;text-align:center;white-space:nowrap;vertical-al=\nign:middle;background-image:none;border:1px solid transparent;border-radius=\n:4px;text-decoration:none;margin:20px 0px\" href=3D\"https://u2704591.ct.send=\ngrid.net/wf/click?upn=3DTpXmbSM-2B3ASa2k6LNluNOJ00zBQXQ0C3Ik-2BmaeeBPtH8Jl0=\nrRHVd73r44GB-2BjDmbdLfBVIiUaF-2BA67sRhjCkSw-3D-3D_US5yf2kX9aFp-2BReNjwZn1wy=\nukcdQXftK4YmtndkjZK1ByVaQ4XNtfwcUQnjevpsDSnck5pKWcz25h2px4a4D2Cz2c5KelLKSl7=\nsjZbAD07CvHJreK7XbZe0bSuQFIMc-2Bpy75QN0ppppisPzL6GYVjI-2FzhzFLbJ9d-2FsEMgBX=\nyf-2BEhJ2-2FDvRkNBcKQYj4PuzFDYHFGFPwAQFmrs0yPWlMLJQ-3D-3D\" target=3D\"_blank=\n\">Reply to this comment</a></div>\n<img style=3D\"width:150px;display:block;margin:30px 0 20px 0\" src=3D\"http:/=\n/example.io/assets/example-b5f327e71153bc893d3540f7ed9e0efa.png\" alt=3D=\n\"example\"></div>\n<img src=3D\"https://u2704591.ct.sendgrid.net/wf/open?upn=3DUS5yf2kX9aFp-2BR=\neNjwZn1wyukcdQXftK4YmtndkjZK1ByVaQ4XNtfwcUQnjevpsDhu41tUelBZi-2FdFuwEV9ltV7=\nNWUIMPyXhO-2FHTe99dbD8IQMarwVm0lz-2FZIk5BFoPzqJhwNLlG-2BT3q6fuzpE7NjSV61kED=\nY57kHK1rxTYJ82e8gJvxLL-2FYZtpNahVqGXcoqhFx7WOJPL5yXfqcEDd4ZaDB5vhqXk6DSQsk5=\nitGKGY-3D\" alt=3D\"\" width=3D\"1\" height=3D\"1\" border=3D\"0\" style=3D\"min-heig=\nht: 1px !important; width: 1px !important; border-width: 0px !important; ma=\nrgin: 0px !important; padding: 0px !important; display: none !important;\"><=\n/div>\n</div>\n<br>\n<br clear=3D\"all\">\n<div><br></div>\n--<br>\n<div class=3D\"gmail_signature\" data-smartmail=3D\"gmail_signature\">\n<div dir=3D\"ltr\">\n<div>\n<div dir=3D\"ltr\">\n<div>\n<div dir=3D\"ltr\">\n<div dir=3D\"ltr\">\n<div dir=3D\"ltr\">\n<div dir=3D\"ltr\">\n<div dir=3D\"ltr\">\n<div dir=3D\"ltr\">\n<div dir=3D\"ltr\">\n<div>--</div>\n<div><br></div>\n<div>Paul</div></div></span></blockquote></body></html>=\n\n--57e11331_40bef2e3_2cc9--\n\r\n", "to"=>["382@replies.example.io"], "from"=>"Sam Cambridge <sam@example.co.uk>", "sender_ip"=>"74.125.82.46", "spam_report"=>"Spam detection software, running on the system \"mx0033p1mdw1.sendgrid.net\", has\nidentified this incoming email as possible spam. The original message\nhas been attached to this so you can view it (if it isn't spam) or label\nsimilar future email. If you have any questions, see\n@@CONTACT_ADDRESS@@ for details.\n\nContent preview: Testy mctest --\xC2 Sam Cambridge Sent with Airmail On 20 September\n 2016 at 11:43:31, Paul (paul@example.co.uk) wrote: [...] \n\nContent analysis details: (0.0 points, 5.0 required)\n\n pts rule name description\n---- ---------------------- --------------------------------------------------\n 0.0 HTML_MESSAGE BODY: HTML included in message\n 0.0 T_MIME_NO_TEXT No text body parts\n\n", "envelope"=>"{\"to\":[\"382@replies.example.io\"],\"from\":\"sam@example.co.uk\"}", "subject"=>"Re: Fwd: (Comments Test) Re: Testing comment system", "spam_score"=>"0.011", "charsets"=>"{\"to\":\"UTF-8\",\"subject\":\"UTF-8\",\"from\":\"UTF-8\"}", "SPF"=>"permerror", "controller"=>"griddler/emails", "action"=>"create", "cc"=>[], "attachments"=>[]}, @to=[{:token=>"382", :host=>"replies.example.io", :email=>"382@replies.example.io", :full=>"382@replies.example.io", :name=>nil}], @from={:token=>"sam", :host=>"example.co.uk", :email=>"sam@example.co.uk", :full=>"Sam Cambridge <sam@example.co.uk>", :name=>"Sam Cambridge"}, @subject="Re: Fwd: (Comments Test) Re: Testing comment system", @body="", @raw_text=nil, @raw_html=nil, @raw_body=nil, @headers={}, @cc=[], @bcc=[], @raw_headers=nil, @attachments=[]>

As you can see I'm expecting to see the parsed body with Testy mctest in it.

Any ideas on what I'm doing wrong?

Answer 1 · 2016-09-20T14:32:01.000Z

Can you give it a try using the master branch of griddler-sendgrid?

Also, is there a reason you copied all of the default settings into the initializer/routes file? You only need to put that stuff in there if you are overriding the default Griddler engine's settings.

Answer 2 · 2016-09-20T14:58:01.000Z

no reason, just so any other devs can see what's what I guess. I'll try with the master branch now.

EDIT

unfortunately that didn't help.

Answer 3 · 2016-09-20T16:40:41.000Z

Alright, so taking a look at your inspect dump, it appears the text and/or the html keys from SendGrid are not being posted to your server (eg they are missing from the params hash). Looking at the params, SendGrid flagged your email as possible spam (see spam_report). I am not familiar with the finer points of SendGrid's parse API, but my guess would be they are flagging your email as spam and not sending the body along with the rest of the email payload.

Answer 4 · 2016-09-22T14:21:42.000Z

I got to the bottom of this - I had POST the raw, full MIME message in my send grid settings, apparently thats bad.

Answer 5 · 2016-10-14T10:41:39.000Z

I have similar problem but with some email applications such as Outlook. Receiving email from Outlook just doesn't populate raw_html or body I have feeling that Outlook is not sending html content/type mail but rather text.
Do you have some advices how to handle these cases ?

Answer 6 · 2016-10-14T16:46:20.000Z

@dixpac you should be able to take a look at what SendGrid is POSTing to your controller. If Rails is not getting a text or html parameter POSTed to it from SendGrid then there's nothing to parse (and your raw_html and body attributes will be empty).