/rb-sisimai

Mail Analyzing Interface for email bounce: A Ruby library to parse RFC5322 bounce mails and generating structured data as JSON from parsed results. Ruby version of Sisimai: an error mail analyzer.

Primary LanguageRubyOtherNOASSERTION

License Coverage Status Build Status Codacy Badge Ruby Gem Version

What is Sisimai

Sisimai is a Ruby library for analyzing RFC5322 bounce emails and generating structured data from parsed results. The Ruby version of Sisimai is ported from the Perl version of Sisimai at github.com/sisimai/p5-sisimai.

Key Features

  • Convert Bounce Mails to Structured Data
    • Supported formats are Ruby(Hash, Array) and JSON(String)
  • Easy to Install, Use.
    • gem install
    • git clone & make
  • High Precision of Analysis
    • 2 times higher than bounceHammer
    • Support 68 MTAs/MDAs/ESPs
    • Support Feedback Loop Message(ARF)
    • Can detect 29 error reasons

Command line demo

The following screen shows a demonstration of Sisimai at the command line using Ruby(rb-sisimai) and Perl(p5-sisimai) version of Sisimai.

Setting Up Sisimai

System requirements

More details about system requirements are available at Sisimai | Getting Started page.

Install

From RubyGems

$ sudo gem install sisimai
Fetching: sisimai-4.25.5.gem (100%)
Successfully installed sisimai-4.25.5
Parsing documentation for sisimai-4.25.5
Installing ri documentation for sisimai-4.25.5
Done installing documentation for sisimai after 6 seconds
1 gem installed

From GitHub

$ cd /usr/local/src
$ git clone https://github.com/sisimai/rb-sisimai.git
$ cd ./rb-sisimai
$ sudo make depend install-from-local
gem install bundle rake rspec coveralls
...
4 gems installed
bundle exec rake install
sisimai 4.25.5 built to pkg/sisimai-4.25.5.gem.
sisimai (4.25.5) installed.

Usage

Basic usage

make() method provides feature for getting parsed data from bounced email messages like following. Beginning with v4.25.6, new accessor origin which keeps the path to email file as a data source is available.

#! /usr/bin/env ruby
require 'sisimai'
v = Sisimai.make('/path/to/mbox')       # or path to Maildir/

# Beginning with v4.23.0, both make() and dump() method of Sisimai class can
# read bounce messages from variable instead of a path to mailbox
f = File.open('/path/to/mbox', 'r');    # or path to Maildir/
v = Sisimai.make(f.read)

# If you want to get bounce records which reason is "delivered", set "delivered"
# option to make() method like the following:
v = Sisimai.make('/path/to/mbox', delivered: true)

if v.is_a? Array
  v.each do |e|
    puts e.class                # Sisimai::Data
    puts e.recipient.class      # Sisimai::Address
    puts e.timestamp.class      # Sisimai::Time

    puts e.addresser.address    # shironeko@example.org # From
    puts e.recipient.address    # kijitora@example.jp   # To
    puts e.recipient.host       # example.jp
    puts e.deliverystatus       # 5.1.1
    puts e.replycode            # 550
    puts e.reason               # userunknown
    puts e.origin               # /var/spool/bounce/Maildir/new/1740074341.eml

    h = e.damn                  # Convert to HASH
    j = e.dump('json')          # Convert to JSON string
    puts e.dump('json')         # JSON formatted bounce data
  end
end

Convert to JSON

Sisimai.dump() method provides feature for getting parsed data as JSON string from bounced email messages like following.

# Get JSON string from parsed mailbox or Maildir/
puts Sisimai.dump('/path/to/mbox')  # or path to Maildir/

# dump() method also accepts "delivered" option like the following code:
puts Sisimai.dump('/path/to/mbox', delivered: true)

Callback feature

Beginning with Sisimai 4.19.0, make() and dump() methods of Sisimai accept a Lambda (Proc object) in hook argument for setting a callback method and getting the results generated by the method via Sisimai::Data.catch method.

#! /usr/bin/env ruby
require 'sisimai'
callbackto = lambda do |v|
  r = { 'x-mailer' => '', 'queue-id' => '' }

  if cv = v['message'].match(/^X-Postfix-Queue-ID:\s*(.+)$/)
    r['queue-id'] = cv[1]
  end
  r['x-mailer'] = v['headers']['x-mailer'] || ''
  return r
end

data = Sisimai.make('/path/to/mbox', hook: callbackto)
json = Sisimai.dump('/path/to/mbox', hook: callbackto)

puts data[0].catch['x-mailer']      # Apple Mail (2.1283)

More information about the callback feature is available at Sisimai | How To Parse - Callback Page.

One-Liner

% ruby -rsisimai -e 'puts Sisimai.dump($*.shift)' /path/to/mbox

Output example

[{"catch":{"x-mailer":"","return-path":"<shironeko@mx.example.co.jp>"},"token":"cf17945938502bd876603a375f0e9517c921bbab","lhost":"localhost","rhost":"mx-s.neko.example.jp","alias":"","listid":"","reason":"hasmoved","action":"failed","origin":"set-of-emails/maildir/bsd/lhost-sendmail-22.eml","subject":"Nyaaaan","messageid":"0000000011111.fff0000000003@mx.example.co.jp","replycode":"","smtpagent":"Sendmail","softbounce":0,"smtpcommand":"DATA","destination":"example.net","senderdomain":"example.co.jp","feedbacktype":"","diagnosticcode":"450 busy - please try later 551 not our customer 503 need RCPT command [data]","diagnostictype":"SMTP","deliverystatus":"5.1.6","timezoneoffset":"+0900","addresser":"shironeko@example.co.jp","recipient":"kijitora@example.net","timestamp":1397054085}]

Sisimai Specification

Differences between Ruby version and Perl version

The following table show the differences between Ruby version of Sisimai and Perl version of Sisimai. Information about differences between Sisimai and bounceHammer are available at Sisimai | Differences page.

Features Ruby version Perl version
System requirements Ruby 2.1 - 2.6 Perl 5.10 -
JRuby 9.0.4.0-
Analytical precision ratio(2000 emails)[1] 1.00 1.00
The speed of parsing email(1000 emails) 2.22s[2] 1.35s
How to install gem install cpanm, cpm
Dependencies (Except core modules) 1 module 2 modules
LOC:Source lines of code 10400 lines 10500 lines
The number of tests(spec/,t/,xt/) directory 241000 tests 270000 tests
License BSD 2-Clause BSD 2-Clause
Support Contract provided by Developer Available Available
  1. See ./ANALYTICAL-PRECISION
  2. Xeon E5-2640 2.5GHz x 2 cores | 5000 bogomips | 1GB RAM | Ruby 2.3.4p301

Other specification of Sisimai

Contributing

Bug report

Please use the issue tracker to report any bugs.

Emails could not be parsed

Bounce mails which could not be parsed by Sisimai are saved in the repository set-of-emails/to-be-debugged-because/sisimai-cannot-parse-yet. If you have found any bounce email cannot be parsed using Sisimai, please add the email into the directory and send Pull-Request to this repository.

Other Information

Related Sites

See also

Author

@azumakuniyuki

Copyright

Copyright (C) 2015-2021 azumakuniyuki, All Rights Reserved.

License

This software is distributed under The BSD 2-Clause License.