drichert/sep

Separates strings, parses punctuation and spacing

Sep

Separates strings, parses punctuation and spacing

Installation

Add this line to your application's Gemfile:

gem "sep"

And then execute:

$ bundle

Or install it yourself as:

$ gem install sep

Usage

text = '   "I doubt, therefore I think, therefore I am."'

# Separate text
sep = Sep.load(text)

sep.words_data
#=> [
#=>   { punc_pre: '"', word: "I", punc_post: "", space_post: " " },
#=>   { punc_pre: "", word: "doubt", punc_post: ",", space_post: " " },
#=>   { punc_pre: "", word: "therefore", punc_post: "", space_post: " " },
#=>   { punc_pre: "", word: "I", punc_post: "", space_post: " " },
#=>   { punc_pre: "", word: "think", punc_post: "," space_post: " " },
#=>   { punc_pre: "", word: "therefore", punc_post: "", space_post: " " },
#=>   { punc_pre: "", word: "I", punc_post: "", space_post: " " },
#=>   { punc_pre: "", word: "am", punc_post: '."', space_post: "" }
#=> ]

sep.text #=> '"I doubt, therefore I think, therefore I am."'
sep.words #=> ["I", "doubt", "therefore", "I", "think", "therefore", "I", "am"]
sep.leading_space #=> "   "
sep.space #=> ["   ", " ", " ", " ", " ", " ", " ", " ", ""]

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

Copyright

Copyright (c) 2013 Dan Richert. See LICENSE for details.