felipecsl/wombat

Unable to run example

Closed this issue · 7 comments

blackheaven commented

Hi,

I really want to try your work, so I copy&paste this code:

require 'wombat'

puts Wombat.crawl do
  base_url "http://www.github.com"
  path "/"

  headline "xpath=//h1"
  what_is "css=.column.secondary p", :html
  repositories "css=a.repo", :list

  explore "xpath=//ul/li[2]/a" do |e|
    e.gsub(/Explore/, "LOVE")
  end

  benefits do
    first_benefit "css=.column.leftmost h3"
    second_benefit "css=.column.leftmid h3"
    third_benefit "css=.column.rightmid h3"
    fourth_benefit "css=.column.rightmost h3"
  end
end

But it fails:

/home/black/.rvm/gems/ruby-1.9.3-p362/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:595:in `resolve': absolute URL needed (not "") (ArgumentError)
        from /home/black/.rvm/gems/ruby-1.9.3-p362/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in `fetch'
        from /home/black/.rvm/gems/ruby-1.9.3-p362/gems/mechanize-2.5.1/lib/mechanize.rb:407:in `get'
        from /home/black/.rvm/gems/ruby-1.9.3-p362/gems/wombat-2.1.0/lib/wombat/processing/parser.rb:41:in `parser_for'
        from /home/black/.rvm/gems/ruby-1.9.3-p362/gems/wombat-2.1.0/lib/wombat/processing/parser.rb:29:in `parse'
        from /home/black/.rvm/gems/ruby-1.9.3-p362/gems/wombat-2.1.0/lib/wombat/crawler.rb:30:in `crawl'
        from /home/black/.rvm/gems/ruby-1.9.3-p362/gems/wombat-2.1.0/lib/wombat.rb:10:in `crawl'
        from amazon.rb:3:in `<main>'

I have the following gems:

activesupport (3.2.11)
blankslate (2.1.2.4)
bluecloth (2.2.0)
bundler (1.2.3)
capistrano (2.14.1)
curb (0.8.3)
domain_name (0.5.7)
gli (2.5.3)
highline (1.6.15)
i18n (0.6.1)
json (1.7.6)
mechanize (2.5.1)
mg (0.0.8)
mime-types (1.19)
multi_json (1.5.0)
net-http-digest_auth (1.2.1)
net-http-persistent (2.8)
net-scp (1.0.4)
net-sftp (2.0.5)
net-ssh (2.6.3)
net-ssh-gateway (1.1.0)
nokogiri (1.5.6)
ntlm-http (0.1.1)
parslet (1.5.0)
pdfkit (0.5.2)
rack (1.4.2)
rack-protection (1.3.2)
rake (10.0.3)
rest-client (1.6.7)
rmagick (2.13.1)
rubygems-bundler (1.1.0)
rvm (1.11.3.5)
showoff (0.7.0)
sinatra (1.3.3)
tilt (1.3.3)
unf (0.0.5)
unf_ext (0.0.5)
webrobots (0.0.13)
wombat (2.1.0)

Have you got an idea of the problem?

For your help,
Thanks by advance.

felipecsl commented

@blackheaven if you take out the puts from the start, it works.
Also, please notice that the GitHub initial page changed, so this example is not valid anymore.
I will update the README with an updated working example.

Thanks!

blackheaven commented

Ok, but, how can I get the result to deal with it?

Thanks by advance.

felipecsl commented

The same. Just remove the puts from the start:

    require 'wombat'

    puts Wombat.crawl do
      base_url "http://www.github.com"
      path "/"

      headline "xpath=//h1"
      what_is "css=.column.secondary p", :html
      repositories "css=a.repo", :list

      explore "xpath=//ul/li[2]/a" do |e|
        e.gsub(/Explore/, "LOVE")
      end

      benefits do
        first_benefit "css=.column.leftmost h3"
        second_benefit "css=.column.leftmid h3"
        third_benefit "css=.column.rightmid h3"
        fourth_benefit "css=.column.rightmost h3"
      end
    end

This outputs in my irb:

{"headline"=>"Build software better, together.",
 "what_is"=>nil,
 "repositories"=>[],
 "explore"=>"Search",
 "benefits"=>
  {"first_benefit"=>nil,
   "second_benefit"=>nil,
   "third_benefit"=>nil,
   "fourth_benefit"=>nil}}
blackheaven commented

Is there a way to use it without irb?

felipecsl commented

Hey @blackheaven sorry for the delay in the response. I hope you were able to solve your problems.
I used irb as just a tool for simple testing of short snippets of code, but you don't need it to use wombat or any other ruby gem. Just write a text file (for example with the .rb extension), and then run it like ruby yourfile.rb for example. That is the same thing.
Hope it helps!

blackheaven commented

In fact I was using RVM and when I was puting a = before Wombat.crawl it failed, I remove RVM and it works now !
Don't worry, thanks for your help.

tralston commented

@felipecsl I think @blackheaven was right in this problem. I just installed the gem and ran the example from pry (or irb) and it showed a result. But when I put the example verbatim in an example file (example.rb) and ran it, no output. This is because irb will return the last statement's return value. When run from the command line though, it won't print anything out. I tried to put the puts at the beginning, but got the same error as mentioned above. The fix I found was to say:

result = Wombat.crawl do
...
...
end
puts result

Only then will it output it to the shell.

  • 👍4
  • 🎉1