postmodern/spidr

Issues

• Redirect fails silently - need debug messages

  #13 opened 14 years ago by perplexes
  4

• Any way to automatically obey robots.txt?

  #11 opened 9 years ago by cbmeeks
  3

• Spidr behaves oddly on bad host-names under Ruby 1.8.7-p249

  #6 opened 14 years ago by postmodern
  4

• Multiple cookies from same page

  #10 opened 14 years ago by ethicalhack3r
  1

• Malformed mailto error

  #12 opened 14 years ago by ethicalhack3r
  2

• undefined method `attr' for #<Nokogiri::XML::Element:0xb77423c4> (NoMethodError)

  #9 opened 14 years ago by ethicalhack3r
  2

• handle css @import spidering

  #8 opened 15 years ago by zapnap
  0

• Odd Net::HTTP error when requesting the last page.

  #7 opened 15 years ago by postmodern
  1

• Passing Headers to Net::HTTP

  #5 opened 15 years ago by evoltech
  4

• Problem building gem

  #4 opened 15 years ago by evoltech
  1

• static course specs

  #3 opened 15 years ago by zapnap
  0

• SSL and network, in general

  #2 opened 15 years ago by justfalter
  4

• URL Normalization.

  #1 opened 15 years ago by justfalter
  3