zenithght/ebot

Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details

EBOT (http://www.redaelli.org/matteo-blog/projects/ebot/)
USAGE: see the wiki homepage (http://wiki.github.com/matteoredaelli/ebot/) and ebot_test.erl for details
LICENSE: GPL v3+, see file LICENSE
Copyright (c) 2010, 2011 - Matteo Redaelli at libero dot it

Tested with Linux (Ubuntu and Debian) and Erlang R13B03 and R14A

*******************************************************************************
* DOWNLOAD "TAG" releases and not the latest sources that could be unstable...
*******************************************************************************