marco Highly scalable distributed web crawler built with Ruby, MongoDB, Redis, RabbitMQ, Docker Swarm and Containers. Architecture
