Spinneret is a web spider and dynamic load generator for testing, benchmarking, and warming caches.
git clone git@github.com:steder/spinneret.git cd spinneret mkdir -p ~/.virtualenvs virtualenv ~/.virtualenvs/spinneret python setup.py develop
spinneret spider --base_url http://localhost:8000
Because this can take a while I've included a sample sitemap.yaml that can be used to immediately generate traffic.
spinneret traffic --base_url http://localhost:8000 --inflight 100