amerigo
Amerigo is an individual website crawler. It is named after Amerigo Vespucci, the 15th century Italian explorer and cartographer.
It crawls a website to output the pages and their relationships. It is designed to take a the website graph and output the serialization as it crawls. This output is then fed to tools which may use the data as they please.