Link Crawler This is the basic crawler which scraps all links from the website, saves them if they match criteria and continues to do same with the new ones.