/crawler

Primary LanguageJavaScript

crawler for cars.com

- crawl from base link
- use set depth
- catalog all urls found on a page.
	- place urls into queue
		- record parent link and target
		id, url, parent
        -  phantomjs to save screenshot
        resize image
        - jsdom to find modules

Adding storage branch to work on persistence

ilkajsdflksdjf Codeship Status for darrellpratt/crawler