For this projects I utilized the open source library Scrapy to create a spider to scrape information from Walmart, Bestbuy, and Newegg. I had a functioning proxy attached from ScrapeOps to ensure my spider wasn't blocked. The goal for this spider was to crawl the sites to gather information on specific computer components that I wanted to choose by having it ask for an input before the spider started crawling.
I plan on continuing to update this spider to become more efficiant.