mwaqarsikandar/Scrapy_Projects_01

USED SCRAPY FOR PYTHON WEB SCRAPING

Used Parse Method Fantasy_19

For website http://books.toscrape.com/catalogue/category/books/fantasy_19/

scrapy crawl fant -o o.csv

Used Parse Method Sequential-art_5

For website http://books.toscrape.com/catalogue/category/books/sequential-art_5/

scrapy crawl sart -o o.csv

Used Parse Method Thriller_37

For website http://books.toscrape.com/catalogue/category/books/thriller_37/index.html

scrapy crawl spider -o o.csv

Used Rules And Dictionary

for whole website http://books.toscrape.com/

scrapy crawl crawl -o o.csv

Used Rules And Items

for whole website https://www.sainsburys.co.uk/shop/gb/groceries'

scrapy crawl sains -o o.csv

Code For Rules

rules = [Rule(LinkExtractor(allow='groceries/'),
                  callback='parse_filter_book', follow=True)]

Code For Items

    def parse_filter_book(self, response):
        exists = response.xpath('//div[@class="pdp"]').extract_first()
        if exists:
            title = response.xpath('//div[@class="productTitleDescriptionContainer"]/h1/text()').extract_first()
            book = SainsburyItem() # New line
            book['title'] = title

            yield book
          
        else:
            print(response.url)