Scrapy Plugins

Plugins for the Scrapy framework

The Internet

Pinned Repositories

scrapy-deltafetch
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
Language:Python274 13 2348
scrapy-djangoitem
Scrapy extension to write scraped items using Django models
Language:Python503 34 1487
scrapy-jsonrpc
Scrapy extension to control spiders using JSON-RPC
Language:Python300 24 1571
scrapy-jsonschema
Scrapy schema validation pipeline and Item builder using JSON Schema
Language:Python44 13 812
scrapy-magicfields
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
Language:Python56 9 17
scrapy-playwright
🎭 Playwright integration for Scrapy
Language:Python1.3k 23 260143
scrapy-spider-metadata
Language:Python5 4 43
scrapy-splash
Scrapy+Splash for JavaScript integration
Language:Python3.2k 124 256456
scrapy-zyte-api
Zyte API integration for Scrapy
Language:Python38 6 5722
scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
Language:Python365 55 4190

Scrapy Plugins's Repositories

scrapy-plugins/scrapy-streamitem
Scrapy support for working with streamcorpus Stream Items.
Language:Python11 57 04
scrapy-plugins/scrapy-bigml
Scrapy pipeline for writing items to BigML datasets
Language:Python4 5 03