Pinned Repositories
scrapy-deltafetch
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
scrapy-djangoitem
Scrapy extension to write scraped items using Django models
scrapy-jsonrpc
Scrapy extension to control spiders using JSON-RPC
scrapy-jsonschema
Scrapy schema validation pipeline and Item builder using JSON Schema
scrapy-magicfields
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
scrapy-monkeylearn
A Scrapy pipeline to categorize items using MonkeyLearn
scrapy-playwright
🎭 Playwright integration for Scrapy
scrapy-splash
Scrapy+Splash for JavaScript integration
scrapy-zyte-api
Zyte API integration for Scrapy
scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
Scrapy Plugins's Repositories
scrapy-plugins/scrapy-splash
Scrapy+Splash for JavaScript integration
scrapy-plugins/scrapy-playwright
🎭 Playwright integration for Scrapy
scrapy-plugins/scrapy-djangoitem
Scrapy extension to write scraped items using Django models
scrapy-plugins/scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
scrapy-plugins/scrapy-jsonrpc
Scrapy extension to control spiders using JSON-RPC
scrapy-plugins/scrapy-deltafetch
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
scrapy-plugins/scrapy-magicfields
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
scrapy-plugins/scrapy-jsonschema
Scrapy schema validation pipeline and Item builder using JSON Schema
scrapy-plugins/scrapy-monkeylearn
A Scrapy pipeline to categorize items using MonkeyLearn
scrapy-plugins/scrapy-zyte-api
Zyte API integration for Scrapy
scrapy-plugins/scrapy-headless
scrapy-plugins/scrapy-pagestorage
A scrapy extension to store requests and responses information in storage service
scrapy-plugins/scrapy-querycleaner
Scrapy spider middleware to clean up query parameters in request URLs
scrapy-plugins/scrapy-splitvariants
Scrapy spider middleware to split an item into multiple items using a multi-valued key
scrapy-plugins/scrapy-streaming
scrapy-plugins/scrapy-dotpersistence
A scrapy extension to sync `.scrapy` folder to an S3 bucket
scrapy-plugins/scrapy-crawlera-fetch
Scrapy Downloader Middleware for Crawlera Fetch API
scrapy-plugins/scrapy-feedexporter-sftp
scrapy-plugins/scrapy-statsd
scrapy-plugins/scrapy-hcf
Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs
scrapy-plugins/scrapy-snowflake-stage-exporter
Snowflake database loading utility with Scrapy integration
scrapy-plugins/scrapy-spider-metadata
scrapy-plugins/scrapy-feedexporter-google-drive
scrapy-plugins/scrapy-feedexporter-azure-storage
scrapy-plugins/scrapy-feedexporter-dropbox
Scrapy feed exporter for Dropbox
scrapy-plugins/scrapy-feedexporter-google-sheets
scrapy-plugins/scrapy-feedexporter-onedrive
Export to OneDrive
scrapy-plugins/scrapy-incremental
scrapy-plugins/scrapy-settings-log
An extension that allows a user to display all or some of their scrapy spider settings at runtime.
scrapy-plugins/zyte-spidermon