/scrapy-cookies

A middleware of cookies persistence for Scrapy

Primary LanguagePythonBSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

Scrapy Cookies

PyPI PyPI - Python Version Travis branch PyPI - Wheel Codecov branch

Overview

This middleware enable Scrapy manage, save and restore cookies in various ways. With this middleware Scrapy can easily re-use cookies which saved before or in multiple spiders, and share cookies between spiders, even in spider-cluster.

Requirements

  • Python 2.7 or Python 3.4+
  • Works on Linux, Windows, Mac OSX, BSD

Installation

The quick way:

pip install scrapy-cookies

For more details see the installation section in the documentation: https://scrapy-cookies.readthedocs.io/en/latest/intro/installation.html

Documentation

Documentation is available online at https://scrapy-cookies.readthedocs.io/en/latest/ and in the docs directory.

Releases

You can find release notes at https://scrapy-cookies.readthedocs.io/en/latest/news.html

Community (blog, twitter, mail list, IRC)

Keeping this section same as Scrapy is intending to benefit back to Scrapy.

See https://scrapy.org/community/

Contributing

Keeping this section same as Scrapy is intending to be easier when this repo merge back to Scrapy.

See https://doc.scrapy.org/en/master/contributing.html

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

Keeping this section same as Scrapy is intending to benefit back to Scrapy.

See https://scrapy.org/companies/

Commercial Support

Keeping this section same as Scrapy is intending to benefit back to Scrapy.

See https://scrapy.org/support/

TODO

  • [X] Remove the support lower than python 3.6
  • [ ] Use JSON1 extension in sqlite backend
  • [ ] Update backend arguments calling way
  • [ ] Replace pymongo with txmongo in MongoDB backend
  • [ ] Replace redis sync driver with async driver in Redis backend
  • [ ] Change LICENSE to GPLv3
  • [ ] Use versioneer for version management