rmax/scrapy-redis

Issues

• Stats ValueError: invalid literal for int() with base 10: b'1739222295.11384'

  #303 opened 2 months ago by lexxai
  0

• Question: How to remove specific url from dupefilter set on redis?

  #302 opened 4 months ago by matheus-rossi
  5

• 日志报警：String request is deprecated

  #265 opened 2 years ago by maintain99
  1

• AttributeError: 'BossSpider' object has no attribute 'settings'

  #301 opened 6 months ago by gaye746560359
  0

• Add Kafka Topic Integration to Scrapy Redis

  #288 opened 8 months ago by hamed21q
  1

• Is there a way to stop spider check duplicate with redis ?

  #242 opened 3 years ago by milkeasd
  7

• `scrapy_redis.scheduler.Scheduler` not compatible with `scrapy.dupefilters.BaseDupeFilter`

  #293 opened 9 months ago by HairlessVillager
  0

• [Question] Fetch request url from redis fail

  #285 opened 2 years ago by KokoTa
  8

• There is a compatibility issue with the latest scrapy-redis package of the new version of scrapy

  #289 opened a year ago by georgeJzzz
  13

• Scrapy-redis usage related issues

  #291 opened a year ago by Hao1617
  7

• [QUESTION] Is there a way to use response.meta ?

  #290 opened a year ago by SteliosGiannatos
  1

• Add metadata to URLs to retrieve from Redis

  #287 opened a year ago by tteguayco
  2

• How does the CrawlSpider work?

  #284 opened 2 years ago by bch80
  0

• Add Type annotations pep-0483

  #283 opened 2 years ago by LuckyPigeon
  0

• [dev] Add Type annotations

  #282 opened 2 years ago by LuckyPigeon
  0

• [RFC]A new journey

  #203 opened 3 years ago by whg517
  14

• 警告： Passing a 'spider' argument to ExecutionEngine.crawl is deprecated

  #266 opened 2 years ago by maintain99
  4

• 你好，大佬，请问一个问题，我用scrapy-redis执行爬虫的时候，设置最大并发是20，但是怎么感觉像创建20个队列一样，上一个20执行完之后，下一个20再执行，大佬，您能为我解答一下吗，感激不尽

  #263 opened 2 years ago by zhanghongdou
  2

• make_request_from_data implementation in RedisMixin

  #267 opened 2 years ago by rjbks
  2

• Cleanup requirements

  #268 opened 2 years ago by devfox-se
  1

• Scrapy 2.8.0 deprecated function scrapy.utils.request.request_fingerprint() warning

  #272 opened 2 years ago by ThanatosDi
  1

• Is there a planned support for Python3.11?

  #273 opened 2 years ago by lion137
  5

• why I can not see request record in redis

  #262 opened 2 years ago by minikiller
  2

• Playwright?

  #271 opened 2 years ago by martmalo
  2

• Push json will failed

  #238 opened 3 years ago by songhao8080
  1

• How to use scrapy-redis if I'm using start_requests() instead of start_urls in my spider?

  #264 opened 2 years ago by ashwinbala99
  3

• [docs] Move docs to github Wiki

  #224 opened 2 years ago by LuckyPigeon
  0

• [IMPROVEMENT] Add template to issues and PRs

  #213 opened 2 years ago by LuckyPigeon
  0

• Redis data persistence

  #248 opened 3 years ago by MRPTY1
  4

• Scrapy 2.6.1 Unsupport function make_requests_from_url to use

  #250 opened 3 years ago by ihandmine
  5

• error object has no attribute 'make_requests_from_url'

  #257 opened 3 years ago by zhajingwen
  2

• [spiders] remove duplicate check setting types

  #260 opened 3 years ago by LuckyPigeon
  0

• Update basic settings

  #226 opened 3 years ago by LuckyPigeon
  0

• [style] Fix pylint style

  #245 opened 3 years ago by LuckyPigeon
  0

• [docs] Chinese documentation

  #243 opened 3 years ago by LuckyPigeon
  0

• ImportError: cannot import name 'Iterable' from 'collections'

  #241 opened 3 years ago by milkeasd
  2

• [IMPROVEMENT] Update support for json data

  #216 opened 3 years ago by LuckyPigeon
  1

• [IMPROVEMENT] Refactor max_idle_time as an attribute

  #210 opened 3 years ago by LuckyPigeon
  0

• [BUG] MySpider' object has no attribute 'settings'

  #217 opened 3 years ago by LuckyPigeon
  0

• [FEATURE] Finish TODO tasks

  #219 opened 3 years ago by LuckyPigeon
  0

• [IMPROVEMENT] Update collections import method before deprecated

  #211 opened 3 years ago by LuckyPigeon
  0

• [IMPROVEMENT] Update pytest config and related doc

  #212 opened 3 years ago by LuckyPigeon
  2

• [IMPROVEMENT] Flake8 update for files

  #207 opened 3 years ago by LuckyPigeon
  0

• .

  #205 opened 3 years ago by wj4832
  0

• how can i get sorted url in scrapy_redis zset? and how can spider get url by scrapy_redis's zset？

  #198 opened 3 years ago by FSNH
  2

• WRONGTYPE Operation against a key holding the wrong kind of value

  #192 opened 3 years ago by JQ-K
  2

• redis stats

  #196 opened 4 years ago by nieweiming
  9

• redis stats

  #195 opened 4 years ago by nieweiming
  0

• Queues defined in queue.py not seem to be used anywhere?

  #194 opened 4 years ago by abhinavkulkarni
  0

• Crawl urls not from corresponding redis key

  #185 opened 4 years ago by nghuyong
  0