istresearch/scrapy-cluster

Inter-spider communication

Closed this issue 7 years ago · 1 comments

shenbakeshkishore commented 7 years ago

Hi,

Please share sample code to perform inter-spider communication.(In reference to https://scrapy-cluster.readthedocs.io/en/latest/topics/crawler/controlling.html#inter-spider-communication)

madisonb commented 7 years ago

If you read that section it gives you an example, but here is a more complete one:

req = Request(some_url, callback=self.parse)
req.meta['spiderid'] = 'myotherspidername'
yield req

Closing.