crawl: error: Invalid -s value, use -s NAME=VALUE

[saavedra29]

When scrapyd runs a spider with scrapy==2.8.0 no log files are created inside the logs folder (logs_dir). Using scrapy==2.7.1 works perfectly.

[imaxwen]

+1

[jpmckinney]

Please upgrade to Scrapyd 1.4.0.

[imaxwen]

Using scrapy==2.8.0 and scrapyd==1.4.0 still got error , and logs not created in the logs_dir.

scrapyd-01  | 2023-02-10T12:07:39+0800 [-] Process started:  project='my-spiders' spider='test-spider' job='74738e4ea8f811ed9b1502420a110703' pid=13 log='logs/my-spiders/test-spider/74738e4ea8f811ed9b1502420a110703.log' items='file:///scrapyd/items/my-spiders/test-spider/74738e4ea8f811ed9b1502420a110703.jl'
scrapyd-01  | 2023-02-10T12:07:40+0800 [Launcher,13/stderr] Usage
scrapyd-01  | 	=====
scrapyd-01  | 	  scrapy crawl [options] <spider>
scrapyd-01  | 	crawl: error: Invalid -s value, use -s NAME=VALUE

[imaxwen]

@jpmckinney Please have a look, consider reopen this issue?

[jpmckinney]

crawl: error: Invalid -s value, use -s NAME=VALUE

That message is unusual. Can you edit your copy of scrapy/commands/__init__.py, and change the line raise UsageError("Invalid -s value, use -s NAME=VALUE", print_help=False), to raise UsageError(repr(opts.set)) ?

[jpmckinney]

@saavedra29 Please provide your scrapyd log (like in #473 (comment)). Otherwise, this is impossible to debug. It was tested and it works in normal conditions.

Edit: Nevermind - if downgrading to Scrapy 2.7.1 fixed your issue, then you must not be using Scrapyd 1.4.0 yet.

[jpmckinney]

@imaxwen Alternatively, install this testing branch:

pip install -e git+https://github.com/scrapy/scrapyd.git@launcher-log#egg=scrapyd

[jpmckinney]

Aha, it is because the FEEDS argument wasn't properly encoded. 1.4.1 is released.

[imaxwen]

thanks Very efficient 😆