This is a simple extension that helps you run essential Scrapy commands such as running your Scrapy spiders directly on VS Code.
The extension adds 2 commands to VS Code's Command Palette. Both commands behave similarly but trigger a different shell function. When either commands are ran, it finds the name of the spider in the currently active file and runs a specific, user-defined, shell function (details in the "requirements" section below) with the spider's name.
Commands:
Scrapy: Run Spider
- runs thespider-run
shell script. Used for running a spider.Scrapy: Test Spider
- runs thespider-test
shell script. Used for testing a spider (if applicable).
The current version of the extension requires a few things in order to work properly:
- Single spider class per file.
- The spider's class
name
field should be defined as early as possible, best if right under theclass
definition. - The extension (currently) relies on the existence of specific shell functions to run. If there's demand, this may change in the future. The shell functions should be defined in your VS Code's default shell's startup file (
.bash_profile / .bashrc / .zshrc / etc..
).spider-run
: should first clear the Scrapy logs & output files (if applicable) and then run thescrapy crawl
command with the passed argument (the spider name). Example:spider-run() { echo "" > logs.log echo "" > output.json scrapy crawl $1 -o output.json }
spider-test
: should first clear the Scrapy logs & test output files (if applicable) and then run thescrapy crawl
command with the passed argument (the spider name). Example:spider-test() { echo "" > logs.log echo "" > test_output_log.log python path/to/spider_tester_script.py $1 -o test_output_log.log }
See CHANGELOG.md.