pip install .
After installing, execute the command line tool:
> linkedin_scraper input_data.csv output.csv
Use linkedin_scraper --help
to learn the additional options
If instead of a command line tool you need to use this as a library, to integrate with an existing app, check the documentation
for the ScraperController
class.
The following settings can be changed by setting them in environment variables:
LOG_LEVEL
: Set the logging level, default is INFO
LINKEDIN_SCRAPER_PROXY
: Proxy to use for google scraping (highly recommended for google scraping).
LINKEDIN_SCRAPER_GOOGLE_CONCURRENCY
: Number of concurrent Google scrape instances, default: 20
LINKEDIN_SCRAPER_LINKEDIN_CONCURRENCY
: Number of concurrent Linkedin (playwright) scrape instances, default: 10
LINKEDIN_SCRAPER_MAX_GOOGLE_RETRY
: Maximum number of times a Google scrape task should be retried. (This is to account for Bot detection, 429s)
pip install -e .
pip install mock
python -m unittest discover -v .
pip install sphinx sphinx_rtd_theme
cd docs && make clean html