Scrapy daemon is not developed as a daemon that allows authentication for access. Therefore, to protect your daemon, you need to serve scrapyd through a web server like nginx. Bypassing the request to scrapyd internally and returning the output to nginx.
In this setup, nginx is working on port 80 and scrapyd in port 6800; externally, you will need to map only the port 80 as is mentioned in the docker-compose.yml file.
logparser is needed if you are using ScrapydWeb for managing your projects. It is included and running in the docker instance, communicating directly with scrapyd.
You must setup your .env file accordingly to what your spider is requesting. For authentication purposes only, use:
USERNAME=USERNAME
PASSWORD=PASSWORDAfter the setup you can reference in the docker-compose.yml file.
You can find the image scrapyd-auth in dockerhub.
If you have any problems with or questions about this image, please file an issue on the GitHub repository.
Pull requests welcome :-)