web-scraperd To run system: skaffold dev To run spiders add starting urls and schedule jobs Examples: lpush crichd:start_urls https://hd.crichd.com/crichd-home curl http://localhost:6800/schedule.json -d project=scraperapp -d spider=crichd Commands for troubleshooting scraperd scraperd kubectl port-forward --namespace default service/scrd-web-scraperd 6800:6800 curl http://localhost:6800/listprojects.json curl http://localhost:6800/listjobs.json?project=scraperapp curl http://localhost:6800/listspiders.json?project=scraperapp curl http://localhost:6800/daemonstatus.json Redis kubectl port-forward --namespace default service/redis-master 6379:6379 docker run --rm --name redis-commander -p 8081:8081 --env REDIS_HOST=host.docker.internal --env REDIS_PASSWORD=redispassword rediscommander/redis-commander:latest auth Splash kubectl port-forward --namespace default service/sps-web-splash 8050:8050