Trandoshan Crawler

Project is migrated to https://github.com/darkspot-org/bathyscaphe

France

Pinned Repositories

amqp
Go wrapper for streadway/amqp
Language:Go0 2 01
api
REST API used by other processes to gather data
Language:Go5 2 84
crawler
Go process used to crawl websites
Language:Go150 8 921
dashboard
Web UI to manage a Trandoshan instance
Language:TypeScript11 2 45
feeder
Oneshot process used to put an URL in the todo queue. Useful to start crawling process
Language:Go2 2 32
k8s
Kubernetes configuration file to boostrap trandoshan cluster
Language:Shell46 2 311
persister
Go process used to abstract database interaction. It consumes messages from queue and put result in database allowing asynchronous saving operation
Language:Go7 2 84
scheduler
Go orchestrator process used to schedule website parsing
Language:Go6 2 84
trandoshan-parent
Parent repository used to centralize tools to run the application, manage source code, ...
Language:Shell21 2 57

trandoshan-io/crawler
Go process used to crawl websites
Language:Go150 8 921
trandoshan-io/k8s
Kubernetes configuration file to boostrap trandoshan cluster
Language:Shell46 2 311
trandoshan-io/trandoshan-parent
Parent repository used to centralize tools to run the application, manage source code, ...
Language:Shell21 2 57
trandoshan-io/dashboard
Web UI to manage a Trandoshan instance
Language:TypeScript11 2 45
trandoshan-io/persister
Go process used to abstract database interaction. It consumes messages from queue and put result in database allowing asynchronous saving operation
Language:Go7 2 84
trandoshan-io/scheduler
Go orchestrator process used to schedule website parsing
Language:Go6 2 84
trandoshan-io/api
REST API used by other processes to gather data
Language:Go5 2 84
trandoshan-io/feeder
Oneshot process used to put an URL in the todo queue. Useful to start crawling process
Language:Go2 2 32
trandoshan-io/amqp
Go wrapper for streadway/amqp
Language:Go0 2 01