Task queues backed by ES: Tasques.
Some of these may be goals :p
- Easily scalable:
- Servers are stateless; easily spin more up as needed
- The storage engine is Elasticsearch, nuff' said.
- Tasks can be configured
- Priority
- When to run
- Retries
- Tasks can be configured to retry X times, with exponential increase in run times
- Timeouts
- Tasks that are picked up by workers that either don't report in or finish on time get timed out.
- Unclaiming
- Tasks that were picked up but can't be handled now can be requeued without consequence.
- Go 1.13+
- Install
Go
- Clone this repo
- Stand up an ES (e.g. with Docker
docker run -p 9200:9200 --env discovery.type=single-node elasticsearch:7.5.2
) - Copy the
config/tasques.example.yaml
toconfig/tasques.yaml
, adjust if needed go run ./app
and hit the endpoints.
- For Swagger, go to localhost:8080/swagger/index.html
The server supports APM, as configured according to the official docs.
- Install
Go
- Use your favourite editor/IDE
- For updating Swagger docs:
- Install Swaggo
- Run
swag init -g app/main.go
from the root project dir- Check that there are no
time.Time
fields... there's a race condition in there somewhere
- Check that there are no
- Commit the generated files.
- For updating the Go Client:
- Install go-swagger
- Run
swagger generate client -f docs/swagger.yaml
- Commit the generated files.