How to troubleshoot a service when it fails to run [workflow for docker swarm deploy]
AYEG opened this issue · 3 comments
Hi!
I'm trying to deploy a cluster using these instructions .
https://github.com/tiangolo/full-stack/blob/master/docker-swarm-cluster-deploy.md
It seem like the DB is not coming online and causing the backend to also fail after many attempts from tenacity.
$ docker service logs db tells me "no such task or service: db", but I can see it is listed with
$ docker service ls, with replicas 0/1
I don't have previous experience with swarm, is there a way to access failed service logs?
I tried following advice like print journalctl, but that doesnt seem very useful in this case. Should I include something extra to output logs?
One more thing I tried is edit docker-stack.yml to include
logging:
driver: journald
Then run this command:
$ journalctl -u docker CONTAINER_NAME=db
As described here:
https://stackoverflow.com/questions/43663013/access-logs-of-a-killed-docker-container/43663253
It tells me:
-- Logs begin at Mon 2018-11-12 21:37:13 CET, end at Thu 2018-12-27 14:15:03 CET. --
-- No entries --
Found a way to print errors here is a possible solution for anyone else wondering how to do this:
docker service ls // list the services
docker service ps <id>
// shows container id and a truncated error message in second to last column
docker inspect <container id from first column>
| grep Err // print out the error message belonging to the service container
"Err": "no suitable node (scheduling constraints not satisfied on 1 node)"
Running these commands no longer requires enabling experimental docker features, it works by default
Sorry for the delay.
When you create a Docker Swarm mode stack, all the "services" get a name prefixed by the stack name.
So, if you create a stack my-super-app-com
, your backend
service will be named my-super-app-com_backend
.
Then you can see the logs with:
docker service logs my-super-app-com_backend
To see only the last 100 logs:
docker service logs my-super-app-com_backend --tail=100
To see only the last 100 logs and keep showing any new logs live, "following" the logs:
docker service logs my-super-app-com_backend --tail=100 -f
(This last one is what I use the most).