testdrivenio/django-on-docker

Intermittent 502 Errors When Customizing Dockerized Django / Gunicorn / Docker / Postgres

banagale opened this issue · 6 comments

banagale commented

This is a useful lesson and example, I customized it to ignore the advice of a managed database, because I wanted to keep costs down for my deployed staging instances on sites I was working on.

So this may not be present in the subject of this repo, and feel free to close this. However the fix was hard fought and I wanted to share it just in case someone else stumbled into it

I've detailed the issue of these troublesome 502's in this post. Hope this helps someone.

mjhea0 commented

Thanks for adding this comment. Intermittent 502s are difficult to troubleshoot. Curious - Why did you add 172.21.0.2:5432 to Nginx in the first place? Is that so you can access psql locally?

banagale commented

Curious - Why did you add 172.21.0.2:5432 to Nginx in the first place? Is that so you can access psql locally?

Yes, in your tutorials you highly recommend letting DO manage the DB. I agree though for staging and "pre-production" production that would be an extra $30 a month per project.

I wanted to allow at least staging deployed instances to use a local psql dB because it is "free" and all of the services can run on a 1gb ram DO droplet.

Does the configuration I have there match up with that explanation in your mind?

edit: For added context, I also suggested mentioning how to choose cheaper droplets in a reply to the GA post on reddit. :)

mjhea0 commented

It does. I just wouldn't add psql to the load balancer. Django can access it container to container. By exposing it to the world you open up a security issue along with the passive health check problem.

banagale commented

I see. Security is a major concern to me, so I definitely want to get that right.

Can you give a bit more information about the proper way to containerize the database, perhaps point me at an example? This may not be the forum for this question, but I can share the docker-compose.prod.yml if it would be helpful.

mjhea0 commented

Right now the entire Internet can access it. Just remove it from Nginx so that only Django can access it.

banagale commented

Thank you for that feedback. I have been somewhat guide-driven and in mashing together different examples but not spending enough time on the fundamentals of Docker networking, I did not see this exposure.

I was able to verify your assessment by scanning the port on the two deployed IPs. I removed the ports: - 5432:5432 from the db service, re-deployed and was able to see it was then filtered.

I think possibly what got me confused was I replaced your suggested entrypoint db healthcheck method with docker-compose-wait, which uses WAIT_HOSTS: db:5432 on the web container to make sure postgres is ready.

I hadn't realized that the port would still be available without doing that extra piece.