ConnectionError: Need to add both User-Agent header and timeout to requests
jdholstein opened this issue · 5 comments
Non-deterministically, tests will fail on master branch and all others due to requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))
This even happens on Docker and is not due to waiting for containers to spin up.
This can be prevented by adding a timeout and a User-Agent header to the request objects in api_client (in master) and wherever request is used in previous chapters
Not will investigate further, seems that this does not fix it. Will leave issue up in case others have it. Note I am running on an ARM mac with docker desktop
Another useful thing I found to do is remove the "up" dependency in docker compose for test commands, this causes the containers to restart and can cause the intermittent failure as well.
Closing this issue as of now. Doing the above has solved the issue for me to some degree. The real problem is that Arm (m1) macs and docker containers are in a somewhat bad state. Docker desktop works, but that doesn't mean the containers do. My advice would be to spin up a VM or use another OS for this book, as there are multiple subtle issues to to the lack of chip architecture support across the board.
For one more post, I finally found the issue in case anyone else on linux or mac sees the same thing. It is an issue with using the flask dev server. This is because of multiple reasons, one being the IPv4 to IPv6 translation on localhost. This will lead to a 5s delay while being rerouted to the ipv4 '0.0.0.0' address for the first request. Figured out the OS will route properly for the requests following if they are immediate, but if there is lag (say re running the test with make), it will happen again. I figured this out by running hey load test against the server. One requests was always 5 seconds latency out of thousands. There are many workarounds, but what I found best is to just use gunicorn or something similar in place of the default flask dev server. Now things work perfectly and the e2e tests are lightning fast.
@jdholstein i'm having the same issue. any chance you could share your docker file and any other changes you made in order to use gunicorn?