Video file gets corrupted with [error] 82#82: *1819 upstream prematurely closed connection while reading upstream
ingeorgepaul opened this issue · 7 comments
Describe the bug
We are using OpenVidu Pro 2.22.0. Video file gets corrupted when downloading the recordings. We faced this issue with 3 files, and all the files were large, more than 1GB. While analysing the OV logs we found below error for all the three corrupted files:
2023/03/10 18:52:22 [error] 82#82: *1819 upstream prematurely closed connection while reading upstream, client: [IP Address obfuscated], server: , request: "GET /openvidu/recordings/ses_NwM3hGGdog~1/78c91576203cf75e17de9547438ff0c9.mp4 HTTP/1.1", upstream: "http://127.0.0.1:5443/openvidu/recordings/ses_NwM3hGGdog~1/78c91576203cf75e17de9547438ff0c9.mp4", host: ""
Expected behavior
Irrespective of the file size, all the files should successfully recorded and downloaded which can be played in any video player that supports the format in which file is recorded.
Wrong current behavior
File gets corrupted and cannot be played though the file is successfully downloaded and has size more than 1GB.
OpenVidu tutorial where to replicate the error
This is an intermittent issue, there are no particular steps involved to reproduce the issue. Only clue is that every time it occurred, a large file more than 1GB was involved.
OpenVidu deployment info
How is your OpenVidu Server instance deployed when you get the bug. A couple of possible examples are listed below:
- Docker container as explained in OpenVidu tutorials, run with command
docker run ...on macOS Catalina 10.15.1
IMPORTANT NOTE: please, if you think the bug might be related to OpenVidu Server side, specify here if you are also getting the error by using OpenVidu Server Demos instance. This instance is publicly available (use it only for this test, because it is not secure!!!): URL:
https://demos.openvidu.io:4443, SECRET:MY_SECRET
Client device info (if applicable)
Describe the client device(s) or platform(s) where you are able to replicate the error. For example:
- On one end, any android or iPhone device acting as Subscriber and Publisher
- On other end, Chrome browser acting as Publisher and Subscriber.
Screenshots
N/A
Additional context
Add any other context about the problem here. For example, attach any useful logs related to the issue.
Please have a look into this.
I think this can be related to an NGNIX timeout.
NGNIX is being used as a reverse proxy to route all traffic in the OpenVidu Pro deployment. For large file downloads it may take some time to complete, and the NGINX default configuration may not be enough for this downloads to complete.
We will take a look at this, stay tuned.
Do you have any more detailed log of the whole downloading process that ends up with that failure?
I'd like to see the openvidu-server-pro container log with timestamps included.
Hello @ingeorgepaul , you can customize your Nginx as desired by using this guide here: https://docs.openvidu.io/en/2.26.0/troubleshooting/#162-modify-openvidu-nginx-configuration
Make adjustments, and if you identify which parts of the Nginx configuration are affecting your use case, I can modify or parameterize it for the next version.
Hello @ingeorgepaul, were you able to solve the problem by modifying the nginx container?
Sorry, was busy with other stuffs. Just saw your message. Will have a look and revert.
This issue was resolved by following the instructions here and adding the following to line 88 of custom-nginx.conf:
proxy_max_temp_file_size 20480m
Edit to add: it's still a mystery why this is not necessary on over OVP installs on the same version.