NGINX install error and 502 Bad Gateway
Closed this issue · 14 comments
I recently had some updates go through (primarily updating to Python 3.9.7) and I'm now getting a 502 Bad Gateway when trying to access Fermentrack. I believed I have traced it to an installation error with NGINX as shown by these results:
Traceback (most recent call last):
File "/usr/bin/apt-listchanges", line 30, in <module>
import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
Setting up nginx-full (1.14.2-2+deb10u4) ...
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xe" for details.
invoke-rc.d: initscript nginx, action "start" failed.
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2022-10-04 14:11:35 CDT; 60ms ago
Docs: man:nginx(8)
Process: 3077 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Process: 3078 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=1/FAILURE)
Oct 04 14:11:34 raspberrypi nginx[3078]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Oct 04 14:11:34 raspberrypi nginx[3078]: nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
Oct 04 14:11:34 raspberrypi nginx[3078]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Oct 04 14:11:34 raspberrypi nginx[3078]: nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
Oct 04 14:11:35 raspberrypi nginx[3078]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Oct 04 14:11:35 raspberrypi nginx[3078]: nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
Oct 04 14:11:35 raspberrypi nginx[3078]: nginx: [emerg] still could not bind()
Oct 04 14:11:35 raspberrypi systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
Oct 04 14:11:35 raspberrypi systemd[1]: nginx.service: Failed with result 'exit-code'.
Oct 04 14:11:35 raspberrypi systemd[1]: Failed to start A high performance web server and a reverse proxy server.
dpkg: error processing package nginx-full (--configure):
installed nginx-full package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of nginx:
nginx depends on nginx-full (<< 1.14.2-2+deb10u4.1~) | nginx-light (<< 1.14.2-2+deb10u4.1~) | nginx-extras (<< 1.14.2-2+deb10u4.1~); however:
Package nginx-full is not configured yet.
Package nginx-light is not installed.
Package nginx-extras is not installed.
nginx depends on nginx-full (>= 1.14.2-2+deb10u4) | nginx-light (>= 1.14.2-2+deb10u4) | nginx-extras (>= 1.14.2-2+deb10u4); however:
Package nginx-full is not configured yet.
Package nginx-light is not installed.
Package nginx-extras is not installed.
dpkg: error processing package nginx (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
nginx-full
nginx
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)
I don't really know where to go from here. I tried to remove NGINX then reinstall, but it returned the same output when trying to uninstall. Any help for this NOOB is greatly appreciated.
I was able to resolve the apt_pkg issue I believe and now I get a
“Welcome to nginx!
If you see this page, the nginx web server is successfully installed and working. Further configuration is required.”
Are you using the containerized version of Fermentrack? If so, just reinstall the container and you should be good.
Are you using the containerized version of Fermentrack? If so, just reinstall the container and you should be good.
Not sure. I ran docker.update.sh and found “docker-compose==1.21.0 distribution was not found. I will see if I can track that down.
Also tried a full reinstall using the one-line command and it just quit the install when it found Redis was running on port 6379.
Yeah - you’re almost certainly using the non-containerized version then.
Do you have a backup of your data (or do you not care about the data that you had in Fermentrack previously)? If so, then it should be pretty easy to migrate. If not, we’ll need to figure out a fix for your existing install first.
Yeah - you’re almost certainly using the non-containerized version then.
Do you have a backup of your data (or do you not care about the data that you had in Fermentrack previously)? If so, then it should be pretty easy to migrate. If not, we’ll need to figure out a fix for your existing install first.
I will see about doing a backup, otherwise I don't really have anything there I can't lose.
I did manage to fix the docker-compose issue using the non-docker fix environment command line. Still won't take a Fermentrack install though as it runs into Redis edit NEEDING port 6379, but it is in use already, and shuts down. I thought it was using it.
UPDATE: Still can't get past Redis needing port 6379 which is currently being used by Redis.
`pi@raspberrypi:~ $ sudo lsof -t -i:6379
1519
(I deleted the majority of the response from ps-ef as it didn't translate well into the post, but you can see PID 1519 running redis-server.)
pi@raspberrypi:~ $ ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1469 2 0 22:44 ? 00:00:00 [kworker/u8:0-events_unbound]
root 1499 1 0 22:44 ? 00:00:00 /usr/bin/containerd-shim-runc-v2
999 1519 1499 0 22:44 ? 00:00:01 redis-server 127.0.0.1:6379
root 1551 2 0 22:46 ? 00:00:00 [kworker/1:0-mm_percpu_wq]
root 1552 2 0 22:47 ? 00:00:00 [kworker/0:1-events]
pi 1558 1257 0 22:49 pts/0 00:00:00 ps -ef
pi@raspberrypi:~ $
`
Tried shutdown down the service with sudo kill -9 1519. But the service immediately reopens under a new PID. Deleted fermentrack-tools. Ran sudo apt-get remove -y redis-server then tried again with killing the Redis service. Still popping up on that port under a different PID each time. Ran a search for Redis files or folders and found a few folders remaining, but all are empty. Out of ideas.
...doh! The fact that you have containerd in there means you were on the docker version. Weird.
Anyways, download fermentrack-tools again, and re-run the installation script. It should clean everything up (and ideally shut the container holding redis down). If it doesn't - and still gives you the error - run docker ps and let me know what it reads.
Okay here is the initialization script and docker ps results.
pi@raspberrypi:~ $ sudo fermentrack-tools/non_docker_install/install.sh
::: This script was launched as root. Continuing installation.
_____ _ _
| ___|__ _ __ _ __ ___ ___ _ __ | |_ _ __ __ _ ___| | __
| |_ / _ \ '__| '_ ` _ \ / _ \ '_ \| __| '__/ _` |/ __| |/ /
| _| __/ | | | | | | | __/ | | | |_| | | (_| | (__| <
|_| \___|_| |_| |_| |_|\___|_| |_|\__|_| \__,_|\___|_|\_\
Welcome to the installation of Fermentrack. This script will install fermentrack.
A new user will be created and Fermentrack will be installed in that users home directory.
When the installation is done with no errors Fermentrack is started and monitored automatically.
Please note - Any existing apps that require Apache (including RaspberryPints and BrewPi-www)
will be deactivated. If you want support for these apps it can be optionally installed later.
Please read http://apache.fermentrack.com/ for more information.
For more information about Fermentrack please visit: http://fermentrack.com/
Do you want to continue to install Fermentrack? [y/N] y
::: Ok, let's go!
::: To accept the default answer, just press Enter.
::: The default is capitalized in a Yes/No question: [Y/n]
::: or shown between brackets for other questions: [default]
The time is currently set to Thu 06 Oct 2022 10:56:27 AM CDT. Is this correct? [Y/n]y
::: All scripts associated with Fermentrack are now installed to a user's home directory
::: Hitting 'enter' will accept the default option in [brackets] (recommended).
*** WARNING: Any data in the user's home directory may be ERASED during install!
What user would you like to install Fermentrack under? [fermentrack]: y
fatal: not a git repository (or any of the parent directories): .git
::: Configuring under user fermentrack
::: Configuring in directory /home/fermentrack
::: Checking for Internet connection:
::: Internet connection Success!
::: Installing dependencies using apt-get
::: apt is updated - installing git-core, nginx, python-dev, and a handful of other packages.
::: (This may take a few minutes during which everything will be silent) ...
::: All packages installed successfully.
::: Verifying free disk space...
Install directory is NOT empty, are you SURE you want to use this path? [y/N] y
::: Ok, we warned you!
::: Creating and configuring user accounts.
::: User 'fermentrack' already exists, skipping...
::: Checking install directories
::: Script install directory is NOT empty, backing up to this users home dir and then deleting contents...
::: Making sure everything is owned by fermentrack
::: Downloading most recent Fermentrack codebase...
Cloning into '/home/fermentrack/fermentrack'...
::: Making sure everything is owned by fermentrack
::: Creating virtualenv directory...
::: Python 3.7 is installed. Using Python 3.7 to create the venv.
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt-get install python3-venv
You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.
Failing command: ['/home/fermentrack/venv/bin/python3.7', '-Im', 'ensurepip', '--upgrade', '--default-pip']
::: Forcing reinstallation of pip within the virtualenv
sudo: /home/fermentrack/venv/bin/pip: command not found
sudo: /home/fermentrack/venv/bin/pip: command not found
::: Manually installing PyZMQ and Circus - This could take ~10-15 mins.
/home/fermentrack/venv/bin/python3: No module named pip
/home/fermentrack/venv/bin/python3: No module named pip
::: Numpy and Scipy are available through system packages. Linking to those.
::: Venv has been created!
::: Enabling python to query bluetooth without being root
::: Running make_secretsettings.sh from the script repo
Generating SECRET_KEY and writing to fermentrack_django/secretsettings.py
secretsettings.py created!
::: Running upgrade.sh from the script repo to finalize the install.
::: This may take up to an hour during which everything will be silent...
::: Copying nginx configuration to /etc/nginx and activating.
::: Making sure everything is owned by fermentrack
::: Running updateCronCircus.sh from the script repo
ERROR: Could not find circusd
::: Starting circus process monitor.
ERROR: Could not find circusd
Waiting for Fermentrack install to initialize and become responsive.
Fermentrack may take up to 5 minutes to first boot as the database is being initialized.
....................................................................................................................................................................................missing.
*** ----------------------------------
*** ERROR ERROR ERROR ERROR ERROR ***
Unable to find an initialized, responsive instance of Fermentrack
*** ----------------------------------
*** See above lines for error message
*** Setup NOT completed
*** More information in the "install.log" file
pi@raspberrypi:~ $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
48be3168e9f1 fermentrack_nginx "/docker-entrypoint.…" 4 months ago Restarting (1) 31 seconds ago fermentrack-tools_nginx_1
c27ba9e66570 fermentrack_redis "docker-entrypoint.s…" 4 months ago Restarting (1) 51 seconds ago fermentrack-tools_redis_1
pi@raspberrypi:~ $
Try using the regular install script, as you seem to have a docker installation Message ID: @.***>
Same issue with the Port 6379.
This was becoming too time consuming to figure out and I didn't really have anything I couldn't lose so I wiped my Raspberry Pi, fresh install of Raspi OS and Debian 11, then fresh install of Fermentrack. Everything back up and running UNTIL.........
My Tilt was not before and is not still communicating with Fermentrack. As before, and what began this problem, I ran the Troubleshoot Tilt connection and it says my Python flags are missing. The Troubleshoot says to run sudo setcap cap_net_raw,cap_net_admin+eip /usr/local/bin/python3.9. Did this, but as before, I got a
"Failed to set capabilities on file 'usr/local/bin/python3.9' (No such file or directory)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
I found no files of any kind in /usr/local/bin but I did find a file in usr/bin so I tried the command on that and now not only does the Tilt still not work. I hope to didn't goof anything up trying that option. Otherwise things are back up. I'd sure like to have my Tilt communicating again.
I see others running a separate install of Python in /usr/local/bin from their "system" install /usr/bin. Is this the solution? If so, how is the easiest way to do this?
@doubleAbrew sorry it took so long for you to get an answer- with only the context here, I think you have redis running on the host already, and then docker tries to run and grab the same port that the host redis is using. Try systemctl status redis-server (use tab complete on that, i think its redis-server) to see if redis is running, and then systemctl stop redis-server to stop it. Using 'stop' just turns it off until restart, so i think (Google is smarter than me) to perm turn it off you want to use systemctl disable redis-server after stopping, but again I would suggest Googling to make sure. Hope this helps!
I’m hoping that this is resolved at this point as I think it was environmental. If not, please feel free to comment here and we can reopen.