Udacity Full Stack Developer Nanodegree :: Deploying Item Catalog web application in a linux server
Take a baseline installation of a Linux distribution on a virtual machine and prepare it to host your web applications, to include installing updates, securing it from a number of attack vectors and installing/configuring web and database servers.
- Linux Server Configuration
- Deployment Details
- Deployment Steps
- 1. Initial login to the server as user
ubuntu
- 2. Update the operating system pakages and reboot if required
- 3. Configure automatic security and critical updates
- 4. Set timezone to UTC
- 5. Create user
grader
- 6. Create passwords for users
ububtu
andgrader
to do ssh-copy-id - 7. Add user
grader
to sudoers list - 8. Create SSH Keypairs for users
ubuntu
andgrader
- 9. Create a keypair in your local machine
- 10. Enable SSH logins through passwords in server temporarily
- 11.
ssh-copy-id
the local machines' public key tograder
- 12. Disable SSH logins through passwords in server permanently
- 13. Install Apache2 and enable required proxy modules
- 14. Configure firewall to allow OpenSSH, 'Apache Full', port 80, 123 and 2200
- 15. Enable port 2200 and HTTPS in the Lightsail VM Networking settings
- 16. Disable root login through SSH, change SSH port and add aloowed users to SSH config
- 17. Install Pip and Virtualenv, create a virtual environment for webapp
- 18. Install PostgreSQL and setup item-catalog database
- 19. Clone item-catalog git repository, put oauth2 data and install python packages
- 20. Configure gunicorn server and systemd service to manage the backend server
- 21. Configue Apache2 server to be a reverse proxy, add domain name
- 22. Configure HTTPS and SSL
- 1. Initial login to the server as user
- References
-
Application URL: https://md-item-catalog.ml
-
Virtual Server: Amazon Lightsail Instance
-
Operating System: Ubuntu 18.04 LTS
-
IP Address:
13.233.215.53
-
SSH Port:
2200
(Only Key based logins supported) -
Web Server: Apache2 Web Server acting as reverse proxy
-
Backend Server: Gunicorn with 4 workers running the web app listening to port 5000 in localhost. Managed by systemd.
$ ssh -i ~/.ssh/Lightsail_Key.pem ubuntu@13.233.215.53
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade
$ sudo reboot
Follow the official documentation: Ubuntu Automatic Update Configuration
Check if the current timezone is set to UTC using:
$ date
Tue Jan 29 15:42:28 UTC 2019
If not UTC set timezone to UTC using the command below:
(select 'None of the above' from the menu and then select 'UTC'.)
$ sudo dpkg-reconfigure tzdata
Current default time zone: 'Etc/UTC'
Local time is now: Tue Jan 29 15:45:18 UTC 2019.
Universal Time is now: Tue Jan 29 15:45:18 UTC 2019.
$ sudo adduser grader
Adding user 'grader' ...
Adding new group 'grader' (1002) ...
Adding new user 'grader' (1002) with group 'grader' ...
Creating home directory '/home/grader' ...
Copying files from '/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for grader
Enter the new value, or press ENTER for the default
Full Name []: Udacity Grader
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
$ sudo passwd ubuntu
Enter new UNIX password: ********
Retype new UNIX password: ********
passwd: password updated successfully
$ sudo passwd grader
Enter new UNIX password: ********
Retype new UNIX password: ********
passwd: password updated successfully
$ usermod -aG sudo grader
# As user ubuntu
$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa):
Created directory '/home/ubuntu/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:6xESKDYeewIYOE2upVA71LwRzayaA0DttwPHAWwVmzs ubuntu@ip-172-26-6-133
The keys randomart image is:
+---[RSA 4096]----+
|++*++B. |
|==.=+o* |
|=.% o*o |
|oB Oo+.. |
|o.+o=Eo S |
| +o o.. o |
| . . o |
| . . |
| . |
+----[SHA256]-----+
# As user grader
$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/grader/.ssh/id_rsa):
Created directory '/home/grader/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grader/.ssh/id_rsa.
Your public key has been saved in /home/grader/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:6xESKDYeewIYOE2upVA71LwRzayaA0DttwPHAWwVmzs grader@ip-172-26-6-133
The keys randomart image is:
+---[RSA 4096]----+
|++*++B. |
|==.=+o* |
|=.% o*o |
|oB Oo+.. |
|o.+o=Eo S |
| +o o.. o |
| . . o |
| . . |
| . |
+----[SHA256]-----+
$ ssh-keygen -t rsa -b 4096 -C maneeshd77@gmail.com
Generating public/private rsa key pair.
Enter file in which to save the key (/c/Users/mdivana/.ssh/id_rsa): fsnd_key
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in fsnd_key.
Your public key has been saved in fsnd_key.pub.
The key fingerprint is:
SHA256:GKW7yzA1J1qkr1Cr9MhUwAbHbF2NrIPEgZXeOUOz3Us maneeshd77@gmail.com
The keys randomart image is:
+---[RSA 2048]----+
|.*++ o.o. |
|.+B + oo. |
| +++ *+. |
| .o.Oo.+E |
| ++B.S. |
| o * =. |
| + = o |
| + = = . |
| + o o |
+----[SHA256]-----+
$ sudo vi /etc/ssh/sshd_config
Locate 'PasswordAuthentication no' and change it to 'PasswordAuthentication yes'. Save.
$ sudo serivce sshd restart
$ ssh-copy-id -i ~/.ssh/fsnd_key grader@13.233.215.53
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/c/Users/mdivana/.ssh/fsnd_key.pub"
The authenticity of host '13.233.215.53 (13.233.215.53)' cannot be established.
ECDSA key fingerprint is SHA256:/9TWCe3NH67EQErMsagjifJ8hFa7uIyu0Nq6r1Pu1Iw.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
grader@13.233.215.53 password:
Number of key(s) added: 1
Now try logging into the machine, with: 'ssh -i ~/.ssh/fsnd_key grader@13.233.215.53'
and check to make sure that only the key(s) you wanted were added.
Now we can login without using password from our local machine.
$ sudo vi /etc/ssh/sshd_config
Locate 'PasswordAuthentication yes' and change it to 'PasswordAuthentication no'. Save.
$ sudo serivce sshd restart
$ sudo apt-get install apache2
$ sudo a2enmod
give the below list of modules to enable
proxy proxy_ajp proxy_http rewrite deflate headers proxy_balancer proxy_connect proxy_html
$ sudo systemctl restart apache2
$ sudo ufw app list
Available applications:
Apache
Apache Full
Apache Secure
OpenSSH
$ sudo ufw allow OpenSSH
Rules updated
Rules updated (v6)
$ sudo ufw allow 'Apache Full'
Rules updated
Rules updated (v6)
$ sudo ufw allow 80
Rules updated
Rules updated (v6)
$ sudo ufw allow 2200
Rules updated
Rules updated (v6)
$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
$ sudo ufw status numbered
Status: active
To Action From
-- ------ ----
[ 1] OpenSSH ALLOW IN Anywhere
[ 2] Apache Full ALLOW IN Anywhere
[ 3] 80 ALLOW IN Anywhere
[ 4] 2200 ALLOW IN Anywhere
[ 5] OpenSSH (v6) ALLOW IN Anywhere (v6)
[ 6] Apache Full (v6) ALLOW IN Anywhere (v6)
[ 7] 80 (v6) ALLOW IN Anywhere (v6)
[ 8] 2200 (v6) ALLOW IN Anywhere (v6)
$ sudo reboot
Open the Lightsail Instance console. Go to Netorking tab. Then in 'Firewall' settings
- Add a Custom TCP protocol with Port as 2200 to be enabled.
- Add HTTPS protocol to be enabled.
$ sudo vi /etc/ssh/sshd_config
Locate 'Post 22'. Uncomment line and change to 'Post 2200' (without quotes)
Locate 'PasswordAuthentication'. Change to 'PasswordAuthentication no' (without quotes)
Add line 'AllowUsers ubuntu grader' (without quotes)
Add line 'PermitRootLogin no' (without quotes)
Save.
$ sudo service sshd restart
Now the we can only ssh to server using: 'ssh -i ~/.ssh/fsnd_key grader@13.233.215.53'
# As grader
$ sudo apt-get install build-essential python3-pip
$ sudo pip3 install virtualenv -U
$ virtualenv fenv
Using base prefix '/usr'
New python executable in /home/grader/fenv/bin/python3
Also creating executable in /home/grader/fenv/bin/python
Installing setuptools, pip, wheel...
done.
$ source ~/fenv/bin/activate
(fenv) $ pip list
Package Version
---------- -------
pip 19.0.1
setuptools 40.7.1
wheel 0.32.3
-- Install postgresql
$ sudo apt-get install postgresql
-- Switch into postgresql superuser 'postgres'
$ sudo su - postgres
-- Enter psql shell
$ psql
-- Create user 'catalog'
postgres=# CREATE ROLE catalog WITH LOGIN PASSWORD 'catalog';
CREATE ROLE
-- Create database 'catalog'
postgres=# CREATE DATABASE catalog;
CREATE DATABASE
-- Grant all previleges on database 'catalog' to user 'catalog'
postgres=# REVOKE ALL ON SCHEMA public FROM public;
REVOKE
postgres=# GRANT ALL ON SCHEMA public TO catalog;
GRANT
postgres=# GRANT ALL PRIVILEGES ON DATABASE catalog TO catalog;
GRANT
-- Exit psql shell
postgres=# \q
-- Exit postgres user
$ exit
Use postgresql://catalog:catalog@localhost/catalog
as database url in db_models.py
db_util.py
and server.py
in item-catalog
(fenv) $ git clone https://github.com/maneeshd/restaurants-menu.git item-catalog
Cloning into 'item-catalog'...
remote: Enumerating objects: 93, done.
remote: Counting objects: 100% (93/93), done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 93 (delta 26), reused 83 (delta 22), pack-reused 0
Unpacking objects: 100% (93/93), done.
(fenv) $ cd item-catalog
(fenv) $ pip install -r requirements.txt -U
# Put the database url in the files stated and then -
(fenv) $ python db_models.py
Using db_uri: postgresql://catalog:catalog@localhost/catalog to create models...
Database models have been created successfully.
(fenv) $ python db_util.py
Database has been populated successfully.
(fenv) $ vi wsgi.py
# Put the following inside wsgi.py
from server import APP
if __name__ == "__main__":
APP.run("localhost", port=5000, threaded=True)
Put the Google and Facebook OAuth2 data in oauth_data directory. (Refer oauth_data/README.txt
in item-catalog repo for more info)
$ cd ~/item-catalog
$ mkdir gunicorn_logs
$ vi gunicorn.conf
# Put the following data inside gunicorn.conf
workers = 3
errorlog = '/home/grader/item-catalog/gunicorn_logs/errors.log'
accesslog = '/home/grader/item-catalog/gunicorn_logs/access.log'
# Create a systemd service for gunicorn
$ sudo vi /etc/systemd/system/ItemCatalog.service
# Put the following data between --- inside ItemCatalog.service
--------------------------------
[Unit]
Description=Gunicorn instance to serve Item-Catalog
After=network.target
[Service]
User=grader
Group=grader
Restart=on-failure
WorkingDirectory=/home/grader/item-catalog
ExecStart=/home/grader/fenv/bin/gunicorn -c gunicorn.conf -b localhost:5000 wsgi:APP --preload --capture-output --log-level debug
[Install]
WantedBy=multi-user.target
--------------------------------
# Reload daemon
$ sudo systemctl daemon-reload
# Enable ItemCatalog service
$ sudo systemctl enable ItemCatalog
# Start ItemCatalog serivice
$ sudo systemctl start ItemCatalog
$ sudo systemctl status ItemCatalog
● ItemCatalog.service - Gunicorn instance to serve Item-Catalog
Loaded: loaded (/etc/systemd/system/ItemCatalog.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-01-28 13:00:07 UTC; 1 day 8h ago
Main PID: 17468 (gunicorn)
Tasks: 4 (limit: 547)
CGroup: /system.slice/ItemCatalog.service
├─17468 /home/grader/fenv/bin/python3 /home/grader/fenv/bin/gunicorn -c gunicorn.conf -b localhost:5000 wsgi:APP --preload
├─17491 /home/grader/fenv/bin/python3 /home/grader/fenv/bin/gunicorn -c gunicorn.conf -b localhost:5000 wsgi:APP --preload
├─17492 /home/grader/fenv/bin/python3 /home/grader/fenv/bin/gunicorn -c gunicorn.conf -b localhost:5000 wsgi:APP --preload
└─17493 /home/grader/fenv/bin/python3 /home/grader/fenv/bin/gunicorn -c gunicorn.conf -b localhost:5000 wsgi:APP --preload
Jan 28 13:00:07 ip-172-26-6-133 systemd[1]: Stopped Gunicorn instance to serve Item-Catalog.
Jan 28 13:00:07 ip-172-26-6-133 systemd[1]: Started Gunicorn instance to serve Item-Catalog.
lines 1-13/13 (END)
$ sudo vi /etc/apache2/sites-available/000-default.conf
<VirtualHost *:80>
ServerName md-item-catalog.ml
ServerAlias www.md-item-catalog.ml
ServerAdmin maneeshd77@gmail.com
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPreserveHost On
<Location "/">
ProxyPass "http://localhost:5000/"
ProxyPassReverse "http://localhost:5000/"
</Location>
</VirtualHost>
# Check for syntax errors
$ sudo apache2ctl configtest
Syntax OK
$ sudo apache2ctl restart
# Enable mod_ssl in apache2
$ sudo a2enmod ssl
# Install Certbot
$ sudo add-apt-repository ppa:certbot/certbot
Press ENTER to accept.
$ sudo apt-get install python-certbot-apache
Certbot is now ready to use, but in order for it to configure SSL for Apache we need to assign a ServerName in
the apache config i.e. a domain has to be assigned to ServerName and the www.domain assigned to ServerAlias.
$ cat /etc/apache2/sites-available/000-default.conf
Make sure ServerName and ServerAlias is set with required values.
$ sudo apache2ctl configtest
Syntax OK
$ sudo systemctl reload apache2
# Obtain a SSL certificate from LetsEncrypt
$ sudo certbot --apache -d md-item-catalog.ml -d www.md-item-catalog.ml
...
Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
-------------------------------------------------------------------------------
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
The configuration will be updated, and Apache will reload to pick up the new settings.
certbot will wrap up with a message telling you the process was successful and where your certificates are stored.
The server is now ready and secured. The web application is also secured with HTTPS/SSL encryption.