zachnewburgh/udacity-linux-server-configuration

Udacity Linux Server Configuration: udacity-catalog

This Python application provides a list of items within a variety of categories, as well as a user registration and authentication system.

Note: The walkthrough is the solution to the final project of the Udacity Full Stack Web Developer Nanodegree and deploys the solution of project 3 on the virtual machine.

Accessibility

• The web application accessible under the AWS-server instance. It uses port 2200.

Step by Step Walkthrough

The steps that are listed below are, in large part, a replication of the following guide. Items that are in CAPS should be replaced by app-specific information.

1 & 2 - Create Development Environment: Launch Virtual Machine and SSH into the server

Source: Udacity

1. Create a new development environment.
2. Download private keys and write down your public IP address.
3. Move the private key file into the folder ~/.ssh:
   $ mv ~/Downloads/udacity_key.rsa ~/.ssh/
4. Set file rights (only owner can write and read.):
   $ chmod 600 ~/.ssh/udacity_key.rsa
5. SSH into the instance:
   <pre>$ ssh -i ~/.ssh/udacity_key.rsa root@PUBLIC-IP-ADDRESS

3 & 4 - User Management: Create a new user and give user the permission to sudo

Source: DigitalOcean

1. Create a new user:
   $ adduser grader
2. Give new user the permission to sudo
3. Open the sudo configuration:
   $ visudo
4. Add the following line below root ALL...:
   grader ALL=(ALL:ALL) ALL
5. *List all users (Source: Ask Ubuntu):
   $ cut -d: -f1 /etc/passwd

5 - Update and upgrade all currently installed packages

Source: Ask Ubuntu

1. Update the list of available packages and their versions:
   $ sudo apt-get update
2. Install newer vesions of packages you have:
   $ sudo sudo apt-get upgrade

6 - Change the SSH port from 22 to 2200 and configure SSH access

Source: Ask Ubuntu

1. Change ssh config file:

2. Open the config file:
   $ sudo nano /etc/ssh/sshd_config

3. Change to Port 2200.

4. Change PermitRootLogin from without-password to no.

5. Temporalily change PasswordAuthentication from no to yes.

6. Append UseDNS no.

7. Append AllowUsers NEWUSER.
   Note: All options on UNIXhelp

8. Restart SSH Service:
   $ /etc/init.d/ssh restart or # service sshd restart

9. Create SSH Keys:
   Source: DigitalOcean

10. Generate a SSH key pair on the local machine:
    $ ssh-keygen

11. Copy the public id to the server:
    $ ssh-copy-id username@remote_host -pPORTNUMBER

12. Login with the new user:
    $ ssh -v grader@PUBLIC-IP-ADDRESS -p2200

13. Open SSHD config:
    $ sudo nano /etc/ssh/sshd_config

14. Change PasswordAuthentication back from yes to no.

7 - Configure the Uncomplicated Firewall (UFW) to only allow incoming connections for SSH (port 2200), HTTP (port 80), and NTP (port 123)

Source: Ubuntu documentation

1. Turn UFW on with the default set of rules:
   $ sudo ufw enable
2. *Check the status of UFW:
   $ sudo ufw status
3. Allow incoming TCP packets on port 2200 (SSH):
   $ sudo ufw allow 2200/tcp
4. Allow incoming TCP packets on port 80 (HTTP):
   $ sudo ufw allow 80/tcp
5. Allow incoming UDP packets on port 123 (NTP):
   $ sudo ufw allow 123/udp

8 - Configure the local timezone to UTC

Source: Ubuntu documentation

1. Open the timezone selection dialog:
   $ sudo dpkg-reconfigure tzdata
2. Then chose 'None of the above', then UTC.
3. *Setup the ntp daemon ntpd for regular and improving time sync:
   $ sudo apt-get install ntp

9 - Install and configure Apache to serve a Python mod_wsgi application

Source: Udacity

1. Install Apache web server:
   $ sudo apt-get install apache2
2. Open a browser and open your public ip address, e.g. http://35.165.202.220/ - There should be an Apache landing page that says 'It works!' at the top.
3. Install mod_wsgi for serving Python apps from Apache and the helper package python-setuptools:
   $ sudo apt-get install python-setuptools libapache2-mod-wsgi
4. Restart the Apache server for mod_wsgi to load:
   $ sudo service apache2 restart

10 - Install and configure PostgreSQL

Source: DigitalOcean (alternatively, nice short guide on Kill The Yak as well)

1. Install PostgreSQL:
   $ sudo apt-get install postgresql postgresql-contrib
2. Check that no remote connections are allowed (default):
   $ sudo nano /etc/postgresql/9.3/main/pg_hba.conf
3. Open the database setup file:
   $ sudo nano catalog_database.py
4. Change the line starting with "engine" to (fill in a password):
   engine = create_engine('postgresql://catalog:PW-FOR-CATALOG-DB@localhost/catalog')
5. Change the same line in catalog.py and catalog_populator.py
6. Rename catalog.py:
   $ mv catalog.py __init__.py
7. Create needed linux user for psql:
   $ sudo adduser catalog (choose a password)
8. Change to default user postgres:
   $ sudo su - postgres
9. Connect to the system:
   $ psql
10. Add postgres user with password:
    Sources: Trackets Blog and Super User
11. Create user with LOGIN role and set a password:
    # create user catalog with password 'PW-FOR-CATALOG-DB';
12. Allow the user to create database tables:
    # alter user catalog createdb;
13. *List current roles and their attributes: # \du
14. Create database:
    # create database catalog with owner catalog;
15. Connect to the database catalog # \c catalog
16. Revoke all rights:
    # revoke all on scheme public from public;
17. Grant only access to the catalog role:
    # grant all on schema public to catalog;
18. Exit out of PostgreSQl and the postgres user:
    # \q, then $ exit
19. Create postgreSQL database schema:
    $ python catalog_database.py

11 - Install git, clone and setup your Catalog App project

11.1 - Install and configure git

Source: GitHub

1. Install Git:
   $ sudo apt-get install git
2. Set your name for the commits:
   $ git config --global user.name "YOUR NAME"
3. Set up your email address to connect your commits to your account:
   $ git config --global user.email "YOUR EMAIL ADDRESS"

11.2 - Setup for deploying a Flask Application on Ubuntu VPS

Source: DigitalOcean

1. Extend Python with additional packages that enable Apache to serve Flask applications:
   $ sudo apt-get install libapache2-mod-wsgi python-dev
2. Enable mod_wsgi (if not already enabled):
   $ sudo a2enmod wsgi
3. Create a Flask app:
4. Move to the www directory:
   $ cd /var/www
5. Setup a directory for the app:
   1. $ sudo mkdir catalog
   2. $ cd catalog and $ sudo mkdir catalog
   3. $ cd catalog and $ sudo mkdir static templates
   4. Create the file that will contain the flask application logic:
   $ sudo nano __init__.py 5. Paste in the following code:
   python from flask import Flask app = Flask(__name__) @app.route("/") def hello(): return "Hello!" if __name__ == "__main__": app.run()
6. Install Flask
7. Install pip installer:
   $ sudo apt-get install python-pip
8. Install virtualenv:
   $ sudo pip install virtualenv
9. Set virtual environment to name 'venv':
   $ sudo virtualenv venv
10. Enable all permissions for the new virtual environment (no sudo should be used within):
    Source: Stackoverflow
    $ sudo chmod -R 777 venv
11. Activate the virtual environment:
    $ source venv/bin/activate
12. Install Flask inside the virtual environment:
    $ pip install Flask
13. Run the app:
    $ python __init__.py
14. Deactivate the environment:
    $ deactivate
15. Configure and Enable a New Virtual Host#
16. Open the existing config file: $ sudo nano /etc/apache2/sites-available/000-default.conf
17. Paste in the following lines of code and change names and addresses regarding your application:

  <VirtualHost *:80>
      ServerName PUBLIC-IP-ADDRESS
      ServerAdmin admin@PUBLIC-IP-ADDRESS
      WSGIScriptAlias / /var/www/catalog/catalog.wsgi
      <Directory /var/www/catalog/catalog/>
          Order allow,deny
          Allow from all
      </Directory>
      Alias /static /var/www/catalog/catalog/static
      <Directory /var/www/catalog/catalog/static/>
          Order allow,deny
          Allow from all
      </Directory>
      ErrorLog ${APACHE_LOG_DIR}/error.log
      LogLevel warn
      CustomLog ${APACHE_LOG_DIR}/access.log combined
  </VirtualHost>

3. Enable the virtual host:
   $ sudo a2ensite catalog
4. Create the .wsgi File and Restart Apache
5. Create wsgi file:
   $ cd /var/www/catalog and $ sudo nano catalog.wsgi
6. Paste in the following lines of code:

  #!/usr/bin/python
  import sys
  import logging
  logging.basicConfig(stream=sys.stderr)
  sys.path.insert(0,"/var/www/catalog/")
  
  from catalog import app as application
  application.secret_key = 'Add your secret key'

7. Restart Apache:
   $ sudo service apache2 restart

11.3 - Clone GitHub repository and make it web inaccessible

1. Clone project 3 solution repository on GitHub:
   $ git clone https://github.com/zachnewburgh/udacity-catalog.git
2. Move all content of created udacity-catalog directory to /var/www/catalog/catalog/ and delete the leftover empty directory.

$ sudo mv udacity-catalog/* catalog

$ sudo rm -r udacity-catalog

3. Make the GitHub repository inaccessible:
   Source: Stackoverflow
4. Create and open .htaccess file:
   $ cd /var/www/catalog/ and $ sudo nano .htaccess
5. Paste in the following:
   RedirectMatch 404 /\.git

11.4 - Install needed modules & packages

1. Activate virtual environment:
   $ source venv/bin/activate
2. Install httplib2 module in venv:
   $ pip install httplib2
3. Install requests module in venv:
   $ pip install requests
4. *Install flask.ext.seasurf (only seems to work when installed globally):
   $ *sudo pip install flask-seasurf
5. Install oauth2client.client:
   $ sudo pip install --upgrade oauth2client
6. Install SQLAlchemy:
   $ sudo pip install sqlalchemy
7. Install the Python PostgreSQL adapter psycopg:
   $ sudo apt-get install python-psycopg2

11.5 - Run application

1. Restart Apache:
   $ sudo service apache2 restart
2. Open a browser and put in your public ip-address as url, e.g. 35.165.202.220 - if everything works, the application should come up
3. *If getting an internal server error, check the Apache error files:
   Source: A2 Hosting
4. View the last 20 lines in the error log: $ sudo tail -20 /var/log/apache2/error.log
5. *If a file like 'google_client_secrets.json' couldn't been found:
   Source: Stackoverflow

11.6 - Get OAuth-Logins Working

Source: Udacity and Apache

1. Open http://www.hcidata.info/host2ip.cgi and receive the Host name for your public IP-address, e.g. for 35.165.202.220, it's ec2-35-165-202-220.us-west-2.compute.amazonaws.com
2. Open the Apache configuration files for the web app: $ sudo nano /etc/apache2/sites-available/000-default.conf
3. Paste in the following line below ServerAdmin:
   ServerAlias HOSTNAME, e.g. ec2-35-165-202-220.us-west-2.compute.amazonaws.com
4. Enable the virtual host:
   $ sudo a2ensite catalog
5. To get the Google+ authorization working:
6. Go to the project on the Developer Console: https://console.developers.google.com/project
7. Navigate to APIs & auth > Credentials > Edit Settings
8. add your host name and public IP-address to your Authorized JavaScript origins and your host name + oauth2callback to Authorized redirect URIs, e.g. ec2-35-165-202-220.us-west-2.compute.amazonaws.com/oauth2callback
9. To get the Facebook authorization working:
10. Go on the Facebook Developers Site to My Apps https://developers.facebook.com/apps/
11. Click on your App, go to Settings and fill in your public IP-Address including prefixed hhtp:// in the Site URL field
12. To leave the development mode, so others can login as well, also fill in a contact email address in the respective field, "Save Changes", click on 'Status & Review'