This document outlines the steps needed to set up and deploy the SRPC backend for development purposes and production use.
- Prerequisites
- Code Repository Management
- Virtual Environment Setup
- Running the Backend Locally
- Deployment in AWS EC2 Instance
- Launching a New EC2 Instance
- SSH Access to EC2 Instance
- Initial Steps after Connecting to an EC2 Instance
- Checking if Application is Running for Testing Purposes
- Installing NGINX
- Configuring NGINX as a Reverse Proxy
- Installing Gunicorn
- Creating a Gunicorn Systemd Service File
- Installing SSL Certificate Using Certbot
- Updating the changes in the code from local base to github repository to EC2 instance
- Ensure Python 3.8.0 or greater is installed on your system.
pip
should be available to manage Python packages.
-
Configure SSH for GitHub to push changes securely: SSH Key Setup Guide.
-
Clone the repository using the SSH URL:
git clone git@github.com:sunnypranay/SRPC_BACK_END_PRIVATE.git
Note: Perform this step first to obtain the project files.
A virtual environment is used to create an isolated Python environment to manage dependencies required by the project.
-
Navigate to the project directory:
cd SRPC_BACK_END_PRIVATE
-
Install
virtualenv
:pip install virtualenv
-
Create a virtual environment:
virtualenv <env_name>
Note: Replace
<env_name>
with your desired environment name. -
Activate the virtual environment:
source <env_name>/bin/activate
-
Install project dependencies:
pip install -r requirements.txt
-
Deactivate the virtual environment when done:
deactivate
-
Initialize the database (only needed for the first run or when models change):
python manage.py migrate
-
Start the Django development server:
python manage.py runserver
-
Access the backend at:
http://127.0.0.1:8000/
-
Test all changes locally before pushing to the repository.
After testing the code locally, the changes can be committed and pushed to the GitHub repository using the following commands:
git add .
git commit -m "<commit_message>"
git push
Note: Replace <commit_message>
with a short description of the changes made.
This section guides you through deploying the SRPC Backend on an AWS EC2 instance.
-
Create a New EC2 Instance:
- Go to the AWS Management Console, find the EC2 service, and click on "Launch Instance".
- Provide a name for your instance such as
SRPC_BACKEND
for clarity and future reference.
-
Choose an Amazon Machine Image (AMI):
- Select "Ubuntu Server 20.04 LTS (HVM), SSD Volume Type" for a stable and widely supported environment.
- Ensure the architecture is set to x86 for compatibility with most software.
-
Select an Instance Type:
- For testing purposes, a
t2.micro
instance is sufficient and cost-effective. - For production, consider
t2.medium
ort2.large
for better performance based on your load expectations.
- For testing purposes, a
-
Configure a Key Pair:
- Choose to create a new key pair.
- Name your key pair (e.g.,
srpc_backend_key
) and download it to secure SSH access to your instance. - Keep the key pair file (
*.pem
) in a secure location and change file permissions usingchmod 400 srpc_backend_key.pem
to make it read-only by the file owner.
-
Configure Security Group:
- Add rules to your security group to open necessary ports:
- Port 22 (SSH): For secure shell access.
- Port 80 (HTTP): For web traffic.
- Port 443 (HTTPS): For secure web traffic.
- Port 8000: For Django development server (only if required for external access; not recommended for production).
- Limit access to only necessary IP addresses for enhanced security, especially for SSH access.
- Add rules to your security group to open necessary ports:
-
Configure Storage:
- Allocate at least 30 GB of General Purpose SSD (gp2) storage. Adjust according to your application’s needs.
-
Review and Launch:
- Review all configurations and make sure everything is set up as required.
- Launch the instance by clicking the "Launch" button.
After launching, don't forget to note your instance's public IP or DNS, as you'll need it for SSH access and potentially configuring DNS for your domain.
- SSH Client:
- It's recommended to use Bitvise SSH Client for a user-friendly interface. Download it from the Bitvise website.
- SSH Client:
- Termius is a versatile SSH client available for macOS. You can download it from the Termius website.
- Termius offers a free tier, but you can also get it for free with the GitHub Student Developer Pack if eligible.
-
Setting up New Host:
- Launch Termius and select "New Host".
- Enter the public IP address of your EC2 instance in the address field.
- For the username, enter
ubuntu
which is the default for AWS Ubuntu instances.
-
Authentication Using SSH Key:
- In the password section, choose to set an SSH key.
- Click on "New Key" and fill in the "Label" field.
- Import your downloaded
.pem
file from when you created the EC2 instance by using the import option. - Save the details to proceed.
-
Establishing the Connection:
- Navigate back to the host list and click on the newly created host to initiate the connection.
- If prompted to add the host to your list of known hosts, confirm by clicking "Yes".
- These steps assume that you have already set up your EC2 instance with the appropriate security group rules allowing SSH access (usually port 22).
- For security reasons, it is recommended not to use the root user for SSH access if available. Instead, use a user with sufficient privileges for your required tasks.
By following these steps, you should be able to securely access your AWS EC2 instance via SSH using your chosen client.
After successfully connecting to your AWS EC2 instance via SSH, perform the following initial setup steps:
-
System Update and Upgrade: Ensure your system's package index is up-to-date and upgrade the installed packages to their latest versions:
sudo apt update && sudo apt upgrade -y
-
SSH Key for GitHub:
- Generate a new SSH key pair on your EC2 instance which you will use to securely communicate with GitHub:
Replace
ssh-keygen -t ed25519 -C "your_email@example.com"
your_email@example.com
with your GitHub email address. - After generating the SSH key, follow the instructions on GitHub to add your public SSH key to your GitHub account. Refer to this GitHub guide for detailed steps.
- Generate a new SSH key pair on your EC2 instance which you will use to securely communicate with GitHub:
-
Clone the Repository:
- Clone your private GitHub repository using the SSH protocol:
git clone git@github.com:sunnypranay/SRPC_BACK_END_PRIVATE.git
- When prompted to confirm the authenticity of the host (GitHub), type
yes
to continue.
- Clone your private GitHub repository using the SSH protocol:
-
Navigate to Project Directory:
- Change to the directory containing your project:
cd SRPC_BACK_END_PRIVATE
- Use
ls
to view the contents of the directory to confirm you're in the right place.
- Change to the directory containing your project:
Before proceeding to a production setup with Gunicorn and Nginx, it's important to verify that the Django application runs correctly using the built-in development server.
-
Navigate to Project Directory:
cd SRPC_BACK_END_PRIVATE
-
Install PIP: PIP is the package installer for Python. You can install it using the following command:
sudo apt install python3-pip -y
-
Install Project Dependencies: Use PIP to install all the required dependencies listed in the
requirements.txt
file:pip install -r requirements.txt
-
Run the Development Server: Start the Django development server on port 8000, making it accessible from any IP address:
python3 manage.py runserver 0.0.0.0:8000
-
Access the Application:
- Open a web browser and go to your EC2 instance's public IP address or DNS name followed by
:8000
, like so:http://<your_ec2_instance_public_ip>:8000
- Replace
<your_ec2_instance_public_ip>
with the actual public IP address of your EC2 instance. - A Django error page with a "Page Not Found (404)" message is normal if there's no index page set up. This confirms that the server is running.
- Open a web browser and go to your EC2 instance's public IP address or DNS name followed by
-
Debug Mode Notice:
- Ensure that the
DEBUG
setting in Django is set toFalse
when you're deploying to a production environment for security reasons.
- Ensure that the
-
Next Steps:
- Now that you've confirmed the application is running, you can proceed to set up a more robust web server configuration using Gunicorn and Nginx.
- After that, you can install an SSL certificate to enable HTTPS on your domain.
NGINX is a high-performance web server that can also be used as a reverse proxy and load balancer. Follow these steps to install and verify NGINX on your EC2 instance:
-
Install NGINX: Use the package manager to install NGINX:
sudo apt update sudo apt install nginx -y
-
Check NGINX Version: Confirm that NGINX is installed correctly by checking its version:
nginx -v
-
Verify NGINX Service Status: Check if the NGINX service is running:
sudo systemctl status nginx
- Press
q
to quit the status output.
- Press
-
Access NGINX Default Page:
- Access the default NGINX landing page to verify that the web server is serving content:
- Open a web browser and navigate to your EC2 instance's public IP address or DNS name using
http://
(nothttps://
):http://<your_ec2_instance_public_ip>
- Replace
<your_ec2_instance_public_ip>
with the actual IP address of your EC2 instance.
- Open a web browser and navigate to your EC2 instance's public IP address or DNS name using
- If you see the "Welcome to nginx!" page, then NGINX is successfully installed and serving the default page.
- Access the default NGINX landing page to verify that the web server is serving content:
Purpose of a Reverse Proxy:
- NGINX as a reverse proxy can manage external access to your web services and can provide load balancing, SSL termination, and caching.
- Using a reverse proxy allows your Django app to run securely as a non-root user on a high port while still being accessible on the standard web ports (80 for HTTP and 443 for HTTPS).
Setting Up NGINX Server Block:
-
Create NGINX Server Block:
- Edit a new configuration file using a text editor such as nano:
sudo nano /etc/nginx/sites-available/srpc_backend
- Edit a new configuration file using a text editor such as nano:
-
Enter Server Block Configuration:
- Add the following configuration to the file, adjusting the
proxy_pass
if your Django app is running on a different port:server { listen 80; server_name _; location / { proxy_pass http://127.0.0.1:8000; # Ensure this matches the port of your Django app proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } }
- Add the following configuration to the file, adjusting the
-
Enable the New Configuration:
- Create a symbolic link to enable the server block:
sudo ln -s /etc/nginx/sites-available/srpc_backend /etc/nginx/sites-enabled/
- Ensure that the default configuration does not conflict with your new server block, and remove or edit it if necessary:
sudo rm /etc/nginx/sites-enabled/default
- Create a symbolic link to enable the server block:
-
Test and Restart NGINX:
- Test your NGINX configuration for syntax errors:
sudo nginx -t
- If the test passes, reload NGINX to apply the changes:
sudo systemctl reload nginx
- Test your NGINX configuration for syntax errors:
-
Verify Reverse Proxy Functionality:
-
To check if the reverse proxy is working, run the Django development server on port 8000:
python3 manage.py runserver
-
Access your EC2 instance's public IP address or DNS name using
http://
(nothttps://
):http://<your_ec2_instance_public_ip>
-
Replace
<your_ec2_instance_public_ip>
with the actual IP address of your EC2 instance. -
If you see the Django application running, then the reverse proxy is working correctly.
-
In later steps we will configure Gunicorn to run the Django application server on port 8000 and use systemd to manage it as a service to run it as a background process and restart it automatically if it crashes or the system reboots.
-
Gunicorn is a WSGI server that can run multiple Django applications in parallel. Follow these steps to install and verify Gunicorn on your EC2 instance:
- Navigate to Project Directory:
cd SRPC_BACK_END_PRIVATE
- Install Gunicorn:
Use apt to install Gunicorn:
sudo apt install gunicorn
- Test Gunicorn:
- Start Gunicorn on port 8000:
gunicorn srpc_new.wsgi:application --bind 127.0.0.1:8000
srpc_new
is the name of the Django project, andwsgi
is the name of the WSGI module.- Access your EC2 instance's public IP address or DNS name using
http://
(nothttps://
):http://<your_ec2_instance_public_ip>
- Start Gunicorn on port 8000:
-
Create a new systemd service file for Gunicorn:
sudo nano /etc/systemd/system/srpc.service
-
Add the following configuration to the service file
[Unit] Description=gunicorn daemon for srpc_new After=network.target [Service] User=ubuntu WorkingDirectory=/home/ubuntu/SRPC_BACK_END_PRIVATE ExecStart=/ExecStart=/usr/bin/gunicorn --access-logfile /var/log/srpc_new/access.log --error-logfile /var/log/srpc_new/error.log --workers 3 --bind 127.0.0.1:8000 srpc_new.wsgi:application Restart=on-failure RestartSec=15 ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s TERM $MAINPID PrivateTmp=true [Install] WantedBy=multi-user.target
- Description: A description of the service.
- After: Tells systemd that this service should be started after the networking target has been reached.
- User: The name of the user to run the service as.
- WorkingDirectory: The directory that Gunicorn should use as the base directory.
- ExecStart: The command to start the service.
- In the ExecStart command, we use the full path to the Gunicorn executable, which we can find using the
which
command:which gunicorn
- In the ExecStart command, we can see the we have specified the location of the access and error logs. We will create these log files in the next step.
- Restart: Tells systemd to restart the service if it crashes.
- RestartSec: Specifies the restart delay in seconds.
- ExecReload: Tells systemd how to reload the service.
- ExecStop: Tells systemd how to stop the service.
- PrivateTmp: Sets up a private /tmp directory for the service.
- WantedBy: Specifies that this service should be started automatically at boot.
-
Create the log files:
sudo mkdir -p /var/log/srpc_new sudo chown ubuntu:ubuntu /var/log/srpc_new
-
Reload systemd to load the new service:
sudo systemctl daemon-reload
-
Start the service:
sudo systemctl start srpc.service
-
Enable the service to start at boot:
sudo systemctl enable srpc.service
-
Check the status of the service:
sudo systemctl status srpc.service
- Press
q
to quit the status output.
- Press
-
To check the logs of the service:
cd /var/log/srpc_new/ cat access.log cat error.log
-
Access your EC2 instance's public IP address or DNS name using
http://
(nothttps://
):http://<your_ec2_instance_public_ip>
- Replace
<your_ec2_instance_public_ip>
with the actual IP address of your EC2 instance. - If you see the Django application running, then Gunicorn with systemd is working correctly.
- To check if the systemd service runs after rebooting the system, you can reboot the system using
sudo reboot
after 5 minutes connect to ec2 instance using ssh and check the status of the service again using
sudo systemctl status srpc.service
- Replace
-
Login to the domain registrar and create an A record for your domain pointing to your EC2 instance's public IP address.
-
If your domain in Godaddy provider then follow the below steps to add
A
record for your domain.- Login to your Godaddy account and go to the My Products page.
- Scroll down to the
Domains
section and click on theDNS
button next to your domain. - Click on the
Add
button and selectA
record. - Enter elavant
subdomain name
for exampletestapi
in theHost
field and enter your EC2 instance's public IP address in thePoints to
field. - Click on the
Save
button to save the changes.
-
In the browser check if the domain is pointing to your EC2 instance's public IP address with http://<your_domain_name> according to example that would be
http://testapi.uwsrpc.org
and you should see the django application running. It means the domain is pointing to your EC2 instance's public IP address successfully. -
Now go to your EC2 instance and change the ngnix configuration file to add your domain name in the
server_name
field.sudo nano /etc/nginx/sites-available/srpc_backend
- Replace
your_domain_or_IP
with your domain name in theserver_name
field.
previous configuration
server { listen 80; server_name _; # Dash indicates that this server block will respond to all domain names location / { proxy_pass http://127.0.0.1:8000; # Ensure this matches the port of your Django app proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } }
New configuration
server { listen 80; server_name your_domain; # Instead of _ Dash, Replace with your domain name which is pointing to your EC2 instance's public IP address according to example that would be testapi.uwsrpc.org location / { proxy_pass http://localhost:8000; # Ensure this matches the port of your Django app proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } }
- Save the changes and exit from the file using
ctrl + x
andy
to save the changes.
- Replace
-
Reload the ngnix configuration to apply the changes.
sudo systemctl reload nginx
check the status of the ngnix service to confirm the changes.
sudo systemctl status nginx
-
Now go the website of letsencrypt and follow the steps to install the certbot on your EC2 instance.
-
below are the steps mentioned in CertBot website
-
Install Certbot
sudo apt-get update sudo snap install --classic certbot
-
Prepare the Certbot command
sudo ln -s /snap/bin/certbot /usr/bin/certbot
-
Get and install your certificates..
sudo certbot --nginx
-
You will be prompted to enter your email address and agree to the terms of service. and it will ask you to share your email address with the Electronic Frontier Foundation, which is the nonprofit organization that develops Certbot. You can choose whether or not to share your email address with EFF.
-
It will ask Which names would you like to activate HTTPS for? We recommend selecting either all domains, or all domains in a VirtualHost/server block.
1: testapi.uwmsrpc.org (according to our previous example you should able to see your domain name here that you have added in the ngnix configuration file)
Select the appropriate numbers separated by commas and/or spaces, or leave input blank to select all options shown (Enter 'c' to cancel):
select the appropriate number and press enter.
In my case I have selected1
and press enter. -
It will say something like this
Congratulations! You have successfully enabled https://testapi.uwsrpc.org You should test your configuration at: https://www.ssllabs.com/ssltest/analyze.html?d=testapi.uwsrpc.org
-
Now if you go to your domain name with https://<your_domain_name> according to example that would be
https://testapi.uwsrpc.org
you should see the django application running with https.
- Navigate to Project Directory:
cd SRPC_BACK_END_PRIVATE
- Pull the latest changes from the github repository:
git pull
- Install Project Dependencies:
Use PIP to install all the required dependencies listed in the
requirements.txt
file:pip install -r requirements.txt
- Stop the gunicorn service
sudo systemctl stop srpc.service
- restart the gunicorn service
sudo systemctl start srpc.service
- check the status of the gunicorn service
sudo systemctl status srpc.service
- check the website with https://<your_domain_name> according to example that would be
https://testapi.uwsrpc.org
you should see the django application running with https.