- This is a collection of Ansible Roles and Playbooks that are used to provision web hosting machines.
- Currently, the playbooks are configured to provision a machine with a webserver, Redis cache, and a database rather than using separate machines for each role.
(Commands provided are specific to Ubuntu)
-
Must have Git installed:
apt-get update apt-get -y install git
-
Must have Ansible installed
sudo apt-get install software-properties-common sudo apt-add-repository ppa:ansible/ansible sudo apt-get update sudo apt-get install ansible
-
Must have the 'passlib' Python module for generating passwords
pip install passlib --user python
-
Must have the 'dopy' Python module installed (Digital Ocean API wrapper)
pip install dopy --user python
-
Must have the 'sshpass' program (when connecting to a local Vagrant machine only - All other environments should use private-keys instead of passwords)
brew install https://raw.githubusercontent.com/kadwanev/bigboybrew/master/Library/Formula/sshpass.rb
-
On the managed nodes, you need a way to communicate, which is normally ssh. By default this uses sftp. If that’s not available, you can switch to scp in ansible.cfg.
-
You also need Python 2.4 or later. If you are running less than Python 2.5 on the remotes, you will also need:
python-simplejson
-
Each machine being managed by this codebase should have an entry in the 'inventory':
-
Open the
/inventory
directory and set the URLs and hostnames for the local, staging, and production machines being managed. -
To specify a server that should be provisioned by this script but not dynamically created, add the address (IP or domain) to the
[web]
section of an inventory file.-
Example: To provision a static server for Staging, the
inventory/staging
file should contain:[web] staging.my-domain.com
-
-
To add a Digital Ocean droplet to the inventory, just add the name of the droplet beneath the
[droplets]
heading in an inventory file. If the droplet doesn't already exist when the script runs, it will be dynamically created. If it already exists, it will-
Example: To provision 2 identical Production droplets, the
inventory/production
file should contain:[droplets] my-production-droplet-1 my-production-droplet-2
-
To change the type of Droplet that you want to use for a particular environment, set the variables in the
[droplets:vars]
section of the inventory file (i.e. inventory/production):[droplets:vars] droplet_size_id=2gb droplet_region_id=sfo2 droplet_image_id=ubuntu-16-04-x64
-
-
-
All of the vhosts ('server blocks') that should exist for the Nginx webserver are defined in the
group_vars
folder in a file named after their environment (i.e.group_vars/production.yml
defines users and vhosts on production machines). The structure of these files is as follows:--- GROUPS_LIST: - &developers name: "developers" id: 1000 - &clients name: "clients" id: 1001 DEVELOPERS_LIST: - &jeff name: "jeff" id: 2000 groups: - "developers" - "sudo" ssh_keys: - "jeff.pub" - "jeff_at_macbook.pub" CLIENTS_LIST: - &sunvalleymag name: "sunvalleymag" id: 3017 groups: - "clients" FTPUSERS_LIST: - &sunvalleymag-ftp name: "sunvalleymag-ftp" password: $1$SomeSalt$CyaZSTvM30xGoZ8PajwNA1 root: "/home/sunvalleymag/sites" id: 4000 groups: - "sunvalleymag" user_groups: - <<: *developers - <<: *clients - <<: *ftpusers users: - <<: *jeff - <<: *sunvalleymag databases: - name: "sunvalleymag" vhosts: - listen: "80" server_name: "sunvalleymag.com" server_aliases: - "www.sunvalleymag.com" wordpress: true php: true root: "/home/sunvalleymag/sites/sunvalleymag.com/public"
-
When a vhost is added that didn't previously exist on the server, the following actions are taken on the server:
- A new user is created with a home folder but no password (
/sbin/nologin
) - A server block (vhost) is added to nginx.
- A new MariaDb database is created for each member of the
users.databases
list (defined in thegroup_vars
YAML files). Connections to these db's have no password, but are only allowed from localhost.
- A new user is created with a home folder but no password (
-
When a vhost is removed (or renamed) in the vhosts.yml file, its symlink is deleted from the
/etc/nginx/sites-enabled
folder, but no other changes are made.All of the site data remains in the document root, and the vhost config file remains in
/etc/nginx/sites-available
. -
Example vhost entry with ALL OPTIONS documented:
Example vhost entry with all options:
- <<: *client_username
databases:
- name: "client_database_1"
- name: "client_database_2"
vhosts:
- listen: "443 ssl"
ssl: true
redirect_from_http: true # Redirect traffic from http://ansible-test.com to https://ansible-test.com
server_name: "ansible-test.com" # REQUIRED
server_aliases:
- "www.ansible-test.com www2.ansible-test.com"
root: "/home/ansibletest/ansible-test.com/public" # REQUIRED
# The 'extra_parameters' will get copied directly
# into your nginx server block and can be used to implement
# a custom behavior.
extra_parameters: >
location ~ ^/(assets/|images/|img/|favicon.ico) {
access_log off;
expires max;
}
- listen: "80"
server_name: "olddomain.com"
extra_parameters: >
return 301 https://newdomain.com$request_uri;
redis_port: "6379"
# Add cron jobs to this user's crontab by adding them each on their own line
cron:
- "* * * * * (cd /home/boise/sites/boise.guru && php artisan schedule:run)"
- "* * * * * (cd /home/boise/sites/boise.guru && php artisan inspire)"
# Use the Wordpress variable to apply nginx snippets to this vhost's server config
wordpress: true
# Set `php` to `true` to create a php-fpm pool for this user
php: true
php_fpm:
listen_allowed_clients: 127.0.0.1
pool_user: www-data
pool_group: www-data
pm_start_servers: 5
pm_max_children: 5
pm_min_spare_servers: 5
pm_max_spare_servers: 10
pm_max_requests: 500
php_admin_disable_functions: "exec,passthru,shell_exec,system"
php_admin_allow_url_fopen: "off"
nodejs:
version: '6.6.0'
packages:
- name: eslint
- name: stylelint
pm2_apps:
- run: "/home/oliverfinley/sites/oliverfinley.com/process.json"
env:
NODE_ENV: staging
env:
- APP_DEBUG=false
- APP_ENV=production
- CACHE_DRIVER=redis
- SESSION_DRIVER=file
- QUEUE_DRIVER=sync
# DON'T CREATE DATABASE ENTRIES (DB_HOST, DB_USERNAME, etc)
# THESE ARE AUTOMATICALLY GENERATED FROM THE 'database' KEY ABOVE
upstreams:
- listen: "80"
server_name: "oliverfinley.staging.portonefive.com"
server_aliases:
- "www.oliverfinley.staging.portonefive.com"
name: ofa-frontend
location: "/"
strategy: "ip_hash"
keepalive: 32
servers: {
"localhost:3000",
}
extra_parameters: >
location /assets {
root /home/oliverfinley/sites/admin.oliverfinley.staging.portonefive.com/public;
}
This codebase is meant to reflect the current state of live hosting machines. Previous configurations can be inspected by using version control.
- Changes to this repository should follow a similar Git workflow as other projects:
-
New changes should be made on the
develop
branch. Testing these new changes should be done locally (on a Vagrant machine, for example) -
Applying your changes to the staging server is done by merging into the
staging
branch. This triggers a Bamboo build that runs:$ ansible-playbook portonefive.yml -i inventory/staging.yml
which executes the script on each server in the
inventory/staging.yml
file. -
Applying your changes to the production server is done by merging into the
master
branch. This triggers a Bamboo build that runs:$ ansible-playbook hosting-box.yml -i inventory/production.yml
which executes the script on each server in the
inventory/production.yml
file.
-
All of the nginx configuration variables can be set in the default vars file for the nginx role:
-
Global vars
roles/geerlingguy.nginx/defaults/main.yml
-
Ubuntu-specific vars:
roles/geerlingguy.nginx/vars/Debian.yml
-
If the Ansible playbook fails because of unreachable hosts, it could be caused by Ansible attempting to provision a new Digital Ocean droplet before the droplet has finished booting up. Ansible is supposed to wait for the droplet to become available on port 22 before beginning to provision, but sometimes it doesn't work. This issue resolves itself if the playbook is executed again after the droplet has finished booting up (usually within 10 seconds or so). This issue will only arise the first time the script runs after adding a new droplet to an inventory file.
-
If the playbook fails on the
restart nginx
task:RUNNING HANDLER [geerlingguy.nginx : reload nginx] ***************************** fatal: [138.68.13.234]: FAILED! => {"changed": false, "failed": true, "msg": "Job for nginx.service failed because the control process exited with error code. See "systemctl status nginx.service" and "journalctl -xe" for details.\n"
there is most likely a problem with your vhost configuration. Make sure that any changes to the group_vars files follow the examples in this Readme.
-
404 Not Found
- Try adding an entry to your
hosts
file to connect your Digital Ocean droplet's IP with the hostname that you're trying to visit. This will solve issues where there is either no DNS record configured (no domain has been linked to the droplet's IP) or the DNS hasn't taken effect yet (if you've just recently created the DNS records). - Make sure to check Cloudflare's DNS records if the domain you're trying to connect to is cached by Cloudflare!
- Try adding an entry to your
-
502 Bad Gateway
-
SSH into the droplet and run
systemctl status nginx.service
This will tell you if there's something wrong with the nginx service. -
SSH into the droplet and run
sudo nginx -t
This will check the nginx vhost config files and tell you if there's any syntax errors. Make sure to run this command withsudo
or else the test will report lots of permissions errors because it's not able to access the files that you've asked it to test. -
SSH into the droplet and run
sudo vim /home/VHOST-USER/logs/DOMAIN-error.log
making sure to replacevhost-user
andDOMAIN
with your actual values. For example, the nginx error log forportonefive.com
(owned by theportonefive
user) is located at/home/portonefive/logs/portonefive.com-error.log
. If you see this error:connect() failed (111: Connection refused) while connecting to upstream...
then you have an invalid vhost configuration. This problem arises when you have aproxy_pass
statement inside alocation
block in your vhost config file that points to a hostname that isn't defined by anupstream
statement at the top of that same file. -
SSH into the droplet and run
sudo vim /etc/nginx/sites-available/yourdomain.com
Make sure to replaceyourdomain.com
with the actual domain name that you entered into the vhost settings in the ansible script. This will open up the vhost configuration file for your site.
-