/TEDxStartSomething

TEDx StartSomething

Primary LanguageObjective-C

<img src=“https://codeclimate.com/github/DiUS/TEDxStartSomething/badges/gpa.svg” /> <img src=“https://codeclimate.com/github/DiUS/TEDxStartSomething/badges/coverage.svg” />

TEDxStartSomething Care and Feeding Instructions ↑

Setting up Continuous Integration (CI) ↑

Hardware (AWS EC2) ↑

  1. Create a security group with http, https, ssh, and custom TCP on port 8080.

  2. Create an Ubuntu AMI with hardware capacity of t2.medium.

  3. Attach the created security group to the Ubuntu AMI.

  4. Attach an elastic IP’s to the EC2 instance.

Setup CI ↑

Install Dependencies ↑

  1. sudo apt-get install openjdk-7-jdk

  2. sudo apt-get install git

  3. wget -q -O - jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -

  4. sudo sh -c ‘echo deb pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list’

  5. sudo apt-get update

  6. sudo apt-get install jenkins

  7. sudo apt-get install phantomjs

Install Jenkins ↑

  1. Install MySQL: sudo apt-get install mysql-server mysql-client libmysqlclient-dev.

  2. Install libxml2: sudo apt-get install libxslt-dev libxml2-dev.

  3. Install Qt WebKit: sudo apt-get install libqtwebkit-dev.

Configure Jenkins ↑

  1. Install Jenkins plugins

* github (also installs git, which is what is really wanted) * green balls * build pipeline

  1. Change time zone.

Configure Build ↑

Tests Job ↑

  1. Create a Tests Jenkins job.

  2. In Source Code Management, Repository URL enter: github.com/DiUS/TEDxStartSomething.git.

  3. Poll SCM: */3 * * * *, rather than use Github hooks, which is also good.

  4. In Build, Execute Shell:

#!/bin/bash -xl \curl -sSL get.rvm.io | bash -s stable -ruby=1.9.3 source /var/lib/jenkins/.rvm/scripts/rvm source /var/lib/jenkins/.rvm/scripts/rvm rvm use gemset actionman bundle install export DB_USERNAME=actionman export DB_PASSWORD=<db_password> RAILS_ENV=test bundle exec rake db:migrate RAILS_ENV=test bundle exec rake

  1. In Post-build Actions, Downstream Project Names enter: QA_Deploy.

QA_Deploy Job ↑

  1. Create a QA_Deploy Jenkins job.

  2. In Source Code Management, Repository URL enter: github.com/DiUS/TEDxStartSomething.git.

  3. In Build, Execute Shell:

#!/bin/bash -xl \curl -sSL get.rvm.io | bash -s stable –ruby=1.9.3 source /var/lib/jenkins/.rvm/scripts/rvm source /var/lib/jenkins/.rvm/scripts/rvm rvm use gemset actionman bundle install bundle exec cap qa deploy

  1. In Post-build Actions, Downstream Project Names enter: Production_Deploy.

Production_Deploy Job ↑

  1. Create a Production_Deploy Jenkins job.

  2. In Source Code Management, Repository URL enter: github.com/DiUS/TEDxStartSomething.git.

  3. In Build, Execute Shell:

#!/bin/bash -xl \curl -sSL get.rvm.io | bash -s stable –ruby=1.9.3 source /var/lib/jenkins/.rvm/scripts/rvm source /var/lib/jenkins/.rvm/scripts/rvm rvm use gemset actionman bundle install bundle exec cap production deploy

Set up Elastic IP ↑

  1. Allocate a new address.

  2. Attach the new address to the CI EC2 instance.

Jenkins is currently accessible via 54.206.91.221:8080/.

Setting Up Production/Staging Servers ↑

Database Server (AWS RDS) ↑

  1. Create a security group with http, https, ssh, and custom TCP on port 8080, or re-use security group already created for CI and allocate that to the RDS instance.

  2. Create a small RDS instance with 20GB.

Production/Staging Application Servers (AWS EC2) ↑

Hardware (AWS EC2) ↑

  1. Create a security group or groups with http, https, ssh, and custom TCP on port 8080, or re-use security group already created for CI and allocate that to the RDS instance.

  2. Create two Ubuntu AMI’s with hardware capacity of t2.small.

  3. Attach the created security group or groups to the Ubuntu AMI.

  4. Attach elastic IP’s to each of the EC2 instances.

Install Dependencies ↑

  1. Install ansible

See, http://docs.ansible.com/intro_installation.html:

* git clone git://github.com/ansible/ansible.git
* cd ./ansible
* source ./hacking/env-setup
* sudo easy_install pip
* sudo pip install paramiko PyYAML jinja2 httplib2

  1. ansible-playbook -i inventory/hosts.ini rails_app_server.yml –extra-vars “rds_defalt_password=<RDS_root_password> db_password=<RDS_user_password>”

Set up Elastic IP’s ↑

  1. Allocate two new addresses.

  2. Attach the new addresses, one each to the staging and production EC2 instances.

Semi-automating Deployments ↑

  1. Generate ssh keys on CI server.

  2. Copy ssh keys to staging and production servers.

Deployment ↑

  1. Go to: 54.206.91.221:8080/view/Build%20Pipeline/.

  2. Click “Run” to start a new pipeline.

  3. Deployment to production occurs when the second box is green and the “Run” icon is clicked, (hopefully) causing the third and last box to go green. At this point deployment should be complete.

EC2 Blackouts ↑

  1. Install ruby with development headers (the ruby version in apt repository will do the job):

sudo apt-get install ruby
sudo aptitude install ruby1.9.1-dev

  1. On the production EC2 instance, install ec2-blackout gem:

sudo gem install ec2-blackout

  1. Follow ec2-blackout instructions to configure the gem

  2. Tag prdoduction instance with “no blackout”

  3. Add the following to crontab (assuming system is running in UTC):

0 9 * * * AWS_ACCESS_KEY_ID=<access key> AWS_SECRET_ACCESS_KEY=<secret key> /usr/local/bin/ec2-blackout on --exclude-by-tag no_blackout
0 21 * * * AWS_ACCESS_KEY_ID=<access key> AWS_SECRET_ACCESS_KEY=<secret key> /usr/local/bin/ec2-blackout off

Setting Up Local Development Environment ↑

  1. git clone git@github.com:DiUS/TEDxStartSomething.

  2. bundle install.

  3. rake db:create.

  4. rake db:setup.

Deploy from local machine ↑

$ DEPLOY_KEY=<path_to_deployer_pem> AWS_ACCESS_KEY_ID=<key> AWS_SECRET_ACCESS_KEY=<secret> bin/cap canary deploy

Elasticsearch ↑

Install ↑

wget download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz tar -xvzf elasticsearch-0.20.5.tar.gz

IMPORTANT: edit elasticsearch-0.20.5/config/elasticsearch.yml to change the cluster.name to a unique name! (otherwise it’ll form a cluster between the development machines)

Run ↑

cd elasticsearch-0.20.5 bin/elasticsearch -f

Re-seeding data ↑

To re-seed and re-index:

rake db:seed search:import

Just re-indexing:

rake search:import

NOTE: you may see “Too amny open files” in the elasticsearch logs. This causes indexing to fail sometimes.

Solution for this problem can be found here:

www.elasticsearch.org/tutorials/2011/04/06/too-many-open-files.html

Performance testing with httperf ↑

Setup ↑

Note: To reduce latency deviation I would installing the httperf and running the test from the EC2 instance (i.e. QA),

but not from the production machine itself as the test take a lot of resources.

Clone the httperf that supports big sessions:

git clone git://github.com/Gregg/httperf_big_cookies.git

Build and install httperf:

sudo mv httperf_big_cookies /usr/local/lib cd /usr/local/lib/httperf_big_cookies ./configure make sudo make install

Verify that httperf has installed correctly:

httpferf -h

Running the test ↑

Prepare the path_sequence.txt. The file basically describes the sequence for requests

httperf –wsesslog=40,2,path_sequence.txt –rate=1 –timeout=5 –session-cookie –server=54.79.75.3 –port=80

Explanation ↑

The above command will be means the following:

40 session, each starts withing 1 second of each other firing events in path_sequence.txt every two seconds.

Important Metrics ↑

Request rate: 13.3 req/s (75.4 ms/req) Reply rate [replies/s]: min 1.4 avg 13.6 max 16.8 stddev 4.3 (41 samples)

Thie basically means that fore every 13.3 req on everage 13.6 req came back per second (the second number is bigger in this case due to rump up). Watch the standard deviation stays relatively low.

Reply time [ms]: response 297.6 transfer 173.5 Make sure this stays with acceptable rate (< 500ms??)

It is also good to personally play with the app while the test is executing.

Trello ↑

trello.com/b/vHeUZle9/tedxultimo-startsomething