DIT People Finder

Installing for development

This is not how people finder is actually deployed but provides an environment to do development on the app.

Ubuntu install

On a Ubuntu 12.04 LTE box:

install curl, git, postgresql, postgresql-dev-all, nodejs
install rails through rvm. One way is:
- gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
- \curl -L https://get.rvm.io | bash -s stable --ruby
start a new shell
rvm gemset use global
gem install bundler
gem install rails
clone this repository
cd peoplefinder
bundle install
sudo su postgres createuser ubuntu (or the name of the user the application will be running as)
createdb peoplefinder_development (as the user the application will be running as)
bin/rake db:migrate RAILS_ENV=development
bundle exec rails s -b 0.0.0.0

Point your browser to http://0.0.0.0:3000 and you should see the application's start page.

Mac OSX install

Install Java if it is not on your machine. Alternatively, you can install it with Homebrew-Cask:

brew cask install java

Install Homebrew if it is not on your machine.

On Mac OSX:

brew install postgresql
brew install imagemagick
brew install phantomjs

brew install elasticsearch
brew services start elasticsearch


git clone git@github.com:uktrade/peoplefinder.git
cd peoplefinder

gem install eventmachine -v 1.0.5 -- --with-cppflags=-I/usr/local/opt/openssl/include

bundle
bundle exec rake db:setup
bundle exec rake peoplefinder:db:reload # includes demo data
bundle exec rake # runs tests
bundle exec rails s -b 0.0.0.0

In a separate terminal, run job worker for sending emails:

cd peoplefinder
bundle exec rake jobs:work

To catch emails in development, in a separate terminal, install the mailcatcher gem. Run mailcatcher and view emails at http://localhost:1080:

cd peoplefinder
gem install mailcatcher
mailcatcher

Configuration

These should be defined in the config/application.rb or in the enviroments/environment.rb files if the settings need to be defined on a per environment basis.

config.app_title e.g. 'My New People Finder'

config.default_url_options e.g. { host: mail.peoplefinder.example.com }

config.disable_organogram Disable the 'View printable organogram' feature

config.disable_token_auth Disable the 'token-based authentication' feature

config.disable_open_profiles Disables the adding/editing/deleting of another person's profile (see below)

config.elastic_search_url Required for production (see Search section below)

config.support_email e.g. 'peoplefinder-support@example.com'

config.send_reminder_emails Set to true if reminder emails are to be sent by cronjobs

Using an alternate home page

Set ENV['HOME_PAGE_URL'] to redirect requests to the root path, or leave it blank to render homes#show.

Google Analytics

Set the tracking id as: ENV['GA_TRACKING_ID']

Disabling open profiles

An initial premise behind the Peoplefinder was that it should be 'open' and people would be encouraged to collaborate and support each other building profile. A team member could add a profile for a new colleague, or remove a profile when a colleague has left the department, etc.

However, there are some issues implementing this with the DIT's SSO. For example, Alice adds a new profile for Bob, entering bob@example.com as Bob's email address. When Bob logs in via the SSO, we can't predict which email address will be returned in the auth hash, so we can't be confident of identifying him correctly.

By disabling open profiles as config.disable_open_profile, we will prevent a person from editing another person's profile, thereby limiting the risks of not knowing how to link an authenticated user to their profile.

Permitted domains

The system allows logging in for emails which have domains from the whitelist. The whitelist is in the database, managed by PermittedDomain model. At least one domain has to be whitelisted before anyone can log in (that applies to development too).

In rails console:

PermittedDomain.create(domain: 'some.domain.gov.uk')

Authentication

DIT SSO Authentication

Authentication to Log in to People Finder in the various environments (dev/staging/prod) is by setting the following environment variables: DITSSO_INTERNAL_PROVIDER DITSSO_INTERNAL_CLIENT_ID DITSSO_INTERNAL_CLIENT_SECRET

Authentication in the development environment

Setting the DEVELOPER_AUTH_STRATEGY variable in the development environment will swap out the default DIT SSO authentication for Omniauth's developer strategy.

Token-based authentication

NOTE: This has been disabled for DIT SSO Internal

~~An alternative 'token-based' authentication method is also supported. The token authentication method relies upon the users access to their email account to authenticate them.~~

Each time the user wishes to start a session, they need to generate an authentication token. This can be done by entering their email address (from a permitted domain) on the login screen. They will be sent an email message containing a link with a unique random token. Clicking on the link will allow them to login.

E-mails

People Finder sends a few types of e-mail. E-mails are delivered using delayed_job adapter for activejob.

Run bundle exec rake jobs:work to activate the worker.

In production, periodic emails are sent to users that have:

never logged in before;
not updated their profile for a period of time; and
not added a team description when they are a team leader.

Cronjobs are created to daily check for users matching these conditions and send emails. Cronjobs are created via the whenever gem and configured here: https://github.com/uktrade/peoplefinder/blob/master/config/schedule.rb

In Development

E-mails in development environment are setup to be delivered using mailcatcher gem. For that mailcatcher has to be started and then accessed on http://localhost:1080 to read the delivered e-mails.

Search

To run the engine in production mode, config.elastic_search_url must be set in, for example, config/application.rb. See 'Configurable elements' above.

Heroku provides Bonsai Elasticsearch as an add-on.

You can install a development version from Elasticsearch 1.7.3 downloads or with a package manager. e.g. brew install elasticsearch17.

Elasticsearch requires jdk version 7 or greater.

To check the health of the elasticsearch (ES) stack you can use the following, from either host instance:

curl 'localhost:9200/_cat/health?v'

or view ES settings and stats:

curl 'localhost:9200/_cluster/stats/?pretty'
curl 'localhost:9200/_cat/indices?v'
curl 'localhost:9200/_cat/nodes?v'

If you get an IndexMissingException, you will need to index the Person model:

bundle exec rake environment elasticsearch:import:model CLASS='Person' FORCE=y

Or, alternatively:

rake peoplefinder:es:index_people

Or you can create the index from the console:

Person.__elasticsearch__.create_index! index: Person.index_name, force: true

And populate it:

Person.import

You can also delete the index:

Person.delete_indexes

To run specs without Elasticsearch:

bundle exec rspec . --tag ~elastic

Images

Manipulation

We use MiniMagick so either Imagemagick or Graphicsmagick need to be installed for image manipulation and for some of the tests.

If using brew you can use the following command:

brew install imagemagick

Storage

For the dev environment the profile images are stored as files in the container and therefore do not persist between container deploys. For the Demo, Staging and Production environments profile images are stored in their own AWS S3 bucket. The buckets do not grant any group permissions to non-AWS users (i.e. are private). Access to the images is achieved via presigned, time-limited urls generated by the app.

Images that are uploaded to the bucket by the app explicitly prevent read to the "Everyone" AWS group using CarrierWave configuration in its initializer - the default for this config is true/public.

config.fog_public = false # default: true

Profile images had originally applied the AWS Canned-ACL public-read, rendering them public, and therefore two rake tasks have been written to assist with amending the Access Control List for image objects.

To make images private (and therefore accessible only via presigned-urls):

rake peoplefinder:S3:privatise_images

To make images available to the "Everyone" AWS group:

rake peoplefinder:S3:publicise_images

Testing

You'll need to install PhantomJS in order to run the headless browser tests and the smoke_test.

On OSX:

brew install phantomjs

On a Linux box, you won't find a pre-packaged headless phantomjs as of this writing. You might need to recompile it from source (we have it in a private apt repository).

Also, if you'd like test coverage for Javascript you'll need to have Node and Istanbul installed. The easiest way to do this is installing Node via nvm and then use npm to install Istanbul like so:

npm install -g istanbul

View templates

The application layout is set by the moj_internal_template that is installed as part of this engine.

You can override this layout in wrapper application, create your own file:

app/views/layouts/peoplefinder/peoplefinder.html.haml

Translation file

A lot of the text in the views is configurable in the translations file.

You can override these in wrapper application by creating your own file:

config/locales/en.yml

Utilities

Random data generator for testing

The RandomGenerator is able to generate several layers of teams and people with randomly generated details in those teams.

Usage:

  group = Group.find(...)

  # initialise the generator with a parent group
  generator = RandomGenerator.new(group)

  # clean all subgroups and people within the provided parent group
  generator.clear

  # generate team structure and people with the given parameters
  groups_levels = 2 # number of levels to generate
  groups_per_level = 3 # how many teams per each level
  people_per_group = 5 # how many people should be in the bottom most teams
  domain = 'fake.gov.uk' # which e-mail address should be used for e-mails (has to be whitelisted)
  generator.generate(groups_levels, groups_per_level, people_per_group, domain)

You can also generate semi-random data using the rake task peoplefinder:data:demo which is called as part of peoplefinder:db:reload. Repeatedly running peoplefinder:demo:data will add members to the example groups it creates.

Useful rake tasks

Run rake -T | grep people for latest list:

rake peoplefinder:data:demo                    # create basic demonstration data
rake peoplefinder:data:demo_csv[count,file]    # create a valid csv for load testing, [count: number of records=500], [file: path to file=spec/fixtures/]
rake peoplefinder:db:clear                     # drop all tables
rake peoplefinder:db:reload                    # drop tables, migrate, seed and populate with demonstration data for development purposes
rake peoplefinder:db:reset_column_information  # reset all column information
rake peoplefinder:import:csv_check[path]       # Check validity of CSV file before import
rake peoplefinder:import:csv_import[path]      # Import valid CSV file

Mail previews

Mail previews can be found at http://localhost:3000/rails/mailers, assuming the server is running locally on port 3000.

Geckoboard

A Geckboard dashboard is used for the visualization of various metrics from Peoplefinder. These metrics take the form of either pollable endpoints locate under app/metrics/.. OR pushable/publishable geckboard datasets located under lib/geckoboardpublisher/..

The datasets are scheduled for pushing in conf/schedule.rb using the whenever gem and cron installed on the worker instances (production environment only).

Once published, datasets are wired up on Geckboard by adding a widget, specifying "Datasets" for the connection type and selecting the published dataset from the list available.

For testing purposes you can manually publish reports as below:

GeckboardPublisher::ProfilePercentagesReport.new.publish!

which will, on staging, create a dataset called peoplefinder-staging.profile_percentages_report

and to remove the report...

GeckboardPublisher::ProfilePercentagesReport.new.unpublish!

NOTE: there is a limit of 100 datasets per geckboard account and a limit of 5000 records per dataset. Further, you can only push a maximum of 500 records per request.

Front end development

A large part of the audience for Peoplefinder are, at time of writing, still reliant on Windows XP for their OS and IE7 for their browser. Consequently considerable styling and javascript needs IE7 workarounds. To assist, you can install Virtual Box VMs using the guide below

install virtual box - Virtualbox
you can download fairly reliable MS Windows VMs using this site at xdissent. The simplest method is to download the Windows 7 with IE10 VM and use the Developer tools in IE10 to change Browser mode and Document mode to IE7.
```
curl -s https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | env IEVMS_VERSIONS="10" bash
```
once command above has completed power on the machine in virtual box, start IE and use http://10.0.2.2:3000 (i.e. 10.0.2.2 maps to the host's localhost IP)
you can map the VMs localhost to 10.0.2.2 to allow GPlus OAuth login to function by amending the hosts file, C:\Windows\System32\drivers\etc\hosts, to add the line below:

10.0.2.2    localhost

Optional config for your VM for local development:

enable host to VM copy/paste

vbox > machine > settings > General(tab) > Advanced(tab) > Shared clipboard(dropdown) > Bidirectionl

Development tools

CI by Travis.

Software metrics by Code Climate

Reminders

If the Peoplefinder is to be successful, profiles need to be populated and maintained.

Support

A support email address is set as SUPPORT_EMAIL.

Profile API

A profile summary for a person can be retrieved by the API: /api/people?email=bob@example.com Note that a token is required, and should be configured as: ENV['PROFILE_API_TOKEN'] So the API can be access with curl: curl -H "Authorization: Token token=<PROFILE_API_TOKEN>" http://localhost:3000/api/people.json?email=bob@example.com

agiamas/peoplefinder