OmniLayer/omni-devops

Omni Core & Omniwallet install, deploy, and test tools

Omniwallet, OmniEngine, and Master Core Devops

This repository is unmaintained and years out of date. It needs a major overhaul/update before it is useful again.

We are looking for a new maintainer, if you think you can help, please see #10.

Deployment and management scripts for the new Master Core-based Omniwallet.

Omniwallet consists of 4 main components/servers. The Vagrantfile in this repository contains VM configurations for 3 of these components. We are currently focused on using Amazon RDS for the PostgreSQL server component, but PostgreSQL could be run from a Vagrant VM or any PostgreSQL server host.

1. Master Core RPC Server: mastercore VM
2. PostgreSQL Database Server: PostgreSQL or Amazon RDS
3. OmniEngine Server: omniengine VM
4. Omniwallet Server: omniwallet VM (in progress, be prepared to help debug)

Although it should be possible to deploy all four servers in a single virtual (or physical) machine, it is recommended to use four seprate VMs and these scripts are currently designed to build and deploy 4 separate VMs.

Prerequisites

• Virtual Box 4.3.10 or later
• Vagrant 1.6.2 or later

Vagrant is available for Mac OS X, Windows, and Linux. In addition to VirtualBox, Vagrant may be used to provision VMWare, AWS and other virtual environments.

Recommended/Optional tools

• Bash command-line shell
• PostgreSQL command line client tools
• Java JDK
• Groovy language
• Amazon EC2 CLI Tools
• Amazon RDS Command Line Tools

Base Box

The Vagrantfile is currently using (trusting) the ubuntu/trusty64 base box created by Ubuntu and the Mastercoin base box msgilligan/mastercoin-ubuntu-base.

You should consider them untrusted binaries. Only use them with TEST-MSC and small amounts of Bitcoin.

Installing Prerequisites

1. Install VirtualBox

2. Install Vagrant

3. Clone this repository and check out the 'master' branch

    $ git clone git@github.com:mastercoin-MSC/omni-devops.git
    $ cd omni-devops
   

Installing support for AWS

The instructions for setting up the various servers show the --provider=aws option enclosed in square brackets [ ] which indicates that it is an optional option. If used, the --provider=aws option requires that Vagrant have support for AWS installed and a AWS base box named "omni-aws" created. (If the option is not specified, Vagrant will use VirtualBox on your local system, in which case you can skip this section.)

1. Install the Vagrant AWS plugins

    $ vagrant plugin install vagrant-aws
    $ vagrant plugin install vagrant-awsinfo
   

2. Create the Omni AWS Vagrant Base Box

    $ cd omni-devops
    $ vagrant box add omni-aws tools/omni-aws.box
   

Master Core RPC Server Setup

VM name mastercore

1. Copy DefaultVagrantConfig.rb to LocalVagrantConfig.rb and make sure LocalVagrantConfig.rb contains the correct values for Master Core RPC setup. For the mastercore VM you'll need to set the AWS variables (starting with AWS_, except AWS_CREDENTIAL_FILE), the OMNIENGINE_GIT_REPO and OMNIENGINE_GIT_BRANCH setting to specify where to fetch the source to build from, and the BTCRPC_ variables (except BTCRPC_CONNECT which isn't needed to create the mastercore VM, but to access it).

2. Make sure mastercore-synced/openssl.cnf has the values you want for your self-signed SSL certificate.

3. Create and boot a VM with Vagrant and install Master Core

    $ vagrant up mastercore [--provider=aws]
   

The Master Core daemon is now running as an Ubuntu service and will be automatically restarted upon system reboots as well as if the daemon crashes.

1. If you have not done so already set the BTCRPC_CONNECT variable in LocalVagrantConfig.rb to the hostname or IP address of the mastercore VM that you just created.

PostgreSQL Database Server Setup

Although PostgreSQL may be run in a general purpose VM, development at the Mastercoin Foundation has been focused upon using the RDS database service provided by Amazon.

To create your own PostgreSQL instance using Amazon RDS, follow these steps:

1. Make sure you have the AWS EC2 Tools and AWS RDS Tools installed corectly.

2. Make sure you have the Groovy language installed correctly.

3. Copy the environment setup template file.

    $ cp setup-omni-template.sh setup-omni-private.sh
    $ chmod 700 setup-omni-private.sh
   

4. Edit setup-omni-private.sh and make sure to set (at least) the following variables: AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_CREDENTIAL_FILE, PGUSER, PGPASSWORD, OMNI_DB_INSTANCE, OMNI_DB_SEC_GID

5. Run the following commands:

    $ source setup-omni-private.sh
    $ cd rds
    $ ./create-omnidb-instance.sh
    $ ./waitForDBAvailable.groovy
   

6. Use the ./list-rds-instances.sh command to find the hostname of the RDS instance

OmniEngine Server Setup

VM name omniengine

1. Copy DefaultVagrantConfig.rb to LocalVagrantConfig.rb and make sure LocalVagrantConfig.rb contains the correct host, username, and password values for the PostgreSQL and Master Core RPC servers.

2. Create and boot a VM with Vagrant and install and run OmniEngine

    $ vagrant up omniengine [--provider=aws]
   

Omniwallet Server Setup

VM name omniwallet

NOTE: THE OMNIWALLET VM INSTALL IS PRE-ALPHA. It is now working, BUT HAS HAD MINIMAL TESTING. Please give it a try.

FEEDBACK AND/OR PULL REQUESTS WELCOME.

1. Copy DefaultVagrantConfig.rb to LocalVagrantConfig.rb and make sure LocalVagrantConfig.rb contins the correct host, username, and password values for the PostgreSQL and Master Core RPC servers.

2. Create and boot a VM with Vagrant and install and Omniwallet

    $ vagrant up omniwallet [--provider=aws]
   

3. Connect to the omniwallet VM

    $ vagrant ssh omniwallet
   

4. Make sure nginx is running:

    $ sudo service nginx status
   

5. Set an environment variable containing a secret passphrase. It is used to generate salts for individual user IDs, and it needs to be both secret AND not change.

    export OMNIWALLET_SECRET="DontTellAnyoneThis"
   

6. Configure an email server for sending account information.

7. Launch the wrapper (do NOT use sudo)

    $ cd omniwallet
    $ mkdir -p logs
    $ ./app.sh &> logs/appsh.log &
   

8. Use your browser to go to the correct port on the newly installed VM. You should see Omniwallet running there.

Troubleshooting Tips

Vagrant/VirtualBox

1. Make sure that Vagrant (and VirtualBox) are installed correctly. You can use the empty VM provided in the Vagrantfile to quickly test that Vagrant is working correctly. The empty VM does not contain any Master Core or Omni services but offers a quick test that Vagrant and the supporting "provider" are working and configured correctly.

    $ vagrant up empty [--provider=aws]
   

2. If you're running Vagrant from Microsoft Windows make sure that the text files you checked out with Git have UNIX line seperators not MS-DOS/Winodws CRLF line seperators. If you get an error message which complains about \r characters, check your Git configuration. The .gitattributes file in this repository is now configured to ensure that even on Windows, text files have lines separated by LF, not CRLF.

3. If you're using Virtualbox on Windows and your Windows host is becoming unresponsive, you can try uncommenting the following line in Vagrantfile:

    v.customize ["modifyvm", :id, "--cpuexecutioncap", "50"] #limit the use of cpu to 50%
   

4. You can also reduce the number of CPUs used for a particular Virtualbox VM which may also help to reduce load on your host system. The configuration lines in Vagrantfile that you are interested in look like this:

    v.cpus = 2
   

5. If you're on Windows 8.1, don't use the vagrant suspend command because of this VirtualBox bug. (There's also a Vagrant Issue with more specific information and some workarounds.)

Master Core VM

1. Make sure you didn't skip the step to assign an RPC username and password.