GridControl

GridControl provisions and links multiple servers together to form a Grid.

Files are synchronized, Opinionated Pub/Sub system is implemented, Servers get linked together.

You develop, you play, in a scalable way. The more Servers you add to the Grid, the more calculation power you get.

Welcome to the Grid

5 minutes to get started. By the authors of PM2.

Behind the scenes: GridControl is a network layer built on top of PM2 allowing file synchronization, inter-process communication via an opionated PUB/SUB system and a wide-range system discovery

Features

0 conf Auto discovery system via multiple DNS
0 conf P2P application source sharing
Ecosystem Grid management toolbox (CLI, provisioning, Logs, Monitoring)
Secure Diffie Hellman key exchange and password authentication
Decentralized Each Node can trigger actions executed by another Nodes
Fast Grid interconnected via TCP sockets
Fast Services are started once, then stay alive waiting for inputs. This saves non-negligible startup time
Polyglot Services can be written in any language
Compatible with Amazon Lambda, Google Cloud Functions
Rock Solid PM2 behind the scene for process management and cluster capabilities
And a lot more like Buffering, Retry on Failure...

Creating a Grid

Install your Swiss Army Knife to manage a Grid:

$ npm install grid-cli -g

Now the bin grid is available via the CLI.

Commands

1/ Generate a new Gridfile in the current path that contains grid name, grid password, host and SSH keys:

$ grid new

The Gridfile will look like this:

grid_name     = 'grid-name'
grid_password = 'xxxx'

servers = [
  'user@ip1',
  'user@ip2',
  'user@ip3'
]

ssh_key = '''
1024_PRIVATE_SSH_KEY
'''

ssh_public_key = '''
PUBLIC_SSH_KEY
'''

Change each attribute to the desired value. Note that an SSH client should be running on the defaut port (22) on each remote machine

2/ Provision every host listed in the Gridfile:

$ grid provision

This will copy the SSH pub key and install NVM, Node.js, PM2 and Gridcontrol This installation does not need ROOT access rights at any time

3/ Grid management

$ grid dash

Commands to manage your grid:

# List all nodes linked to the grid
$ grid ls

# Display Dashboard
$ grid dash

# Execute a command on each server
$ grid multissh <bash_command>

# Restart/Recover the current Grid
$ grid restart

# Upgrade Gridcontrol to latest version
$ grid upgrade

# Display realtime logs of all tasks
$ grid logs

# Monitor the whole Grid with Keymetrics
$ grid monitor <secret> <public>
$ grid unmonitor

# Interactively SSH into desired machine
$ grid ssh

Interact with the Grid

Now let's play with the Grid. You can generate a sample project by typing:

$ grid sample [project-name]
$ cd [project-name]
$ npm install

Now you'll have a project that looks like this:

.
├── index.js
├── package.json
└── tasks
    └── get-ip.js

Let's look at the content of tasks/get-ip.js, this is a task that will be propagated in the grid:

var request = require('request');

module.exports = function(context, cb) {
  request('https://api.ipify.org/?format=json', function (error, response, body) {
    if (error)
      return cb(error);

    if (!error && response.statusCode == 200) {
      return cb(null, body);
    }
  });
};

To call this function, look at how it's done in index.js:

var grid = require('grid-api').init({
  instances   : 1,
  env         : {
    NODE_ENV  : 'development'
  }
});

function triggerTask() {
  // 'get-ip' is the filename
  grid.exec('get-ip', function(err, data, server) {
    if (err) {
      console.log(err);
      return false;
    }
	  console.log('Got response from server pub_ip=(%s) priv_ip=(%s):',
                server.public_ip,
                server.private_ip);
    console.log(data);
  });
}

grid.on('ready', function() {
  console.log('Gridcontrol Ready');
  setInterval(triggerTask, 1000);
});

Start the main application:

$ node index.js

At the beginning, only the local gridcontrol will respond. Once the other peers are synchronized they will also process the queries:

Got response from server pub_ip=(88.123.12.21) priv_ip=(10.2.2.1):
$HTML
Got response from server pub_ip=(88.123.12.22) priv_ip=(10.2.2.8):
$HTML
Got response from server pub_ip=(88.125.120.20) priv_ip=(10.2.2.10):
$HTML
Got response from server pub_ip=(88.123.23.42) priv_ip=(10.2.2.12):
$HTML

Distributed processing, on-premise!

Contributing

If you find any issues please open an issue on Github

For Contributing please refer to docs/CONTRIBUTING.md

License

Apache V2 (see LICENSE.txt)