WebPlatform server environment configuration

Making a new salt-master

Send a copy of init.sh from webplatform/ops in salt-master/ to an empty Ubuntu 14.04 VM and run the script as root.

To deploy web apps, we are currently assuming that the salt master will host all the code repositories along with the dependency management tools installed.

In order to allow us to have a VM to prepare each web app, the following has to be run on the salt master. We will eventually create a VM specifically for that later in this project.

Security groups configuration

All services that requires TCP/UDP connectivity should have, in the state file or configuration file, an annotation about it.

To find all security groups that requires communication, search the files for notes that contains ## SecurityGroup. This string can be as a comment, or at the end of a configuration line.

For example, MediaWiki Settings.php has a line like this:

$wpdUdp2logDest = 'salt:8420'; ## SecurityGroup port: UDP 8420 @salt demux.py

All security groups are expected to allow local network communication exclusively by allowing only incoming traffic from 10.10.10.0/24 or any other local network IP class available.

As for inter network communication, we should maintain explicitly which security groups are allowing outside communication.

Connecting with SSH

Not ALL VMs in an environment has a publicly available IP address. To connect to them, you have to configure your local SSH client to use the salt master as a proxy. SSHing to a VM is done in the following way:

ssh app1.staging.wpdn

The app1.staging.wdpn is visible from the salt master through either a hosts file entry or a local DNS server but your own local machine cannot see it. This is where your local SSH config comes into play.

To access them, add to your ~/.ssh/config:

## Use salt as a Jump box
##
## Reference:
##   - http://serverfault.com/questions/337274
##   - https://wikitech.wikimedia.org/wiki/Help:Access#Using_ProxyCommand_ssh_option
##   - http://blog.pluralsight.com/linux-ssh-jumpbox
Host staging.wpdn
  Hostname salt.webplatformstaging.org
  ProxyCommand none
Host *.staging.wpdn
  ProxyCommand ssh -e @ -o StrictHostKeyChecking=no -a -W %h:%p staging.wpdn

Config stashes

Bookmarks

Useful related documentation

Maintenance commands

Here are a few commands that can be done with Salt Stack.

Every commands here can be run manually. Ideally there should be always a salt state to enforce anything permanent. But sometimes we have to act quickly and update the states later.

Rewrite a setting in one config file

salt app* file.replace /etc/php5/apache2/php.ini pattern='expose_php = On' repl='expose_php = Off'
Get to know what are the grants for one user

salt db* mysql.user_grants accounts '%'
Reload apache2 config (or any service)

salt app* service.reload apache2
Can we write to the filesystem?

salt bots cmd.run "touch /root/TestIfWeCabWrite && echo 'OK' || echo 'Writing to filesystem failed'" salt bots cmd.run "rm /root/TestIfWeCabWrite && echo 'OK' || echo 'Writing to filesystem failed'"
Upgrade Operating System packages (If it has a service, it’ll also handle its restart)

salt app* pkg.upgrade
Delete a user

salt * user.delete foobar remove=True force=True
New salt minion

nova boot --image Ubuntu-14.04-Trusty --user-data /srv/opsconfigs/userdata.txt --key_name renoirb-staging --flavor lightspeed --security-groups default,all,frontend app1 salt-key -y -a app1 salt app1 test.ping salt app1 grains.get level salt app1 state.highstate
Assing a floating IP

Assuming our VM app1 has private IP 10.10.10.170 and we want floating IP of 173.236.254.223

We need the floating IP id identifier (e.g. foo) in the 4 you’ll get. Technically the second value should be empty, that’s where we should see our VM private IP once its done. Identifier will be a UUID string
```
neutron floatingip-list | grep 173.236.254.223
| foo |      |  173.236.254.223  |    |
```

We need to know which ethernet adapter to bind the floating IP to. We need the first value.

neutron port-list | grep 10.10.10.170
| bar |      | fa:16:3e:c1:6c:a0 | {"subnet_id": "...", "ip_address": "10.10.10.170"}

We do have what we need to assign, enter them in this order:

neutron floatingip-associate --fixed-ip-address 10.10.10.170 foo bar

Before updating salt states, ensure you have all IP addresses

Get on any VM, locally (notice salt-call) what are the IP addresses it has from its neighboors
```
salt-call mine.get \* network.ip_addrs
```
Ask every VMs what they have (basically it asks every node to ask themselves to their neighboors what they have)
```
salt \* mine.get \* network.ip_addrs
```
Force an update of the mine
```
salt \* mine.flush
salt \* mine.update
```
If the previous flush didn’t work, in that case lets restart all salt-minions so it’ll listen to what we mean
```
salt \* service.restart salt-minion
salt \* saltutil.sync_all
salt \* mine.update
```

Get the IP addresses of a VM

All adapters
```
salt \* grains.get ip4_interfaces
```
Only the eth0 adapter
```
salt \* grains.get ip4_interfaces:eth0
```

Create a database and privileges on masterdb. If there is replication, the secondary MySQL servers with replication will get the changes by themselves

Create a database

salt -G 'roles:masterdb' mysql.db_create dbname utf8 utf8_general_ci

Add database user and privileges

salt -G 'roles:masterdb' mysql.user_create dbuser '%' somepass
salt -G 'roles:masterdb' mysql.grant_add 'ALL' 'dbname.*' 'dbuser' '%'

Remove a file from multiple machines

salt * file.remove /etc/monit/conf.d/exim4.conf

hexmode/salt-states