This system is meant to help configure a cluster for the EDH lab we’re doing with NiFi etc.
It is assumed a cluster has been created which needs configuring.
The hostnames have the format http://PREFIX-dn0...... where -dn0.... can be -dn0, -dn1, -dn2, -mn0, followed by anything else.
There are some bash scripts in bin/ which will connect via ssh with a password
There is a dockerized system (I use this to avoid python conflicts). Building it and running it by hand are described later
If you’ve been given the cluster, and you have the cluster prefix, the cluster’s CM hostname, the username and password then you need to execute the following:
bin/new_cluster.sh- This will ssh into each machine on the cluster and install java 1.8 as well as some CSDS. It will ask for your password 5 times
- a docker image (
tobyhferguson/cm_config) - to ensure that java has been configured throughout the cluster, and to get Kudu and NiFi properly installed. It takes about 8 mins to execute.
bin/new_cluster.sh CM_URL username
You can run the docker stuff directly - and this is what is easiest to do:
docker --rm -i -t tobyhferguson/cm_config CM_URL username password
bin/new_cluster.sh http://cl64863-mn0.eastus2.cloudapp.azure.com:7180/ cloudera docker --rm -i -t tobyhferguson/cm_config http://cl64863-mn0.eastus2.cloudapp.azure.com:7180/ cloudera Cloudera_123
If you want to build the docker image (you don’t have to! Its on the web so you don’t need this step)
docker build -t tobyhferguson/cm_config .