Hadoop cluster

Made for an assignement, not fit for anything remotely serious.

Getting started

You need at least one Docker Swarm manager

# On manager
docker swarm init

This will print a command that you'll need to run on all other nodes.

We use Docker Deploy to start and manage the services.

docker stack deploy --compose-file=docker-compose.yml [name of the cluster]

docker stack rm [name of the cluster]

docker service scale [service name]=[number of desired replicas]

You can access a visualizer by using a brower to go to [IP]:8080 where IP is a node in the cluster (any node should work).

To get an aggregated log of all the containers of a specific service, run

docker service logs [service name]

Docker service names can be found with

docker service ls

Executing docker ps will list all containers running on one node.

docker exec -it [container name] bash