/ambari-shell

CLI for Apache Ambari

Primary LanguageJavaApache License 2.0Apache-2.0

Ambari Shell

Ambari is aimed to help provision/manage/monitor Hadoop clusters. Although it has a REST API, right now the only way to use Ambari is by using the web-application.

Ambari Shell's aim is to provide an interactive command line tool which supports:

  • all functionality available through Ambari web-app
  • context aware command availability
  • tab completion
  • required/optional parameter support

Architecture

To implement Ambari Shell the following options were evaluated:

  • python: seems a natural fit, as a few components of Ambari are written in python. Also an Ambari python shell was announced on Ambari wiki page, but after a discussion with committers, it turned out to be abandoned path. The official Ambari python client hasn't been released, so the python option is eliminated.
  • go: go would solve nicely the usual dependency hell (pip, gem, classpath, you name it), but unfortunately our team has no experience with it.
  • java: The write-once-run-everywhere nature provides support for all OS. Spring Shell seems a natural fit as a base framework.

Spring Shell is also battle tested in various Spring projects including:

  • Spring-Roo: lightweight cli tool to aim Rapid Application Development
  • Spring-XD a user-friendly front end for the REST API of Spring XD. Spring XD is a unified, distributed, service for data ingestion, real time analytics, batch processing, and data export.
  • Spring-Rest-Shell a command-line shell that aims to make writing REST-based applications easier. Spring Rest-Shell itself would be enough to communicate against Ambari REST API, but we wanted a more Domain Specific Language (DSL) nature of the command structure.

Installation and usage

Ambari Shell is distributed as a single-file executable jar. So no ClassNotFound should happen. The uber jar is generated with the help of spring-boot-maven-plugin available at Spring-Boot. Spring-Boot also provides a helper to launch those jars: JarLauncher.

After compiling the project, the shell is ready to use (make sure you use Java 7 or above).

java -jar ambari-shell/target/ambari-shell-1.3.0-SNAPSHOT.jar --ambari.server=localhost --ambari.port=8080 --ambari.user=admin --ambari.password=admin

The --ambari options can be omitted if they are the default values, otherwise you only need to specify the difference, e.g if only the port is different

java -jar ambari-shell/target/ambari-shell-1.3.0-SNAPSHOT.jar --ambari.port=49178

    _                _                   _  ____   _            _  _
   / \    _ __ ___  | |__    __ _  _ __ (_)/ ___| | |__    ___ | || |
  / _ \  | '_ ` _ \ | '_ \  / _` || '__|| |\___ \ | '_ \  / _ \| || |
 / ___ \ | | | | | || |_) || (_| || |   | | ___) || | | ||  __/| || |
/_/   \_\|_| |_| |_||_.__/  \__,_||_|   |_||____/ |_| |_| \___||_||_|

Welcome to Ambari Shell. For command and param completion press TAB, for assistance type 'hint'.

Implemented Commands

  • blueprint add - Add a new blueprint with either --url or --file
  • blueprint defaults - Adds the default blueprints to Ambari
  • blueprint list - Lists all known blueprints
  • blueprint show - Shows the blueprint by its id
  • cluster assign - Assign host to host group
  • cluster autoAssign - Automatically assigns hosts to different host groups base on the provided strategy
  • cluster build - Starts to build a cluster
  • cluster create - Create a cluster based on current blueprint and assigned hosts
  • cluster delete - Delete the cluster
  • cluster preview - Shows the currently assigned hosts
  • cluster reset - Clears the host - host group assignments
  • configuration download - Downloads the desired configuration
  • configuration modify - Modify the desired configuration
  • configuration set - Sets the desired configuration
  • configuration show - Prints the desired configuration
  • debug off - Stops showing the URL of the API calls
  • debug on - Shows the URL of the API calls
  • exit - Exits the shell
  • hello - Prints a simple elephant to the console
  • help - List all commands usage
  • hint - Shows some hints
  • host components - Lists the components assigned to the selected host
  • host focus - Sets the useHost to the specified host
  • host list - Lists the available hosts
  • quit - Exits the shell
  • script - Parses the specified resource file and executes its commands
  • services components - Lists all services with their components
  • services list - Lists the available services
  • services start - Starts a service/all the services
  • services stop - Stops a service/all the running services
  • tasks - Lists the Ambari tasks
  • users changepassword - Change a user's password
  • version - Displays shell version

Please note that all commands are context aware - and are available only when it makes sense. For example the cluster create command is not available until a blueprint has been added or selected. A good approach is to use the hint command - as the Ambari UI, this will give you hints about the available commands and the flow of creating or configuring a cluster.

You can always use TAB for completion or available parameters.

Example:

Once you logged in you can say hello.

                .-.._
          __  /`     '.
       .-'  `/   (   a \
      /      (    \,_   \
     /|       '---` |\ =|
    ` \    /__.-/  /  | |
       |  / / \ \  \   \_\
       |__|_|  |_|__\

Initially there are no blueprints available - you cn add blueprints from file or URL. For your convenience we have added 2 blueprints as defaults. You can get these blueprints by using the blueprint defaults command. The result is the following:

  BLUEPRINT              STACK
  ---------------------  -------
  hdp-singlenode-2.2     HDP:2.2
  multi-node-hdfs-yarn   HDP:2.2
  hdp-multinode-default  HDP:2.2
  lambda-architecture    HDP:2.2
  single-node-hdfs-yarn  HDP:2.2

Once the blueprints are available you can use them to create a cluster. You can use the following command: cluster build --blueprint single-node-hdfs-yarn. Now that the blueprint is selected you have to assign the hosts to the available host groups.

Use cluster assign --hostGroup host_group_1 --host server.ambari.com.

You can always cluster reset or cluster preview to modify or check the configuration.

  HOSTGROUP     HOST
  ------------  -----------------
  host_group_1  server.ambari.com

Once you are happy with host assignment, you can choose cluster create to start building the cluster. Progress can be checked either in the Ambari UI or using the tasks command.

  TASK                        STATUS
  --------------------------  -------
  HISTORYSERVER INSTALL       QUEUED
  ZOOKEEPER_SERVER START      PENDING
  ZOOKEEPER_CLIENT INSTALL    PENDING
  HDFS_CLIENT INSTALL         PENDING
  HISTORYSERVER START         PENDING
  NODEMANAGER INSTALL         QUEUED
  NODEMANAGER START           PENDING
  ZOOKEEPER_SERVER INSTALL    QUEUED
  YARN_CLIENT INSTALL         PENDING
  NAMENODE INSTALL            QUEUED
  RESOURCEMANAGER INSTALL     QUEUED
  NAMENODE START              PENDING
  RESOURCEMANAGER START       PENDING
  DATANODE START              PENDING
  SECONDARY_NAMENODE START    PENDING
  DATANODE INSTALL            QUEUED
  MAPREDUCE2_CLIENT INSTALL   PENDING
  SECONDARY_NAMENODE INSTALL  QUEUED

Summary

To sum it up in less than two minutes watch this video:

asciicast