serengeti-ws: Serengeti's Web Service and CLI

Serengeti is an open source project initiated by VMware to enable the rapid deployment of an Apache Hadoop cluster on a virtual platform.

This repository contains the code for the Serengeti Web Service and CLI.

Getting Started

To jump into using Serengeti, follow our ![Installation Guide] (https://github.com/vmware-serengeti/doc/blob/master/installation_guide_from_source_code.txt).

Serengeti Web Service

Serengeti Web Service provides a RESTful API for resources managment and hadoop cluster management running on vSphere. It works as a proxy to invoke the Serengeti provisioning engine and return fine-grained process execution status to the caller.

Web Application architecture

REST APIs

Method	URL Template	Request	Response	Description
GET	/hello	void	void
GET	/tasks	void	List of TaskRead	List all tasks
GET	/task/{taskId}	taskId	TaskRead	Get task by task id
POST	/clusters	ClusterCreate	Redirect to /task/{taskId}	Create cluster
GET	/clusters	void	List of ClusterRead	List all clusters
GET	/cluster/{clusterName}	clusterName	ClusterRead	Get cluster by name
GET	/cluster/{clusterName}/spec	clusterName	ClusterCreate	Get cluster specification by name
PUT	/cluster/{clusterName}	clusterName; state=start/stop/resume	Redirect to /task/{taskId}	Operate a cluster: start; stop or resume a failed creation
PUT	/cluster/{clusterName}/nodegroup/{groupName}	clusterName; groupName; instanceNum	Redirect to /task/{taskId}	Resize cluster with a new instance number
DELETE	/cluster/{clusterName}	clusterName	Redirect to /task/{taskId}	Delete a cluster by name
POST	/resourcepools	ResourcePoolAdd	void	Add a resource pool
GET	/resourcepools	void	List of ResourcePoolRead	List all resource pools
GET	/resourcepool/{rpName}	rpName	ResourcePoolRead	Get resource pool by name
DELETE	/resourcepool/{rpName}	rpName	void	Delete a resource pool by name
POST	/datastores	DatastoreAdd	void	Add a datastore
GET	/datastores	void	List of DatastoreRead	List all datastores
GET	/datastore/{dsName}	dsName	DatastoreRead	Get datastore by name
DELETE	/datastore/{dsName}	dsName	void	Delete a datastore by name
POST	/networks	NetworkAdd	void	Add a network
GET	/networks	details=true/false	List of NetworkRead	List all networks
GET	/network/{networkName}	networkName; details=true/false	NetworkRead	Get a network by name
DELETE	/network/{networkName}	networkName	void	Delete a network by name
GET	/distros	void	List of DistroRead	List all distros
GET	/distro/{distroName}	distroName	DistroRead	Get a distro by name

Authentication

Spring security In-Memory Authentication is used for Serengeti Authentication and user management.

We don't provide html or JSPs for login, instead, the spring default standard URL is used. User need to set j_username and j_password and then POST login information to URL /serengeti/j_spring_security_check for authentication.

Navigate to URL /serengeti/j_spring_security_logout means logout, and the session will be removed from server side.

Session timeout

If the user session is idle more than 30 mintues, server will invalidate the session. The timeout can be set in /opt/serengeti/tomcat6/webapps/serengeti/WEB-INF/web.xml in following format:

<session-config>
  <session-timeout>30</session-timeout>    <!-- 30 minutes -->
</session-config>

Add/Delete a User in Serengeti

Add or delete user at /opt/serengeti/tomcat6/webapps/serengeti/WEB-INF/spring-security-context.xml file, user-service element. Following is a sample to add one user into user-service.

<authentication-manager alias="authenticationManager">
  <authentication-provider>
     <user-service>
        <user name="serengeti" password="password" authorities="ROLE_ADMIN"/>
        <user name="joe" password="password" authorities="ROLE_ADMIN"/>
     </user-service>
  </authentication-provider>
</authentication-manager>

The authorities value should define user role in Serengeti, but in M2, it’s not used.

Modify User Password

Change password is in the same element at /opt/serengeti/tomcat6/webapps/serengeti/WEB-INF/spring-security-context.xml file.

<authentication-manager alias="authenticationManager">
  <authentication-provider>
     <user-service>
        <user name="serengeti" password="password" authorities="ROLE_ADMIN"/>
        <user name="joe" password="welcome1" authorities="ROLE_ADMIN"/>
     </user-service>
  </authentication-provider>
</authentication-manager>

Serengeti CLI

The CLI is built using the Spring Shell project. The CLI supports an interactive shell mode, a command line mode, and execution of script files. After compiling, you can find the jar file under cli/target directory.

Shell mode: java -jar serengeti-cli-0.1.jar. It supports tab key based command hint and completion. It supports history by up/down arrows.
Command line mode: java -jar serengeti-cli-0.1.jar "command1;command2..."
Execution of script file: in shell mode or command line mode, execute "script --file scriptFileName". The shell history file named cli.log will help to generate the script file.

More details can be found at cli/README.md. Some sample cluster creation specification files can be found at cli/samples.

Build serengeti webservice and cli

You need to have maven installed. Please reference our ![Installation Guide] (https://github.com/vmware-serengeti/doc/blob/master/installation_guide_from_source_code.txt) to install maven if you don't have it.

If your server is behind a proxy, add following config into maven-settings.xml and make sure the right proxy setting.

  <proxies>
    <proxy>
      <active>true</active>
      <protocol>http</protocol>
      <host>proxy.domain.com</host>
      <port>3128</port>
      <nonProxyHosts>*.domain.com</nonProxyHosts>
    </proxy>
  </proxies>

then cd $SERENGETI_HOME/src/serengeti-ws

  mvn package -s maven-settings.xml

Downloading

You can download a complete Serengeti distribution on projectserengeti.org There are no published maven artifacts at this time, stay tuned.

Documentation

You can find a link to the user guide here.

Issue Tracking

Serengei's JIRA issue tracker can be found here

Contributing

Pull requests are welcome; see the contributor guidelines.

Staying in touch

Follow @VMWserengeti on Twitter. You can get help with technical issues, ask questions, and share your experiences with Serengeti on the mailing list serengeti-user. To discuss the development of Serengeti sign up on the serengeti-dev mailing list.

themethod/serengeti-ws