A simple way to test Hadoop and Hive using maven.
Run these commands initially or whenever your project is cleaned.
Download Hadoop (into the maven target directory)
mvn wagon:download-single
Extract Hadoop (into the maven target directory)
mvn exec:exec
- hadoop 0.20.X in your path ie /usr/bin/hadoop
- set your HADOOP_HOME environment variable to a hadoop distribution
- hadoop tar extracted to $home/hadoop/hadoop-0.20.2-local
Hive_test gives us an embedded Hive including an embedded Derby database, and a local HiveThriftService. This allows us to create unit tests very easily.
public class ServiceExampleTest extends HiveTestService {
public ServiceExampleTest() throws IOException {
super();
}
public void testExecute() throws Exception {
Path p = new Path(this.ROOT_DIR, "afile");
FSDataOutputStream o = this.getFileSystem().create(p);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(o));
bw.write("1\n");
bw.write("2\n");
bw.close();
client.execute("create table atest (num int)");
client.execute("load data local inpath '" + p.toString()
+ "' into table atest");
client.execute("select count(1) as cnt from atest");
String row = client.fetchOne();
assertEquals(row, "2");
client.execute("drop table atest");
}
}