/brickhouse

Hive UDF's for the data warehouse

Primary LanguageJavaOtherNOASSERTION

Welcome to the Brickhouse

Build Status

Brickhouse is a collection of UDF's for Hive to improve developer productivity, and the scalability and robustness of Hive queries.

Brickhouse covers a wide range of functionality, grouped in the following packages.

  • collect - An implementaion of "collect" and various utilities for dealing with maps and arrays.

  • json - Translate between Hive structures and JSON strings

  • sketch - An implementation of KMV sketch sets, for reach estimation of large datasets.

  • bloom - UDF wrappers around the Hadoop BloomFilter implementation.

  • sanity - Tools for implementing sanity checks and managing Hive in a production environment.

  • hbase - Experimental UDFs for an alternative way to integrate Hive with HBase.

Requirements:

Brickhouse require Hive 0.9.0 or later; Maven 2.0 and a Java JDK is required to build.

Getting Started

  1. Clone ( or fork ) the repo from https://github.com/klout/brickhouse
  2. Run "mvn package" from the command line.
  3. Add the jar "target/brickhouse-<version number>.jar" to your HIVE_AUX_JARS_FILE_PATH, or add it to the distributed cache from the Hive CLI with the "add jar" command
  4. Source the UDF declarations defined in src/main/resource/brickhouse.hql

See the wiki on Github at https://github.com/klout/brickhouse/wiki for more information.

Also, see discussions on the Brickhouse Confessions blog on Wordpress

http://brickhouseconfessions.wordpress.com/

DOI