funkygao/hbase-code

hbase source code explaination

hbase source code explaination

Author:	Funky Gao <funky.gao@gmail.com>
Description:	Let's study hbase from it's source code from scratch!

TOC

• 1   Code Metrics
• 2   Tips
• 3   How to start
  • 3.1   Scripts
    • 3.1.1   bin/hbase-config.sh
    • 3.1.2   bin/hbase
  • 3.2   Main Packages
  • 3.3   Server
  • 3.4   Client
  • 3.5   Config
• 4   Logs

1   Code Metrics

Test code excluded:

• 452 java files
• 146,616 java LOC

2   Tips

Enable log debug level and check out the log file and find the main routines.

conf/log4j.properties

3   How to start

3.1   Scripts

3.1.1   bin/hbase-config.sh

Setup the following environ variables for other scripts:

• JAVA_HOME
• HBASE_HOME
• HBASE_CONF_DIR
• HBASE_REGIONSERVERS
• HBASE_BACKUP_MASTERS
• source ${HBASE_CONF_DIR}/hbase-env.sh

3.1.2   bin/hbase

Invoker for other scripts!

Run a dedicated class. e,g:

bin/hbase org.apache.hadoop.hbase.zookeeper.ZooKeeperMainServerArg

Setup classpath and map command to class name and invoke class method:

• shell

  org.jruby.Main ${HBASE_HOME}/bin/hirb.rb

• master

  org.apache.hadoop.hbase.master.HMaster

• regionserver

  org.apache.hadoop.hbase.regionserver.HRegionServer

• thrift

  org.apache.hadoop.hbase.thrift.ThriftServer

• rest

  org.apache.hadoop.hbase.rest.Main

• avro

  org.apache.hadoop.hbase.avro.AvroServer

• migrate

  org.apache.hadoop.hbase.util.Migrate

• hbck

  org.apache.hadoop.hbase.util.HBaseFsck

• zookeeper

  org.apache.hadoop.hbase.zookeeper.HQuorumPeer

• zkcli

  org.apache.zookeeper.ZooKeeperMain

3.2   Main Packages

org.apache.hadoop.hbase
   |
   |- client
   |   |
   |   |- HConnection
   |   |- Get
   |   |- Put
   |   |- Delete
   |   |- Scan
   |   |- Increment
   |   |- Row
   |   |- RowLock
   |   |- HTable
   |   └─ HConnection
   |
   |- filter
   |
   |- io
   |   |
   |   |- hfile
   |   |   |
   |   |   |- HFile
   |   |   └─ BlockCache
   |   |
   |   └─ HeapSize
   |
   |- ipc
   |
   |- mapreduce(通过mapreduce实现运维工具)
   |
   |- master
   |   |
   |   |- HMaster
   |   └─ LoadBalancer
   |
   |- regionserver
   |   |
   |   |- HRegion
   |   |- HRegionServer
   |   |- MemStore
   |   |- Store
   |   └─ StoreFile
   |
   |- util
   |
   └─ zookeeper
       |
       |- ClusterStatusTracker
       |- MetaNodeTracker
       └─ RegionServerTracker

3.3   Server

org.apache.hadoop.hbase.master.HMaster

org.apache.hadoop.hbase.regionserver.HRegionServer

org.apache.hadoop.hbase.regionserver.HRegion

3.4   Client

org.apache.hadoop.hbase.client.HTable

3.5   Config

org.apache.hadoop.hbase.HBaseConfiguration

It will read hbase-default.xml then hbase-site.xml using the current Java classpath.

4   Logs

• master log
• region server log
• zookeeper log