rcongiu/Hive-JSON-Serde

your code does not work with hbase ...why?

davidmontgom opened this issue · 1 comments

Why does your code not work with hbase? Is it the col mapping? I verified your example code works but clearly not with hbase.

HBASE
create 'MY_TABLE', {NAME=>'TEST', VERSIONS => '3', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}'
put 'MY_TABLE' , '1' ,'TEST:data' ,'{"age":42, "gender":"male"}'

add jar hdfs:///lib/json-serde-1.3.8-jar-with-dependencies.jar;
add jar hdfs:///lib/json-udf-1.3.8-jar-with-dependencies.jar;
hive> CREATE EXTERNAL TABLE dude ( key string, data map<string,string>) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,TEST:') TBLPROPERTIES ('hbase.table.name' = 'MY_TABLE');
OK
Time taken: 4.068 seconds
hive> select * from dude;
OK
Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.hbase.ResultWritable cannot be cast to org.apache.hadoop.io.Text
Time taken: 1.794 seconds
hive>

I mean the data does exist:

hive> CREATE EXTERNAL TABLE uber_test (id string,colname map<string,string>) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,TEST:") TBLPROPERTIES("hbase.table.name" = "MY_TABLE");
OK
Time taken: 0.116 seconds
hive> select * from uber_test;
OK
1 {"data":"{"age":42, "gender":"male"}"}
Time taken: 0.206 seconds, Fetched: 1 row(s)

PS I also tried the below:

hive> CREATE EXTERNAL TABLE dude (key string, data structage:int,gender:string ) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,TEST:') TBLPROPERTIES ('hbase.table.name' = 'MY_TABLE');
OK
Time taken: 0.133 seconds
hive> select * from dude;
OK
Failed with exception java.io.IOException:java.lang.ClassCastException: org.apache.hadoop.hive.hbase.ResultWritable cannot be cast to org.apache.hadoop.io.Text
Time taken: 0.244 seconds