Data Transfer from one datastore to another.

[sabarnwa]

In the medium article, it was mentioned that metastore can act as a medium for converting the dataType of one datastore to another. Can someone give an example for the same.

Also, some documentation/example will be really helpful in using metacat by Thrift.

Thanks,
Sakshi

[ajoymajumdar]

System configuration metacat.type.converter drives what converter to use when showing the types of a column. We set it to 'com.netflix.metacat.connector.pig.converters.PigTypeConverter'. So all the table info is responded with types in PIG.
To override this default at a request level, you need to set the header 'X-Netflix.data.type.context'. For example, if you are reading a mysql table and want the types in hive. You will set the header - 'X-Netflix.data.type.context' - to 'hive'.

Thirft interface can be enabled only for hive catalogs. You can see the configuration metacat.thrft.port in https://github.com/Netflix/metacat/blob/master/metacat-functional-tests/metacat-test-cluster/etc-metacat/catalog/embedded-fast-hive-metastore.properties. You can use class org.apache.hadoop.hive.ql.metadata.Hive to query a hive thrift server. Example: https://github.com/Netflix/metacat/blob/master/metacat-functional-tests/src/functionalTest/groovy/com/netflix/metacat/MetacatSmokeThriftSpec.groovy

[sabarnwa]

Thanks Ajay. It helped.