Utility library to convert term to BigQuery compatible json based on
avro schema
Avro |
BigQuery |
null |
could be used as part of union type |
boolean |
BOOLEAN |
int |
INTEGER |
long |
INTEGER |
float |
FLOAT |
double |
FLOAT |
bytes |
BYTES |
string |
STRING |
Avro |
BigQuery |
record |
RECORD |
enum |
STRING |
array |
REPEATED |
map |
REAPEATED RECORD with key and value |
union |
only supports union with max 2 options and max 1 non nullable type. |
fixed |
BYTES |
Avro |
BigQuery |
date |
DATE |
time-millis |
TIME |
time-micros |
TIME |
timestamp-millis |
TIMESTAMP |
timestamp-micros |
TIMESTAMP |
Avro |
BigQuery |
{"name": "created_at", "type": { "type": "string", "bq.transform": "iso8601_to_timestamp" }} |
{ "mode": "REQUIRED", "name": "created_at", "type": "TIMESTAMP" } |
{"name": "detail","type": { "type": "string", "bq.transform": "any_to_json" }} |
{ "mode": "REQUIRED", "name": "detail", "type": "STRING" } |
Avro |
BigQuery |
{"name": "full_name","type": { "type": "string", "bq.source_name": "full-name" }} |
{ "mode": "REQUIRED", "name": "full_name", "type": "STRING" } |