Cant change datatype with transforms
ktopcuoglu opened this issue · 1 comments
ktopcuoglu commented
We have a string field in json files with %99 int data type.
Want to load it as string into transient table, then using safe_cast transform to persist this fields to final table as int64.
tv072: safe_cast(t.tv072 as int64)
tv050: safe_cast(t.tv050 as float64)
(these fields are string in template table.)
"Errors":[{"Message":"Invalid schema update. Field tv072 has changed type from STRING to INTEGER","Reason":"invalid"}]
I think this situation because of both transient and target table use Schema.Template
table as template. How can we fix this?
adranwit commented
You can define a separate template for a transient table and separate template for dest table
Check https://github.com/viant/bqtail/tree/master/tail#data-ingestion-rules
dest.transient.template
dest.schema.template