"Couldn't match type NullType"
Closed this issue · 1 comments
Hi Sam,
I am trying to save a dataframe and get an odd 'nulltype" error on schema conversion.
"Couldn't match type NullType"
from this bit of code as you will know.
case _ => throw new RuntimeException (s"Couldn't match type $dataType")
The dataFrame is valid, I can print and see the schema find. Seems all the types in the DF schema are covered in the conversion as youd expect :)
Have you come across this before? The dataframe is completely valid of course, meaning every column is valid and has a type - Il keep investigating anyway thought to ask incase you have seen this and know what the root cause might be.
This is on databricks runtime 3.0, using the new Spark 2.2 BTW
Thats basically saying that the datatype is null so it couldnt match it to a datatype and convert it to BQ
You will have to go through your schema to figure out which column is causing this exception