add back possibility to specify schema in load_to in spark
Closed this issue · 1 comments
piccolbo commented
Given the limits in schema inference (takes a character for integer, understandably, no support for date, less understandably) maybe it's better to have this back, even if it is less convenient. Maybe also use in tests for safety. Maybe tests will never pass without this.
CREATE TABLE cars (yearMade double, carMake string, carModel string, comments string, blank string)
USING com.databricks.spark.csv
OPTIONS (path "cars.csv", header "true")
piccolbo commented
This clashes with some suspected bug in spark csv, see https://github.com/databricks/spark-csv/issues/184