databricks/Spark-The-Definitive-Guide

Ch9 - Writing to SQL database - SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info)

shanmugavel04 opened this issue · 1 comments

When executing the below command to write to the below path facing the issue

val newPath = "jdbc:sqlite://tmp/my-sqlite.db"
val tablename = "flight_info"
val props = new java.util.Properties
props.setProperty("driver", "org.sqlite.JDBC")
csvFile.write.mode("overwrite").jdbc(newPath, tablename, props)

Driver Stack trace at high level:
Caused by: java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info)
at org.sqlite.core.DB.newSQLException(DB.java:890)
at org.sqlite.core.DB.newSQLException(DB.java:901)
at org.sqlite.core.DB.throwex(DB.java:868)
at org.sqlite.core.NativeDB.prepare(Native Method)
at org.sqlite.core.DB.prepare(DB.java:211)

could you please review and let us know how to resolve this issue further?

I think SQLite will not necessarily work well in a distributed setting because of its requirement to lock the entire database on write. could you please confirm my understanding?