How to enable Hive Support?
LiuxyEric opened this issue · 2 comments
LiuxyEric commented
I am trying to use spark.sql to read table from hive. However Setl cannot find my table. I read the source code and noticed that when initializing the SparkSession, it didn't enable hive support. So how am I able to read my table from hive?
usually I just called the code below to enable hive support.
SparkSession.builder
.enableHiveSupport
.getOrCreate
qxzzxq commented
You can either add spark.sql.catalogImplementation = "hive"
into your application.conf
(or other configuration files)
# src/main/resources/application.conf
setl.environment = ${app.environment}
setl.config {
spark.app.name = "test_app"
spark.sql.catalogImplementation = "hive"
}
or, you can use Setl.builder().setSparkSession()
to provide your own spark session:
val sparkSession = SparkSession.builder().enableHiveSupport().getOrCreate()
val setl: Setl = Setl.builder()
.setSparkSession(sparkSession)
.withDefaultConfigLoader()
.getOrCreate()
LiuxyEric commented
Thanks for the response !