holdenk/spark-testing-base

SqlContext test cases not running..

Opened this issue · 0 comments

When I run a sample test:

test("simple test") {
val sqlCtx = sqlContext
import sqlCtx.implicits._
val input1 = sc.parallelize(List(1, 2, 3)).toDF
assertDataFrameEquals(input1, input1) // equal

  val input2 = sc.parallelize(List(4, 5, 6)).toDF
  intercept[org.scalatest.exceptions.TestFailedException] {
    assertDataFrameEquals(input1, input2) // not equal
  }
}

I get following in logs and this test case is not even executed.

18/04/26 12:00:34 WARN SparkContext: Using an existing SparkContext; some configuration may not take effect.
18/04/26 12:00:38 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
18/04/26 12:00:38 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
18/04/26 12:00:39 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException