Option to turn Spark UI back on ?

[leobenkel]

Is there an option to turn Spark UI back on ? To do a big more heavy debugging.

[leobenkel]

I found an easy way:

  /**
    * To help debug with the Spark UI
    */
  def enableSparkUI: Boolean = {
    false
  }

  final override def conf: SparkConf = {
    if (enableSparkUI) {
      super.conf
        .set("spark.ui.enabled", "true")
        .set("spark.ui.port", "4050")
    } else {
      super.conf
    }
  }

Add that in your main class trait

and then for the test you want to UI just override enableSparkUI = true