spark metrics namespace configuration issue
tkrasnoperov opened this issue · 2 comments
According to the documentation at https://spark.apache.org/docs/latest/monitoring.html, under Metrics
, we can set the spark.metrics.namespace
configuration to something like ${spark.app.name}
, which presumably is expanded in the the metrics sink to be the value of that variable. We opened up our spark-defaults.conf
file and added the following line.
spark.metrics.namespace ${spark.app.name}
As a result, our metrics changed from
app_20180802190042_0119_driver_DAGScheduler_stage_runningStages{instance="",job="app-20180802190042-0119",role="driver"} 1
... to this ....
Parse_AUT1_records_driver_DAGScheduler_stage_runningStages{instance="",job="${spark.app.name}",role="driver"} 1
The actual name of the metric changed successfully (Parse_AUT1_records
is in fact the name we expected), but it seems the job
tag did not expand properly (says ${spark.app.name}
). We looked into the source code and are pretty sure there is an issue here in https://github.com/banzaicloud/spark-metrics/blob/master/src/main/scala/com/banzaicloud/spark/metrics/sink/PrometheusSink.scala#L77 on line 77. The name seems to be taken verbatim as a string and not expanded as a variable.
@tkrasnoperov thanks for catching and reporting this. We'll look into this issue.
@tkrasnoperov please give a try using version spark-metrics_2.11-2.3-1.0.1.jar