Make Spark Application name configurable
Closed this issue · 0 comments
yruslan commented
Background
Currently, Spark Application name of the Pramen job is set either:
- by Pramen is Spark session does not exist
- by a calling app if Spark session is created by a different app
- only from command line when using Yarn in cluster mode
Spark Application name looks like this:
Pramen - ${pimeline.name}
Feature
Allow customizing Spark Application name by making it configurable
Example
pramen.spark.app.name = "My app - ${pramen.pipeline.name} on ${praman.environment.name}