Does the application uses any cached data?
Rap70r opened this issue · 0 comments
Rap70r commented
Hello,
We need to enable dynamic allocation. One of the spark dynamic allocation configurations is the following:
spark.dynamicAllocation.cachedExecutorIdleTimeout
This has default value: infinity
Does the application uses any type of caching or is it fine if we set the above configuration to 5 minutes for example?
If yes, is it possible to disable caching?
Can the executors be removed safely when idle or does the application depend on already existing executors?
Thank you