Remediate High Spark vulnerability
stahiri opened this issue · 0 comments
stahiri commented
GitHub reports the following High severity vulnerability in HumanCellAtlas/data-consumer-vignettes. Please remediate by the end of Q3 Milestone 3.
Description
org.apache.spark:spark-core_2.11
Suggested Remediation
Upgrade org.apache.spark:spark-core_2.11 to version 2.3.3 or later.
Details
Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. This includes cached blocks that are fetched by disk (controlled by spark.maxRemoveBlockSizeFetchToMem); in SparkR, using parallelize; in Pyspark, using broadcast and parallelize; and use of python udfs.