AlexIoannides/pyspark-example-project

Failed TestCase

marouenes opened this issue · 0 comments

ERROR: test_transform_data (tests.test_etl_job.SparkETLTests)
Test data transformer.

Traceback (most recent call last):
File "/home/brix/pyspark-workloads/pyspark-example-project/tests/test_etl_job.py", line 27, in setUp
self.spark, *_ = start_spark()
File "/home/brix/pyspark-workloads/pyspark-example-project/dependencies/spark.py", line 88, in start_spark
spark_sess = spark_builder.getOrCreate()
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/sql/session.py", line 173, in getOrCreate
sc = SparkContext.getOrCreate(sparkConf)
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 349, in getOrCreate
SparkContext(conf=conf or SparkConf())
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 118, in init
conf, jsc, profiler_cls)
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 195, in _do_init
self._encryption_enabled = self._jvm.PythonUtils.getEncryptionEnabled(self._jsc)
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/py4j/java_gateway.py", line 1487, in getattr
"{0}.{1} does not exist in the JVM".format(self._fqn, name))
py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM