JessicaTegner/pypandoc

Is it possible for pypandoc to be removed from pyspark 2.4?

burrussmp opened this issue · 0 comments

Hello! Our team is encountering an error when installing pyspark 2.4.5 requiring pypandoc to be installed separately in our build.

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [34 lines of output]
      Could not import pypandoc - required to package PySpark
      /usr/local/miniconda/envs/AIModelsEnv/lib/python3.7/site-packages/setuptools/dist.py:720: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
        % (opt, underscore_opt)
      Traceback (most recent call last):
        File "/usr/local/miniconda/envs/AIModelsEnv/lib/python3.7/site-packages/setuptools/installer.py", line 75, in fetch_build_egg
          subprocess.check_call(cmd)
        File "/usr/local/miniconda/envs/AIModelsEnv/lib/python3.7/subprocess.py", line 363, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/usr/local/miniconda/envs/AIModelsEnv/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpz4qeeb6t', '--quiet', 'pypandoc']' returned non-zero exit status 1.

Specs:
python version 3.7
OS Ubuntu 18.04-latest

Based on this commit: apache/spark@bda0669 pypandoc was successfully removed from 3.0.0. Would it be possible to make a revision to 2.4.8 to also remove pypandoc?

Thanks!