markupsafe error when using nitter_scraper as library

[rachmadaniHaryono]

Traceback (most recent call last):
 File "/home/r3r/.local/bin/gallery-dl-hydrus", line 8, in <module>
   sys.exit(app())
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/typer/main.py", line 214, in __call__
   return get_command(self)(*args, **kwargs)
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
   return self.main(*args, **kwargs)
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/click/core.py", line 1055, in main
   rv = self.invoke(ctx)
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
   return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
   return ctx.invoke(self.callback, **ctx.params)
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/click/core.py", line 760, in invoke
   return __callback(*args, **kwargs)
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/typer/main.py", line 532, in wrapper
   return callback(**use_params)  # type: ignore
 File "/mnt/ac54dceb-73a5-4f94-b52c-cb7a426c0f29/Documents/gallery-dl/gallery_dl/hydrus.py", line 466, in send_url
   jq.put(DataJob(url))
 File "/mnt/ac54dceb-73a5-4f94-b52c-cb7a426c0f29/Documents/gallery-dl/gallery_dl/job.py", line 694, in __init__
   Job.__init__(self, url, parent)
 File "/mnt/ac54dceb-73a5-4f94-b52c-cb7a426c0f29/Documents/gallery-dl/gallery_dl/job.py", line 27, in __init__
   extr = extractor.find(extr)
 File "/mnt/ac54dceb-73a5-4f94-b52c-cb7a426c0f29/Documents/gallery-dl/gallery_dl/extractor/__init__.py", line 197, in find
   for cls in _list_classes():
 File "/mnt/ac54dceb-73a5-4f94-b52c-cb7a426c0f29/Documents/gallery-dl/gallery_dl/extractor/__init__.py", line 241, in _list_classes
   module = __import__(module_name, globals_, None, (), 1)
 File "/mnt/ac54dceb-73a5-4f94-b52c-cb7a426c0f29/Documents/gallery-dl/gallery_dl/extractor/nitter.py", line 9, in <module>
   from nitter_scraper import schema, tweets
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/nitter_scraper/__init__.py", line 1, in <module>
   from nitter_scraper.nitter import NitterScraper
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/nitter_scraper/nitter.py", line 11, in <module>
   from jinja2 import Environment, FileSystemLoader
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/jinja2/__init__.py", line 12, in <module>
   from .environment import Environment
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/jinja2/environment.py", line 25, in <module>
   from .defaults import BLOCK_END_STRING
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/jinja2/defaults.py", line 3, in <module>
   from .filters import FILTERS as DEFAULT_FILTERS  # noqa: F401
 File "/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/jinja2/filters.py", line 13, in <module>
   from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/home/r3r/.local/pipx/venvs/gallery-dl/lib/python3.9/site-packages/markupsafe/__init__.py)

based on https://stackoverflow.com/a/72747002/1766261 you have to downgrade markupsafe

for example

from setuptools import setup
# ...
setup(
    # ...
    install_requires=[
        # ...
        "nitter-scraper",
        "markupsafe==2.0.1",
        # ...
    ],
    # ...
)

[scottbarnesg]

I am also having this issue. Was able to resolve it by manually pip installing markupsafe v2.0.1 after installing nitter_scraper.

Is there an open PR for this? Could put one together if not.

[rachmadaniHaryono]

looking at current open pr there are nothing for now

but remember that the repo is not active and last commit is jan 11 2021, almost 2 year ago

[btimby]

Hi all, coincidentally, one year later I ran into this same problem.

I solved it by simply removing jinja2, the use of jinja2 is overkill for such a simple case IMO. This removes the import error. FWIW, soft_unicode has been renamed to soft_str in the jinja2 dependency markupsafe. Instead of jinja2, I am using a multiline string as a Python template.

https://github.com/btimby/nitter_scraper/tree/no-jinja2