exo-lang/exo

Ran out of input when loading grammar on MacOS

SamirDroubi opened this issue · 2 comments

I see the following crash very frequently: action

File "/Users/runner/hostedtoolcache/Python/3.10.11/arm64/bin/exocc", line 5, in <module>
    from exo.main import main
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/exo/__init__.py", line 1, in <module>
    from .API import (
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/exo/API.py", line 8, in <module>
    import exo.LoopIR_scheduling as scheduling
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/exo/LoopIR_scheduling.py", line 4, in <module>
    from .LoopIR import (
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/exo/LoopIR.py", line 581, in <module>
    from . import LoopIR_pprint
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/exo/LoopIR_pprint.py", line 8, in <module>
    from yapf.yapflib.yapf_api import FormatCode
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf/__init__.py", line 41, in <module>
    from yapf.yapflib import yapf_api
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf/yapflib/yapf_api.py", line 38, in <module>
    from yapf.pyparser import pyparser
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf/pyparser/pyparser.py", line 44, in <module>
    from yapf.yapflib import format_token
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf/yapflib/format_token.py", line [23](https://github.com/exo-lang/exo/actions/runs/7769842605/job/21189335644#step:10:24), in <module>
    from yapf.pytree import pytree_utils
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf/pytree/pytree_utils.py", line 30, in <module>
    from yapf_third_party._ylib2to3 import pygram
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf_third_party/_ylib2to3/pygram.py", line 29, in <module>
    python_grammar = driver.load_grammar(_GRAMMAR_FILE)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf_third_party/_ylib2to3/pgen2/driver.py", line [25](https://github.com/exo-lang/exo/actions/runs/7769842605/job/21189335644#step:10:26)2, in load_grammar
    g.load(gp)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yapf_third_party/_ylib2to3/pgen2/grammar.py", line 95, in load
    d = pickle.load(f)
EOFError: Ran out of input

We should look into it and fix it.

We might need to force serial ( https://pre-commit.com/#hooks-require_serial )

I applied this fix for now https://github.com/exo-lang/ExoBLAS/pull/85/files which seems to make the crash disappear. We should keep an eye to see if newer releases fix this issue and then we can bump the library version.