mailgun/flanker

parsetab module is not always generated property by ply

rmihael opened this issue · 9 comments

rmihael commented

It seems that there is some race condition with parsetab module that PLY generates. On flanker-0.7.2 we are getting the error that reproduces randomly in identical environments:

    from flanker.addresslib import address
runtime/lib/python2.7/site-packages/flanker/addresslib/address.py:46: in <module>
    from flanker.addresslib.parser import (Mailbox, Url, mailbox_parser,
runtime/lib/python2.7/site-packages/flanker/addresslib/parser.py:162: in <module>
    start='mailbox', errorlog=log)
runtime/lib/python2.7/site-packages/ply/yacc.py:3287: in yacc
    read_signature = lr.read_table(tabmodule)
runtime/lib/python2.7/site-packages/ply/yacc.py:1985: in read_table
    if parsetab._tabversion != __tabversion__:
E   AttributeError: 'module' object has no attribute '_tabversion'

parsetab.py file is correct and has all expected variables. However parsetab.pyc seems to be an empty module. After removing pyc file from flanker.addresslib import address works correctly.

May be it is worth to add a generated parsetab.py into the package distribution?

jlev commented

I also had this problem intermittently when deploying to Heroku. I solved it by adding a pre-start script to my Procfile that imports flanker so the parsetab is generated, and then runs my production app. Seems to work, although it's a bit of a hack.

rmihael commented

We tried something similar, but with no success. It seems that the issue is somehow connected to gevent. We haven't experienced it with non-gevent processes.

jlev commented

Still having this problem, even with the pre-start script, and without using gevent.

Other options:

  • don't write the parser to the filesystem, and just regenerate it on each application start. This will have some performance implications, but it may not be too bad per dabeaz/ply#79
  • write the parser to the filesystem during the pip install process, as suggested by dabeaz/ply#38 (comment)

I've followed the first approach with spacedogXYZ@54d5008 and it appears to work for my use case, although I haven't benchmarked it heavily.

Viech commented

According to this answer this may be a regression from PLY 3.4 to 3.6. (I'm having the same problem as described in that thread with the SWIG-generated Python 2 bindings for GLPK; I have PLY 3.10 installed.)

horkhe commented

As of #188 we ship parsetab along with Flanker.

pudo commented

We're still seeing this error with 0.9.0, but for a different module -- what could be the fix?

AttributeError: module 'flanker.addresslib._parser.url_parsetab' has no attribute '_tabversion'

horkhe commented

Try pinning ply to v3.10, it is the version used to generate parsetab files shipped with Flanker. If the ply version installed is other than v3.10, then it regenerates the parsetab files. That obviously happens in your case and the regenerated files are missing _tabversion for some reason. Probably due to regression in ply that @Viech talked about.

pudo commented

thanks, that appears to fix it. appreciate the help.

horkhe commented

👍 glad to help