lihaoyi/macropy

Test breakage in core/test/exporters

macobo opened this issue · 5 comments

macobo commented

The tests under core/test/exporters/init.py fail for me under python2.7.

======================================================================
ERROR: test_pyc_exporter (macropy.core.test.exporters.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.py", line 33, in test_pyc_exporter
    f = open(__file__ + "/../pyc_cache.py", "a")
IOError: [Errno 20] Not a directory: '/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.pyc/../pyc_cache.py'

======================================================================
ERROR: test_save_exporter (macropy.core.test.exporters.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.py", line 53, in test_save_exporter
    macropy.exporter = SaveExporter(__file__ + "/../exported", __file__ + "/..")
  File "/home/macobo/projects/macropy/macropy/core/exporters.py", line 20, in __init__
    shutil.copytree(root, directory)
  File "/usr/lib/python2.7/shutil.py", line 171, in copytree
    names = os.listdir(src)
OSError: [Errno 20] Not a directory: '/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.pyc/..'

----------------------------------------------------------------------

test_save_exporter can be fixed by properly using the os.path module get parent directory path. This is outlined in the following commit: macobo@543bc63
Even after that test_pyc_exporter fails for me with an AssertionError on the last line.

======================================================================
FAIL: test_pyc_exporter (macropy.core.test.exporters.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.py", line 52, in test_pyc_exporter
    assert (pyc_cache_count, pyc_cache_macro_count) == (8, 5), (pyc_cache_count, pyc_cache_macro_count)
AssertionError: (8, 4)

----------------------------------------------------------------------
lihaoyi commented

Could you try running tests again after it failed with the AssertionError on the last line? I see this error a lot and haven't fixed it, but it always (often?) seems to go away after the first time I run the tests

The exporter tests are all sketchy self-modifying code tests, so running them is not idempotent and can potentially delete your entire source tree if they do the wrong thing (I've already done that once...). It would be great if they didn't do that, but until we have a good virtual filesystem I can test my exported code in, that's gonna be pretty hard.

macobo commented

Updated the original description, since I copied the wrong traceback there.

Running the test twice results in the following errors (on my branch with path fixes):

======================================================================
FAIL: test_pyc_exporter (macropy.core.test.exporters.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.py", line 52, in test_pyc_exporter
    assert (pyc_cache_count, pyc_cache_macro_count) == (8, 5), (pyc_cache_count, pyc_cache_macro_count)
AssertionError: (8, 4)

======================================================================
FAIL: test_null_exporter (macropy.core.test.exporters.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.py", line 14, in test_null_exporter
    assert (pyc_cache_count, pyc_cache_macro_count) == (1, 1)
AssertionError

======================================================================
FAIL: test_pyc_exporter (macropy.core.test.exporters.Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/macobo/projects/macropy/macropy/core/test/exporters/__init__.py", line 25, in test_pyc_exporter
    assert (pyc_cache_count, pyc_cache_macro_count) == (3, 3)
AssertionError

----------------------------------------------------------------------

Since I'm not sure what the exporters are for and how they should work, I'll leave this last AssertionError alone for now. But should I do a pull request for the file paths error?

Re: virtual filesystem - perhaps that complexity could be avoided by using a mocked "filesystem"?

lihaoyi commented

tl, dr: exporters are used to do stuff with your code as it gets macro expanded, either dumping it in some separate folder to use later, or caching it as .pycs to save the repeated compilation thing. These are their docs:

https://github.com/lihaoyi/macropy#exporting-your-expanded-code

Those tests are testing the save-macro-expanded-code-as-.pyc functionality:

  • pyc_cache.py is a simple file that uses a macro defined in pyc_cache_macro.py.
  • pyc_cache_count is a counter counting how many times pyc_cache.py is executed, and pyc_cache_macro_count counts how many times pyc_cache_macro.py is executed.

I'm not sure exactly what's causing those errors, but one possibility is someone is (by default) saving your compiled code in .pyc files, and thus causing these count-how-many-times-code-is-executed tests to break?

Re: virtual filesystem: these tests involve actually running code to see how the exporters play with the filesystem (e.g. saving pre-compiled .pycs, or post-expansion source .py files) including executing the generated source trees to verify correct behavior. I don't know of any vfs or mocked fs that will let me execute python code with it as the working directory and having the correct results.

You can send a PR for the file paths problem first. What exactly was the problem in your case? Does your OS have different path-separators or something?

Thanks for all the effort you're putting in =)

macobo commented

You can send a PR for the file paths problem first. What exactly was the problem in your case? Does your OS have different path-separators or something?

No, but paths in style macropy/macropy/test/peg.py/../peg_json/fail1.json don't seem to be counted as valid, causing IOErrors

Thanks for all the effort you're putting in =)

Glad to be of use. :)

davidshepherd7 commented

I'm seeing this issue as well, could you merge the pull request?