s390x: test_python2/test_python2_unicode_literals can't assign to () INTERNAL ERROR
Closed this issue · 11 comments
During the build of 19.10b0 in Fedora, the following test failure occurs on s390x (Big Endian) architecture:
======================================================================
FAIL: test_python2 (tests.test_black.BlackTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/black-19.10b0/black.py", line 3754, in assert_equivalent
src_ast = parse_ast(src)
File "/builddir/build/BUILD/black-19.10b0/black.py", line 3686, in parse_ast
return ast27.parse(src)
File "/usr/lib64/python3.8/site-packages/typed_ast/ast27.py", line 50, in parse
return _ast27.parse(source, filename, mode)
File "<unknown>", line 10
SyntaxError: can't assign to ()
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.8/unittest/mock.py", line 1342, in patched
return func(*newargs, **newkeywargs)
File "/builddir/build/BUILD/black-19.10b0/tests/test_black.py", line 543, in test_python2
black.assert_equivalent(source, actual)
File "/builddir/build/BUILD/black-19.10b0/black.py", line 3756, in assert_equivalent
raise AssertionError(
AssertionError: cannot use --safe with this file; failed to parse source file. AST error message: can't assign to () (<unknown>, line 10)
======================================================================
FAIL: test_python2_unicode_literals (tests.test_black.BlackTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib64/python3.8/unittest/mock.py", line 1342, in patched
return func(*newargs, **newkeywargs)
File "/builddir/build/BUILD/black-19.10b0/tests/test_black.py", line 560, in test_python2_unicode_literals
black.assert_equivalent(source, actual)
File "/builddir/build/BUILD/black-19.10b0/black.py", line 3775, in assert_equivalent
raise AssertionError(
AssertionError: INTERNAL ERROR: Black produced code that is not equivalent to the source. Please report a bug on https://github.com/psf/black/issues. This diff might be helpful:
--- src
+++ dst
@@ -1,4 +1,70 @@
Module(
body=
+ ImportFrom(
+ level=
+ 0, # int
+ module=
+ '__future__', # str
+ names=
+ alias(
+ asname=
+ '_unicode_literals', # str
+ name=
+ 'unicode_literals', # str
+ ) # /alias
+ ) # /ImportFrom
+ ImportFrom(
+ level=
+ 0, # int
+ module=
+ '__future__', # str
+ names=
+ alias(
+ asname=
+ None, # NoneType
+ name=
+ 'absolute_import', # str
+ ) # /alias
+ ) # /ImportFrom
+ ImportFrom(
+ level=
+ 0, # int
+ module=
+ '__future__', # str
+ names=
+ alias(
+ asname=
+ 'lol', # str
+ name=
+ 'print_function', # str
+ ) # /alias
+ alias(
+ asname=
+ None, # NoneType
+ name=
+ 'with_function', # str
+ ) # /alias
+ ) # /ImportFrom
+ Expr(
+ value=
+ Constant(
+ value=
+ 'hello', # str
+ ) # /Constant
+ ) # /Expr
+ Expr(
+ value=
+ Constant(
+ value=
+ 'hello', # str
+ ) # /Constant
+ ) # /Expr
+ Expr(
+ value=
+ Constant(
+ value=
+ 'hello', # str
+ ) # /Constant
+ ) # /Expr
type_ignores=
) # /Module
----------------------------------------------------------------------
Ran 119 tests in 18.012s
FAILED (failures=2)
To Reproduce, run the test suite on s390x.
Here is the build log with all the commands: build.log
Here is the root log with all the package versions: root.log
Expected behavior
Test succeed an all architectures.
Environment:
- Version: 19.10b0
- OS and Python version: Linux, Fedora 32 on s390x, Python 3.8.0
Does this bug also happen on master? yes, on 6bedb5c
Additional context The problem does not happen on the same build system with armv7hl or ppc64le.
On master 6bedb5c:
======================================================================
FAIL: test_python2 (tests.test_black.BlackTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/black-master/black.py", line 3754, in assert_equivalent
src_ast = parse_ast(src)
File "/builddir/build/BUILD/black-master/black.py", line 3686, in parse_ast
return ast27.parse(src)
File "/usr/lib64/python3.8/site-packages/typed_ast/ast27.py", line 50, in parse
return _ast27.parse(source, filename, mode)
File "<unknown>", line 10
SyntaxError: can't assign to ()
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.8/unittest/mock.py", line 1342, in patched
return func(*newargs, **newkeywargs)
File "/builddir/build/BUILD/black-master/tests/test_black.py", line 543, in test_python2
black.assert_equivalent(source, actual)
File "/builddir/build/BUILD/black-master/black.py", line 3756, in assert_equivalent
raise AssertionError(
AssertionError: cannot use --safe with this file; failed to parse source file. AST error message: can't assign to () (<unknown>, line 10)
======================================================================
FAIL: test_python2_unicode_literals (tests.test_black.BlackTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib64/python3.8/unittest/mock.py", line 1342, in patched
return func(*newargs, **newkeywargs)
File "/builddir/build/BUILD/black-master/tests/test_black.py", line 560, in test_python2_unicode_literals
black.assert_equivalent(source, actual)
File "/builddir/build/BUILD/black-master/black.py", line 3775, in assert_equivalent
raise AssertionError(
AssertionError: INTERNAL ERROR: Black produced code that is not equivalent to the source. Please report a bug on https://github.com/psf/black/issues. This diff might be helpful:
--- src
+++ dst
@@ -1,4 +1,70 @@
Module(
body=
+ ImportFrom(
+ level=
+ 0, # int
+ module=
+ '__future__', # str
+ names=
+ alias(
+ asname=
+ '_unicode_literals', # str
+ name=
+ 'unicode_literals', # str
+ ) # /alias
+ ) # /ImportFrom
+ ImportFrom(
+ level=
+ 0, # int
+ module=
+ '__future__', # str
+ names=
+ alias(
+ asname=
+ None, # NoneType
+ name=
+ 'absolute_import', # str
+ ) # /alias
+ ) # /ImportFrom
+ ImportFrom(
+ level=
+ 0, # int
+ module=
+ '__future__', # str
+ names=
+ alias(
+ asname=
+ 'lol', # str
+ name=
+ 'print_function', # str
+ ) # /alias
+ alias(
+ asname=
+ None, # NoneType
+ name=
+ 'with_function', # str
+ ) # /alias
+ ) # /ImportFrom
+ Expr(
+ value=
+ Constant(
+ value=
+ 'hello', # str
+ ) # /Constant
+ ) # /Expr
+ Expr(
+ value=
+ Constant(
+ value=
+ 'hello', # str
+ ) # /Constant
+ ) # /Expr
+ Expr(
+ value=
+ Constant(
+ value=
+ 'hello', # str
+ ) # /Constant
+ ) # /Expr
type_ignores=
) # /Module
----------------------------------------------------------------------
Ran 119 tests in 18.123s
FAILED (failures=2)
First I was like LOLWAT but then I was like LOLWAT.
Big endian failing on Black which is itself pure Python and doing relatively little in terms of numeric computation anyway suggests to me that this is probably a typed-ast bug. I have no proof as I don't have an environment to test this in. Miro, do you have an idea where I can get access to a s390x box to look at this?
(also, to be fair, I should mention this will probably land under low-pri)
I don't have an easy to access environment with this, sorry. Low prio for me as well, don't worry.
FWIW, Travis supports s390x now, using that in some open source projects.
But given black is pure Python, maybe it's a CPython bug indeed 🙂
You can also use the openSUSE build service. Currently the mentioned tests fail for s390x and ppc64:
https://build.opensuse.org/package/show/devel:languages:python/python-black
(I am about to send a submit request that skips the tests in question on big endian platforms. But with an account, you can always branch the package and modify the specfile)
Here's a branch to test s390x on Travis CI:
But it's erroring installing dependencies:
- "Could not build wheels for yarl, multidict which use PEP 517 and cannot be installed directly"
- https://travis-ci.org/github/hugovk/black/jobs/720892873
Probably related: python/typed_ast#139
FWIW, I had no issues installing dependencies on a QEMU-emulated s390x.
Running roughly along the lines of
docker run -it --rm --privileged multiarch/qemu-user-static --reset --persistent yes
docker run -it --rm s390x/ubuntu:20.04
apt-get update
apt-get install -y build-essential git python3 python3-dev python-is-python3
apt-get install -y python3-pip
# One can alternatively bootstrap from PyPa in case of any issues
pip3 install -U pip
pip install pipenv pytest
# git clone black and change to its dir etc...
pipenv install --dev
# pipenv shell doesn't quite work, activating manually
source ~/.local/share/virtualenvs/black-Jzvey4U1/bin/activate
python -m unittest
Two tests fail:
<...>
----------------------------------------------------------------------
Ran 148 tests in 203.234s
FAILED (failures=1, errors=1)
(Full output: https://gist.github.com/vytas7/e4d98d66f79e9b4b2307815ca5782985 )
Hmm, yours are different tests than those failing on openSUSE and Fedora
This should be fixed by python/typed_ast#152 (confirmed in python/typed_ast#151 (comment))
And released in typed_ast 1.4.2!
I triggered a new build of Black on s390x, which earlier failed, and it now passes ✅
https://travis-ci.com/github/hugovk/black/builds/211387063
I've created PR #1892 to bump the dependency.