//:protobuf_python with use_fast_cpp_protors does not compile with python3.7 due to python's breaking changes in C API
qzmfranklin opened this issue · 4 comments
qzmfranklin commented
How to reproduce:
git clone https://github.com/qzmfranklin/torrent
git submodule update --init --recursive
bazel build third_party/cc/protobuf:protobuf_python --define use_fast_cpp_protos=true
The bazel -s output:
ERROR: /home/zhongming/git/torrent/third_party/cc/protobuf/BUILD:668:1: C++ compilation of rule '//third_party/cc/protobuf:python/google/protobuf/pyext/_message.so' failed (Exit 1): clang failed: error executing command
(cd /home/zhongming/.cache/bazel/_bazel_zhongming/252308e7db9d6378926a9d84bb91921c/execroot/torrent && \
exec env - \
PWD=/proc/self/cwd \
/opt/clang/5.0.1/bin/clang -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -fno-omit-frame-pointer -isystem/opt/clang/5.0.1/include/c++/v1 -nostdinc++ '-std=c++17' -MD -MF bazel-out/linux_clang-fastbuild/bin/third_party/cc/protobuf/_objs/python/google/protobuf/pyext/_message.so/third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.pic.d '-frandom-seed=bazel-out/linux_clang-fastbuild/bin/third_party/cc/protobuf/_objs/python/google/protobuf/pyext/_message.so/third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.pic.o' -fPIC -iquote . -iquote bazel-out/linux_clang-fastbuild/genfiles -iquote external/bazel_tools -iquote bazel-out/linux_clang-fastbuild/genfiles/external/bazel_tools -isystem third_party/cc/protobuf/python -isystem bazel-out/linux_clang-fastbuild/genfiles/third_party/cc/protobuf/python -isystem third_party/cc/protobuf/src -isystem bazel-out/linux_clang-fastbuild/genfiles/third_party/cc/protobuf/src -isystem external/bazel_tools/tools/cpp/gcc3 -isystem third_party/cc/cpython/Include -isystem bazel-out/linux_clang-fastbuild/genfiles/third_party/cc/cpython/Include -isystem third_party/cc/cpython -isystem bazel-out/linux_clang-fastbuild/genfiles/third_party/cc/cpython -DHAVE_PTHREAD -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -Wno-unused-function -Wno-writable-strings '-DGOOGLE_PROTOBUF_HAS_ONEOF=1' -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.cc -o bazel-out/linux_clang-fastbuild/bin/third_party/cc/protobuf/_objs/python/google/protobuf/pyext/_message.so/third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.pic.o)
Use --sandbox_debug to see verbose messages from the sandbox
third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.cc:172:13: error: assigning to 'char *' from incompatible type 'const char *'
if (PyString_AsStringAndSize(key, &name, &name_size) < 0) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.cc:69:22: note: expanded from macro 'PyString_AsStringAndSize'
((*(charpp) = PyUnicode_AsUTF8AndSize(ob, (sizep))) == NULL? -1: 0): \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.cc:189:13: error: assigning to 'char *' from incompatible type 'const char *'
if (PyString_AsStringAndSize(key, &camelcase_name, &name_size) < 0) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
third_party/cc/protobuf/python/google/protobuf/pyext/descriptor_containers.cc:69:22: note: expanded from macro 'PyString_AsStringAndSize'
((*(charpp) = PyUnicode_AsUTF8AndSize(ob, (sizep))) == NULL? -1: 0): \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
Target //third_party/cc/protobuf:protobuf_python failed to build
After digging through the documentation of cpython at its master, I found this commit: python/cpython@2a404b6
It basically says cpython made a breaking change in its C API, i.e., Python.h.
I know that python 3.7 is not released yet. What is the stance of protobuf group on this matter?
artyompal commented
This breaks the latest TensorFlow build.
anandolee commented
Python 3.7.0 was released last week on 2018-06-27. We need to add support for 3.7 and add test for it in test.sh: https://github.com/google/protobuf/blob/00d32539c6aa0c4efe9577561a00f443edd9a417/tests.sh#L244
qzmfranklin commented
Awesome to see some progress on this!