Add support for Python 3.x
mrry opened this issue Β· 48 comments
Currently we only support Python 2.7, but we should support Python 3.
Main things this involves: print -> print(), handle __floordiv__ / __truediv__ / __div__ correctly.
π to this issue
π
π
π
We're working on it.
Python 3 is a must have. π
How do we contribute towards python3 support? Are there any specific tickets open? Is there a python3 development branch?
@mgcdanny seems they require contributors to sign an agreement first.
https://github.com/tensorflow/tensorflow/blob/master/CONTRIBUTING.md
π
I'm running futurize on the code at the moment; once that's done it'll be easier to parallelize the remaining work. So far futurize --stage1 is checked in and I'm working through futurize --stage2 now (checking each of our divisions very carefully :)).
Unfortunately our contribution process needs a bit of improvement (we're working on streamlining it), but I'll see if there are natural chunks of work to break off and ask for help in fixing once the initial futurize push is done. Before that it's tricky to parallelize.
The only non-obvious questions (so far) are what to do with unicode and where. Cc'ing @mrry since he was taking a look at that. Most of the "names" for things, such as names for ops and names for tensors in a graph, are already restricted to be roughly alphanumeric, so hopefully we should be able to leave them as C++ string while still accepting unicode input from Python.
Also, question for people that have done such conversions before / recently: is six still the recommended way to make code transparently support both? In particular, we need stuff like xrange and iteritems as symbols that exist in both 2.7 and 3.x.
@girving I would follow the guide written by python3 core dev Brett Cannon: https://docs.python.org/3.5/howto/pyporting.html. He's been heavily involved with the push to move existing libs to python3.
@madisonmay: Thanks, should have asked you earlier. Modernize probably would have been a better choice, since after reading that page I am essentially taking the output of futurize and then rewriting it into what modernize might have already spit out. Mostly done with that phase, though, so I'll probably just finish it up using futurize.
For the record: futurize --stage was mostly print_function and a few other safe trivialities. I'll write a detailed comment on what futurize --stage2 involved for us once I'm done with it (still in the middle of reading diffs).
+1
I think it is pretty clear that there are many people interested in this issue. However, I would suggest users to use the Subscribe button instead of adding a new π.
This way developers (and all subscribed users) can get notifications on useful future updates. π
PS: If you already posted a π you might as well consider removing it to keep the discussion cleaner.
Thank you @Peque! It would be great if further comments were limited to technical discussions about Python 3 support.
From the README:
The TensorFlow Python API currently requires Python 2.7:
we are working on adding support for Python 3.0.
If you require 2.7 for the 2.X line, then why restrict yourselves to 3.0 (which was released about the time of 2.6)? If you target 3.3+, then you'll save yourselves a lot of headache, especially if you're using u'γ' unicode strings. (also see unicode_literals).
And π for six for more complicated things.
Sorry, I meant Python 3, not 3.0 -- was in a rush to fix :P. I'll fix this today
@goodmami: Yes, we will likely only support 3.3+, possibly 3.4+ (I'm not sure what the differences are, but will look that up). I also changed the issue title.
I'd really like you support from 3.3 up to the latest release of Python (currently 3.5; then 4.0 in the future). This piece of software is going to be the heart of many things in human life in the future.
@jli05: https://docs.python.org/3/whatsnew/3.4.html makes it look like supporting 3.3 won't be any harder than 3.4, so we should be good to go. It is possible our internal tests will be run only in 3.4 (and 2.7), but if anything ever breaks we'll be happy to accept patches.
i look forward to that progress!
We don't support 3.x yet, but we're getting closer. As of 1d76583, all our Python files import absolute_import, division, and print_function from __future__, and Tensor objects have the correct suite of division operators for compatibility with 2.7 and 3.x. This involved running futurize and them inspecting every call to division to see whether it should be / or //. I've also scrubbed all of our uses of xrange / range / zip / map / similar.
Next steps: set up testing with 3.x, deal with unicode correctly, fix imports for libraries that have moved around, etc.
I want to add TensorFlow to Kaggle Scripts, where we only support Python 3. Do you have an estimated timeframe of getting to Python 3 support (a couple days? a week? a month?).
We're most of the way there: all the hard bits are hopefully done and only build issues and occasional incompatibilities remain. I don't have a timeline though.
how close is this? if you point me the right direction, I may have the bandwidth to help over Thanksgiving break as well
@benhamner: As of today, we're extremely close! All of the tests pass, but there are a few build cleanups before it'll be usable outside of my hacked up git repo. Unfortunately it proved impractical to divide the work up, since it was largely a mechanical process of going through and sticking b's in front of strings and other test-driven development.
@girving: Thanks so much for your efforts to get this done quickly. Super excited to get my hands on TensorFlow!
Any progress ? Do you know, now, if we are weeks or months away from Python 3 support ?
@stonebig: We're very close! I was waiting on some build cleanups, but those are checked in now and I'm going to try them out tomorrow. When I left off, all of tensorflow's source files were Python 3 compatible; only build and configure stuff remained.
I know you can do itοΌ
Hi, thank you for your amazing work.
I guess it will be possible to install TensorFlow with pip3. Will the models.embedding module be available from pip install, or will it be necessary to build it from source?
@Netizen29: We should have a better answer soon about tutorials, models, etc. in pip (see #247).
Status update: Build fixes for tensorflow proper are in, so we just need to upstream a small BUILD change to protobuf.
As of cdf0dbf, tensorflow should be Python 3 compatible. Anyone want to try it out and report problems? I'll leave this bug open for a day or two for that purpose.
Note: To try it out, grab an up-to-date git tree, run configure, and enter /usr/bin/python3 or the equivalent when it asks for the path to Python.
Assuming testing is positive, we should have a 0.6.0 release incorporating Python 3 support and various other features soon.
@girving - successfully built without gpu support on OSX 10.10.5 with Anaconda python3. I tested the MNIST example without problems.
I had no issues installing minus forgetting to change the path on the python package install,
sudo pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
to
sudo pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-py3-none-any.whl
Thanks a ton!
Thanks @xysmas! We'll hopefully have a 0.6.0 release out with this and other stuff fairly soon.
Building the wheel fails with Python 3.5 on Linux unless wheel 0.26 is installed. virtualenv is currently installing 0.24 by default it seems.
Relevant issues:
pip install --upgrade wheel before running the build_pip_package script works. After that, all is well (tested MNIST example no GPU).
Are there automated tests in place to ensure that Python 3 support isn't broken in the future?
Thanks for all your work!
@joshburkart: Unfortunately not just yet, though they are planned. That is: all of the unit tests do work with Python 3 at the moment, but we are not yet running them automatically. This will hopefully change soon.
Closing since it seems to work. Python 3 support will ship with version 0.6.0, which is coming very soon. Future Python 3 bugs should be filed as separate issues!
Thanks for all your work - looking forward to get to finally test it along with the rest of my environment! :)
milestone?
@thelostscientist: Do you mean when? This will be part of 0.6.0.