Samples for Multi GPUs in TensorFlow See the sources and comments. two-GPUs.py Using 2 GPUs /w designated GPU many-GPUs-MNIST.py Using 2+ GPUs /w seperated data by number of GPUs Additional reference Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour