how to implement the Bidirectional Transformer?

[robotzheng]

any clues? thanks.

[jacobdevlin-google]

Not sure exactly what you're asking, but the core Transformer implementation is in modeling.py

Which is based on the one from tensor2tensor (but with less abstraction) and Attention is All You Need

If you're wondering how to instantiate a new instance, you can see run_classifier.py

The Transformer is also bidirectional "by default", it requires special masking to make it unidirectional (see here in OpenAI's transformer implementation on how that is done, although it's not needed for BERT).