/bert_implementation_by_tf2

My simple BERT implementation by TensorFlow 2, currently including BERT, GPT-2, ALBERT, ELECTRA, RoBERTa, and ConvBERT. DETR, DPR, LayoutLM, and BEiT are to be completed.

Primary LanguagePython

Description

My simple BERT implementation by TensorFlow 2, currently including BERT, GPT-2, ALBERT, ELECTRA, RoBERTa, and ConvBERT. DETR, DPR, LayoutLM, and BEiT are to be completed. This code is partly incomplete, and may have some mistakes.

References

https://github.com/tensorflow/tensorflow
https://github.com/tensorflow/addons
https://github.com/google-research/bert
https://github.com/google-research/ALBERT
https://github.com/brightmart/albert_zh
https://github.com/google-research/electra
https://github.com/ymcui/Chinese-ELECTRA
https://github.com/brightmart/roberta_zh
https://github.com/openai/gpt-2
https://github.com/yitu-opensource/ConvBert
https://github.com/facebookresearch/detr
https://github.com/facebookresearch/DPR
https://github.com/microsoft/unilm
https://github.com/huggingface/transformers
https://github.com/bojone/bert4keras