/awesome-bert

bert nlp papers, applications and github resources ， BERT 相关论文和 github 项目

This repository is to collect BERT related resources.

Papers:

  1. arXiv:1810.04805, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , Authors: Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

  2. arXiv:1812.06705, Conditional BERT Contextual Augmentation, Authors: Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, Songlin Hu

  3. arXiv:1812.03593, SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering, Authors: Chenguang Zhu, Michael Zeng, Xuedong Huang

  4. arXiv:1901.02860, Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, Authors: Zihang Dai, Zhilin Yang, Yiming Yang, William W. Cohen, Jaime Carbonell, Quoc V. Le and Ruslan Salakhutdinov.

  5. arXiv:1901.04085, Passage Re-ranking with BERT, Authors: Rodrigo Nogueira, Kyunghyun Cho

Github Repositories:

official implement:

  1. google-research/bert, officical TensorFlow code and pre-trained models for BERT , [10053 stars]

implement of BERT besides tensorflow:

  1. codertimo/BERT-pytorch, Google AI 2018 BERT pytorch implementation

  2. huggingface/pytorch-pretrained-BERT, A PyTorch implementation of Google AI's BERT model with script to load Google's pre-trained models , [2422 stars]

  3. Separius/BERT-keras, Keras implementation of BERT with pre-trained weights, [325 stars]

  4. soskek/bert-chainer, Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"

  5. innodatalabs/tbert, PyTorch port of BERT ML model

  6. guotong1988/BERT-tensorflow, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

  7. dreamgonfly/BERT-pytorch, PyTorch implementation of BERT in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"

  8. CyberZHG/keras-bert, Implementation of BERT that could load official pre-trained models for feature extraction and prediction

  9. soskek/bert-chainer, Chainer implementation of "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"

  10. MaZhiyuanBUAA/bert-tf1.4.0, bert-tf1.4.0

  11. dhlee347/pytorchic-bert, Pytorch Implementation of Google BERT, [106 stars]

  12. kpot/keras-transformer, Keras library for building (Universal) Transformers, facilitating BERT and GPT models, [17 stars]

  13. miroozyx/BERT_with_keras, A Keras version of Google's BERT model, [5 stars]

  14. conda-forge/pytorch-pretrained-bert-feedstock, A conda-smithy repository for pytorch-pretrained-bert. , [0 stars]

other resources for BERT:

  1. brightmart/bert_language_understanding, Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN, [503 stars]

  2. Y1ran/NLP-BERT--ChineseVersion, 谷歌自然语言处理模型BERT：论文解析与python代码 , [83 stars]

  3. yangbisheng2009/cn-bert, BERT在中文NLP的应用, [7 stars]

  4. JayYip/bert-multiple-gpu, A multiple GPU support version of BERT, [16 stars]

  5. HighCWu/keras-bert-tpu, Implementation of BERT that could load official pre-trained models for feature extraction and prediction on TPU, [6 stars]

  6. Willyoung2017/Bert_Attempt, PyTorch Pretrained Bert, [0 stars]

  7. Pydataman/bert_examples, some examples of bert, run_classifier.py 是基于谷歌bert实现了Quora Insincere Questions Classification二分类比赛。run_ner.py是基于瑞金医院AI大赛 第一赛季数据和bert写的一个命名实体识别。

  8. guotong1988/BERT-chinese, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 中文 汉语

  9. zhongyunuestc/bert_multitask, 多任务task

  10. Microsoft/AzureML-BERT, End-to-end walk through for fine-tuning BERT using Azure Machine Learning , [14 stars]

  11. bigboNed3/bert_serving, export bert model for serving, [10 stars]

  12. yoheikikuta/bert-japanese, BERT with SentencePiece for Japanese text. [92 stars]

  13. whqwill/seq2seq-keyphrase-bert, add BERT to encoder part for https://github.com/memray/seq2seq-keyphrase-pytorch, [19 stars]

  14. algteam/bert-examples, bert-demo, [9 stars]

  15. cedrickchee/awesome-bert-nlp, A curated list of NLP resources focused on BERT, attention mechanism, Transformer networks, and transfer learning. [9 stars]

  16. cnfive/cnbert, 中文注释一下bert代码功能, [5 stars]

  17. brightmart/bert_customized, bert with customized features, [20 stars]

  18. yuanxiaosc/BERT_Paper_Chinese_Translation, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 论文的中文翻译。Chinese Translation! , [5 stars]

  19. JayYip/bert-multitask-learning, BERT for Multitask Learning, [29 stars]

BERT QA task:

  1. benywon/ChineseBert, This is a chinese Bert model specific for question answering, [6 stars]

  2. vliu15/BERT, Tensorflow implementation of BERT for QA

  3. matthew-z/R-net, R-net in PyTorch, with BERT and ELMo, [77 stars]

  4. nyu-dl/dl4marco-bert, Passage Re-ranking with BERT, [92 stars]

  5. xzp27/BERT-for-Chinese-Question-Answering, [7 stars]

  6. chiayewken/bert-qa, BERT for question answering starting with HotpotQA, [2 stars]

BERT classification task:

  1. zhpmatrix/Kaggle-Quora-Insincere-Questions-Classification, Kaggle新赛(baseline)-基于BERT的fine-tuning方案+基于tensor2tensor的Transformer Encoder方案

  2. maksna/bert-fine-tuning-for-chinese-multiclass-classification, use google pre-training model bert to fine-tuning for the chinese multiclass classification

  3. NLPScott/bert-Chinese-classification-task, bert中文分类实践, [51 stars]

  4. Socialbird-AILab/BERT-Classification-Tutorial, [151 stars]

  5. fooSynaptic/BERT_classifer_trial, BERT trial for chinese corpus classfication

  6. xiaopingzhong/bert-finetune-for-classfier, 微调BERT模型,同时构建自己的数据集实现分类

  7. brightmart/sentiment_analysis_fine_grain, Multi-label Classification with BERT; Fine Grained Sentiment Analysis from AI challenger, [170 stars]

  8. pengming617/bert_classification, 利用bert预训练的中文模型进行文本分类, [6 stars]

  9. xieyufei1993/Bert-Pytorch-Chinese-TextClassification, Pytorch Bert Finetune in Chinese Text Classification, [7 stars]

  10. liyibo/text-classification-demos, Neural models for Text Classification in Tensorflow, such as cnn, dpcnn, fasttext, bert ..., [6 stars]

  11. circlePi/BERT_Chinese_Text_Class_By_pytorch, A Pytorch implements of Chinese text class based on BERT_Pretrained_Model, [3 stars]

BERT NER task:

  1. JamesGu14/BERT-NER-CLI, Bert NER command line tester with step by step setup guide, [20 stars]

  2. zhpmatrix/bert-sequence-tagging, 基于BERT的中文序列标注

  3. kyzhouhzau/BERT-NER, Use google BERT to do CoNLL-2003 NER ! , [160 stars]

  4. king-menin/ner-bert, NER task solution (bert-Bi-LSTM-CRF) with google bert https://github.com/google-research.

  5. macanv/BERT-BiLSMT-CRF-NER, Tensorflow solution of NER task Using BiLSTM-CRF model with Google BERT Fine-tuning , [349 stars]

  6. FuYanzhe2/Name-Entity-Recognition, Lstm-crf,Lattice-CRF,bert-ner及近年ner相关论文follow, [11 stars]

  7. mhcao916/NER_Based_on_BERT, this project is based on google bert model, which is a Chinese NER

  8. ProHiryu/bert-chinese-ner, 使用预训练语言模型BERT做中文NER, [88 stars]

  9. sberbank-ai/ner-bert, BERT-NER (nert-bert) with google bert, [22 stars]

  10. kyzhouhzau/Bert-BiLSTM-CRF, This model base on bert-as-service. Model structure : bert-embedding bilstm crf. , [3 stars]

BERT Text Generation Task:

  1. asyml/texar, Toolkit for Text Generation and Beyond https://texar.io, Texar is a general-purpose text generation toolkit, has also implemented BERT here for classification, and text generation applications by combining with Texar's other modules. [892 stars]

BERT Knowledge Graph Task :

  1. lvjianxin/Knowledge-extraction, 基于中文的知识抽取，BaseLine：Bi-LSTM+CRF 升级版：Bert预训练

  2. sakuranew/BERT-AttributeExtraction, USING BERT FOR Attribute Extraction in KnowledgeGraph. fine-tuning and feature extraction. 使用基于bert的微调和特征提取方法来进行知识图谱百度百科人物词条属性抽取。 [10 stars]

BERT visualization toolkit:

  1. jessevig/bertviz, Tool for visualizing BERT's attention, [147 stars]

BERT chatbot :

  1. GaoQ1/rasa_nlu_gq, turn natural language into structured data(支持中文，自定义了N种模型，支持不同的场景和任务), [33 stars]

  2. GaoQ1/rasa_chatbot_cn, 基于rasa-nlu和rasa-core 搭建的对话系统demo, [60 stars]

  3. GaoQ1/rasa-bert-finetune, 支持rasa-nlu 的bert finetune, [5 stars]

BERT language model and embedding:

  1. hanxiao/bert-as-service, Mapping a variable-length sentence to a fixed-length vector using pretrained BERT model, [1941 stars]

  2. YC-wind/embedding_study, 中文预训练模型生成字向量学习，测试BERT，ELMO的中文效果, [17 stars]

  3. Kyubyong/bert-token-embeddings, Bert Pretrained Token Embeddings, [24 stars]

  4. xu-song/bert_as_language_model, bert as language model, fork from https://github.com/google-research/bert, [22 stars]

  5. yuanxiaosc/Deep_dynamic_word_representation, TensorFlow code and pre-trained models for deep dynamic word representation (DDWR). It combines the BERT model and ELMo's deep context word representation., [7 stars]