haoyuhu/bert-multi-gpu

Issues

• The main difference between original bert and bert-muti-gpu are these lines below?

  #9 opened 5 years ago by TPF2017
  10

• In prediction, only one gpu is available

  #35 opened 3 years ago by hys50001
  2

• How to freeze bert layers?

  #34 opened 4 years ago by heretree
  2

• mlabel task 结果定义

  #33 opened 4 years ago by jackie930
  2

• Do we see different results with different global_batch_size but same iteration_steps? How does Global Batch Size play a role? Will time taken to complete training change?

  #31 opened 4 years ago by Q-Udita
  3

• Query regarding calculation of iteration_steps and role of global_batch_size.

  #32 opened 4 years ago by Q-Udita
  2

• 能否使用多GPU进行evaluate？

  #29 opened 4 years ago by ZZKa
  2

• 支持多gpu预训练么

  #30 opened 4 years ago by joytianya
  4

• 请问下，设置总batchsize为 每个机器的batchsize乘以GPU个数 的相关核心代码是哪里？

  #28 opened 5 years ago by guotong1988
  4

• ##Not found: Key bert/embeddings/LayerNorm/beta/AdamWeightDecayOptimizer not found in checkpoint

  #12 opened 5 years ago by KelvinBull
  10

• 训练速度是如何加速的？

  #27 opened 5 years ago by freeloop1114
  0

• "Device is available but not used by distribute strategy"

  #24 opened 5 years ago by Jhangsy
  13

• ImportError: No module named 'tensorflow.python.distribute.cross_device_ops'

  #11 opened 5 years ago by LeoWood
  6

• run_pretraining.py script for multi-gpu

  #23 opened 5 years ago by ftamburin
  2

• can I use it for multi gpu prediction?

  #21 opened 5 years ago by TPF2017
  9

• Is fp16 supported under multi-gpu ？I see that mixed_precision related code is commented in custom_optimization.py.

  #18 opened 5 years ago by wrxDM
  3

• When I finetune from roberta_zh_large model, the training process hangs

  #16 opened 5 years ago by LaineyHu
  7

• Reported error when using FP16

  #19 opened 5 years ago by secretsh
  2

• 我怎样知道我的num_gpu_cores

  #14 opened 5 years ago by lu161513
  4

• AttributeError: 'list' object has no attribute 'text_a'

  #13 opened 5 years ago by gr8Adakron
  3

• No OpKernel was registered to support Op 'NcclAllReduce' used by node NcclAllReduce

  #15 opened 5 years ago by qiu-nian
  5

• some problems about multi gpu training

  #8 opened 5 years ago by frost-768
  10

• Is this a mixed precision version?

  #10 opened 5 years ago by hxyshare
  2

• Prediction using Multi-GPU bert

  #7 opened 5 years ago by aswin-giridhar
  3

• Pretrain with multi gpus

  #6 opened 5 years ago by mudong0419
  2

• Question: Does it make sense to integrate tensorflow2?

  #3 opened 6 years ago by AndreasFdev
  6

• Any benchmark results?

  #1 opened 6 years ago by soloice
  12