Summary
ML
DL Layers
Name | Keras | PyTorch | MXNet |
---|---|---|---|
DropBlock | keras-drop-block |
||
Targeted Dropout | keras-targeted-dropout |
||
Graph Convolution | keras-gcn |
||
Self-Attention | keras-self-attention |
||
Embedding Similarity | keras-embed-sim |
torch-embed-sim |
|
Layer Normalization | keras-layer-normalization |
torch-layer-normalization |
|
Position Embedding | keras-pos-embd |
torch-position-embedding |
|
Multi-Head Attention | keras-multi-head |
torch-multi-head-attention |
|
Position-wise Feed Forward | keras-position-wise-feed-forward |
||
Octave Convolution | keras-octave-conv |
mxnet-octave-conv |
|
ON-LSTM | keras-ordered-neurons |
||
DropConnect | keras-drop-connect |
||
Adaptive Softmax | keras-adaptive-softmax |
DL Optimizers
Name | Keras | PyTorch | MXNet |
---|---|---|---|
AdaBound | keras-adabound |
||
LAMB | keras-lamb | ||
LR Multiplier | keras-lr-multiplier |
||
Gradient Accumulation | keras-gradient-accumulation |
||
RAdam | keras-radam |
DL Nets
Name | Keras | PyTorch | MXNet |
---|---|---|---|
Transformer | keras-transformer |
torch-transformer |
|
BERT | keras-bert |
||
GPT-2 | keras-gpt-2 |
torch-gpt-2 |
|
Transformer-XL | keras-transformer-xl |
||
XLNet | keras-xlnet |