kaintels/torchact

TorchAct, collection of activation function for PyTorch. https://pypi.org/project/torchact/

torchact

TorchAct, collection of activation function for PyTorch.

---

Quick Start

import torch
import torch.nn as nn
import torchact.nn as actnn

model = nn.Sequential(
    nn.Linear(5, 3),
    actnn.ReLU(),
    nn.Linear(3, 1),
    nn.Sigmoid()
)

dummy = torch.rand(1, 5)
print(model(dummy))

Installation

pip install torchact

How to Contribute

Thanks for your contribution!

There are several steps for contributing.

0. Fork this repo (you can work dev branch.)
1. Install library using requirements.txt
2. Write your code in torchact folder.
3. Add your module in __init__.py (__version__ cannot be changed. It will be decided later.)

For example.

from .your_module import Your_Module
__all__ = ("ReLU", "SinLU", "Softmax", "Your_Module")

3. If you want to test case, Write test case.

For example.

def test_has_attr():
    for activation_name in __all__:
        if activation_name == "Softmax":
            assert hasattr(str_to_class(activation_name)(), "dim")
        else:
            pass

4. Run black style.black .
5. Send a PR. Code testing happens automatically. (PYPI is upgraded by the admin himself.)

Citing TorchAct

To cite this repository:

@article{hantorchact,
  title={TorchAct, collection of activation function for PyTorch.},
  author={Seungwoo Han},
  publisher={Engineering Archive},
  doi={10.31224/2988},
  url={https://engrxiv.org/preprint/view/2988}
  year={2023}
}