khannatanmai/apertium-init

Scripts to bootstrap Apertium language modules and pairs

Apertium-Init (a.k.a. Apertium Bootstrap)

Bootstrap Apertium language modules and pairs using apertium-init.py.

Usage

First, download the script or install from PyPi with pip install apertium-init.

Usage depends on the desired module and is described below. Remember to search for TODO in the generated module to add example sentences, etc.

Monolingual Lttoolbox module

To bootstrap a monolingual language module apertium-foo using the lttoolbox formalism,

$ python3 apertium-init.py foo

To bootstrap and compile it at the same time,

$ python3 apertium-init.py foo && ( cd apertium-foo/ && ./autogen.sh && make )

Monlingual HFST module

To bootstrap a monolingual language module apertium-foo using the HFST formalism,

$ python3 apertium-init.py foo --analyser=hfst

To bootstrap and compile it at the same time,

$ python3 apertium-init.py foo --analyser=hfst && ( cd apertium-foo/ && ./autogen.sh && make )

To include a twoc file for handling prefixes,

$ python3 apertium-init.py foo --analyser=hfst --with-twoc

To include a spellrelax file for handling typographical variance,

$ python3 apertium-init.py foo --analyser=hfst --with-spellrelax

Bilingual Lttoolbox module

To bootstrap a bilingual language module apertium-foo-bar where the monolingual packages apertium-foo and apertium-bar both use the lttoolbox formalism,

$ python3 apertium-init.py foo-bar

To bootstrap and compile it at the same time,

$ python3 apertium-init.py foo-bar && ( cd apertium-foo-bar/ && ./autogen.sh && make test)

Bilingual HFST/Lttoolbox module

To bootstrap a bilingual language module apertium-foo-bar where the monolingual packages apertium-foo and apertium-bar use the HFST formalism and/or the lttoolbox formalism,

$ python3 apertium-init.py foo-bar --analysers=hfst # Both foo and bar use HFST
$ python3 apertium-init.py foo-bar --analyser1=hfst # Only foo (first language) uses HFST
$ python3 apertium-init.py foo-bar --analyser2=hfst # Only bar (second language) uses HFST

To bootstrap and compile it at the same time,

$ python3 apertium-init.py foo-bar --analysers=hfst && ( cd apertium-foo-bar/ && ./autogen.sh && make test) # Both foo and bar use HFST
$ python3 apertium-init.py foo-bar --analyser1=hfst && ( cd apertium-foo-bar/ && ./autogen.sh && make test) # Only foo (first language) uses HFST
$ python3 apertium-init.py foo-bar --analyser2=hfst && ( cd apertium-foo-bar/ && ./autogen.sh && make test) # Only bar (second language) uses HFST

Bilingual module for monolinguals that don't use apertium-tagger or CG

To bootstrap a bilingual module when one or both of the monolingual modules don't use apertium-tagger,

$ python3 apertium-init.py foo-bar --no-prob1            # Only foo doesn't have .prob
$ python3 apertium-init.py foo-bar --no-prob2            # Only bar doesn't have .prop
$ python3 apertium-init.py foo-bar --no-prob1 --no-prob2 # Neither foo nor bar have .prob

To bootstrap a bilingual module when one or both of the monolingual modules don't use CG,

$ python3 apertium-init.py foo-bar --no-rlx1           # Only foo doesn't have .rlx
$ python3 apertium-init.py foo-bar --no-rlx2           # Only bar doesn't have .rlx
$ python3 apertium-init.py foo-bar --no-rlx1 --no-rlx2 # Neither foo nor bar have .rlx

Bilingual module using recursive transfer

To bootstrap a bilingual module which uses apertium-recursive,

$ python3 apertium-init.py foo-bar --transfer=rtx

Bilingual module with anaphora resolution

To bootstrap a bilingual module which uses apertium-anaphora,

$ python3 apertium-init.py foo-bar --with-anaphora

Bilingual module with discontiguous multiwords

To bootstrap a bilingual module which uses apertium-separable,

$ python3 apertium-init.py foo-bar --with-separable

Adding features to an existing module

Apertium-init can reconfigure an existing module or pair. For example, to add apertium-separable to an existing pair:

$ python3 apertium-init.py foo-bar -r --with-separable

Note that all desired options must be specified. If the foo-bar pair used apertium-anaphora, the above command would remove it.

Pushing to Github

To bootstrap a module or pair and also add it to the apertium incubator,

$ python3 apertium-init.py foo -p     # Bootstrap module apertium-foo and push to Github
$ python3 apertium-init.py foo-bar -p # Bootstrap pair apertium-foo-bar and push to Github

To specify what username to push as (rather than relying on git config),

$ python3 apertium-init.py foo -p -u bar # Bootstrap module apertium-foo and push to Github under username bar

Development

After updating vanilla files, run ./updateBootstraper.py to update the relevant encoded files in apertium-init.py. Or, run make in the root of the repository to update all the encoded files.

You can also do sudo make install to install to /usr/local/bin/apertium-init or e.g. PREFIX=$HOME/local make install to install to $HOME/local/bin/apertium-init.

Use pipenv install --dev to install the requirements required for development, e.g. linters.

Releasing

After installing development resources following the instructions above, deploying to PyPi is relatively straightforward.

Use make dist to create a source distributable inside the dist directory that can be installed locally via pip.

Use make test-release and make release to deploy to the testing PyPi instance and the production PyPi instance respectively. Either step requires PyPi authentication credentials with access to the apertium-init package.