This is a a Python extension module for compiling, optimizing and applying grammar rules. Rules can be compiled into weighted finite state transducers, pushdown transducers, or multi-pushdown transducers. It uses OpenFst finite-state transducers (FSTs) and FST archives (FArs) as inputs and outputs.
This library is primarily developed by Kyle Gorman.
If you use Pynini in your research, we would appreciate if you cite the following paper:
K. Gorman. 2016. Pynini: A Python library for weighted finite-state grammar compilation. In Proc. ACL Workshop on Statistical NLP and Weighted Automata, 75-80.
(Note that some of the code samples in the paper are now out of date and not expected to work.)
- A standards-compliant C++17 compiler (GCC >= 7 or Clang >= 700)
- The compatible recent version of OpenFst (see
NEWSfor this) built with thegrmextensions (i.e., built with./configure --enable-grm) and headers - Python 3.6+ and headers
There are various ways to install Pynini depending on your platform.
While Pynini is neither designed for nor tested on Windows, it can be installed using the Windows Subsystem for Linux (WSL). Simply enter the WSL environment and follow the Linux instructions below.
The pre-compiled library can be installed from
conda-forge by running conda install -c conda-forge pynini.
Alternatively, one can build from source from PyPI by
running pip install pynini.
Finally, one can use Bazel to build from source by
running bazel build //:all anywhere in the source tree.
The pre-compiled library can be installed from
conda-forge by running conda install -c conda-forge pynini.
Alternatively, one can install a pre-compiled
manylinux wheel from
PyPI by running pip install pynini. This will install the
pre-compiled manylinux wheel (if available for the release and compatible with
your platform), and build and install from source if not. Unlike the
conda-forge option above, which also installs OpenFst
and Graphviz, this does not install the OpenFst or
Graphviz command-line tools. See the enclosed
Dockerfile for instructions for building and
deploying manylinux wheels.
Finally, one can use Bazel to build from source by
running bazel build //:all anywhere in the source tree.
To confirm successful installation, run pip install -r requirements, then
python tests/pynini_test.py. If all tests pass, the final line will read OK;
a successful run will log some errors to STDERR (this is working as expected).
Pynini 2.0.0 and onward support Python 3. Pynini 2.1 versions (onward) drop Python 2 support. The current release supports Python 3.6--3.12.
Pynini is released under the Apache license. See LICENSE for more
information.
See CONTRIBUTING for more information.
This is not an official Google product.