Fuzzy-Pattern Tsetlin Machine

A paradigm shift in the Tsetlin Machine family of algorithms.

Abstract

The "all-or-nothing" clause evaluation strategy is a core mechanism in the Tsetlin Machine (TM) family of algorithms. In this approach, each clause—a logical pattern composed of binary literals mapped to input data—is disqualified from voting if even a single literal fails. Due to this strict requirement, standard TMs must employ thousands of clauses to achieve competitive accuracy. This paper introduces the Fuzzy-Pattern Tsetlin Machine (FPTM), a novel variant where clause evaluation is fuzzy rather than strict. If some literals in a clause fail, the remaining ones can still contribute to the overall vote with a proportionally reduced score. As a result, each clause effectively consists of sub-patterns that adapt individually to the input, enabling more flexible, efficient, and robust pattern matching. The proposed fuzzy mechanism significantly reduces the required number of clauses, memory footprint, and training time, while simultaneously improving accuracy.

On the IMDb dataset, FPTM achieves 90.15% accuracy with only one clause per class, a 50× reduction in clauses and memory over the Coalesced Tsetlin Machine. FPTM trains up to 316× faster (45 seconds vs. 4 hours) and fits within 50 KB, enabling online learning on microcontrollers. Inference throughput reaches 34.5 million predictions/second (51.4 GB/s). On Fashion-MNIST, accuracy reaches 92.18% (2 clauses), 93.19% (20 clauses) and 94.68% (8000 clauses), a ∼400× clause reduction compared to the Composite TM’s 93.00% (8000 clauses). On the Amazon Sales dataset with 20% noise, FPTM achieves 85.22% accuracy, significantly outperforming the Graph Tsetlin Machine (78.17%) and a Graph Convolutional Neural Network (66.23%).

Changes compared to Tsetlin.jl

New fuzzy clause evaluation algorithm.
New hyperparameter LF that sets the number of literal misses allowed for the clause. The special case LF = 1 corresponds to the same internal logic used in the Tsetlin.jl library.

The changes compared to Tsetlin.jl are located in the following functions: check_clause(), feedback!() and train!(). Please, see the comments.

Here are the training results of the tiny 20-clause model on the MNIST dataset:

How to Run Examples

Ensure that you have the latest version of the Julia language installed.
Some examples require dataset preparation scripts written in Python. To install the necessary dependencies, run the following command:

pip install -r requirements.txt

In all Julia examples, we use -t 32, which specifies the use of 32 logical CPU cores. Please adjust this parameter to match the actual number of logical cores available on your machine.

IMDb Example (1 clause per class)

Prepare the IMDb dataset:

python examples/IMDb/prepare_dataset.py --max-ngram=4 --features=12800 --imdb-num-words=40000

Run the IMDb training and benchmarking example:

julia --project=. -O3 -t 32 examples/IMDb/imdb_minimal.jl

IMDb Example (200 clauses per class)

Prepare the IMDb dataset:

python examples/IMDb/prepare_dataset.py --max-ngram=4 --features=65535 --imdb-num-words=70000

Run the IMDb training and benchmarking example:

julia --project=. -O3 -t 32 examples/IMDb/imdb_optimal.jl

Noisy Amazon Sales Example

Prepare the noisy Amazon Sales dataset:

python examples/AmazonSales/prepare_dataset.py --dataset_noise_ratio=0.005

Run the Noisy Amazon Sales training example:

julia --project=. -O3 -t 32 examples/AmazonSales/amazon.jl

Fashion-MNIST Example Using Convolutional Preprocessing

Run the Fashion-MNIST training example:

julia --project=. -O3 -t 32 examples/FashionMNIST/fmnist_conv.jl

Fashion-MNIST Example Using Convolutional Preprocessing and Data Augmentation

To achieve maximum test accuracy, apply data augmentation when preparing the Fashion-MNIST dataset:

julia --project=. -O3 -t 32 examples/FashionMNIST/prepare_augmented_dataset.jl

Run the example that trains a large model on Fashion-MNIST:

julia --project=. -O3 -t 32 examples/FashionMNIST/fmnist_conv_augmented.jl

CIFAR-10 Example Using Convolutional Preprocessing

Prepare the CIFAR-10 dataset:

julia --project=. -O3 -t 32 examples/CIFAR10/prepare_dataset.jl

Run the CIFAR-10 training example:

julia --project=. -O3 -t 32 examples/CIFAR10/cifar10_conv.jl

MNIST Example

Run the MNIST training example:

julia --project=. -O3 -t 32 examples/MNIST/mnist.jl

To run the MNIST inference benchmark, please use the following command:

julia --project=. -O3 -t 32 examples/MNIST/mnist_benchmark_inference.jl

Citation

If you use the Fuzzy-Pattern Tsetlin Machine in a scientific publication, please cite the following paper: arXiv:2508.08350

BibTeX:

@article{hnilov2025fptm,
    title={Fuzzy-Pattern Tsetlin Machine}, 
    author={Artem Hnilov},
    journal={arXiv preprint arXiv.2508.08350},
    year={2025},
    eprint={2508.08350},
    archivePrefix={arXiv},
    primaryClass={cs.LG},
    url={https://arxiv.org/abs/2508.08350},
    doi = {10.48550/arXiv.2508.08350},
}

BooBSD/FuzzyPatternTM