Fuzzy-Pattern Tsetlin Machine for Text Generation

BooBSD2 pts1 comments

GitHub - BooBSD/Tsetlin.jl: The Fuzzy-Pattern Tsetlin Machine library, with zero external dependencies, performs blazingly fast. · GitHub

/" data-turbo-transient="true" />

Skip to content

You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

{{ message }}

BooBSD

Tsetlin.jl

Public

Notifications<br>You must be signed in to change notification settings

Fork

Star<br>83

main

BranchesTags

Go to file

CodeOpen more actions menu

Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit

History<br>308 Commits<br>308 Commits

examples

examples

src

src

test

test

.gitignore

.gitignore

CITATION.cff

CITATION.cff

LICENSE

LICENSE

README.md

README.md

View all files

Repository files navigation

Tsetlin Machine: Fresh Thinking in ML

Speed is the most important feature.

Fred Wilson

This repository provides an alternative Fuzzy-Pattern Tsetlin Machine implementation with zero external dependencies and blazingly fast performance.<br>Achieves 37 million MNIST predictions per second at 98% accuracy with 4.7 GB/s throughput on a desktop CPU and demonstrates the first Tsetlin Machine–based text generation example.

Key Features

Up to 15× faster training and 41× faster inference compared to the original FPTM implementation, achieved through the use of bitwise operations, SIMD instructions, and a specialized memory layout.

Binary classifier.

Multi-class classifier.

Single-threaded and multi-threaded training and inference.

Specialized BitSet index over literals to improve performance on very large, sparse binary vector inputs.

Model compilation to reduce memory usage and increase inference speed.

Save and load trained models for production deployment or continued training with modified hyperparameters.

Automatic selection of UInt8 or UInt16 Tsetlin Automata based on the number of TA states.

Automatic switching between binary and multi-class classification depending on the dataset.

Built-in benchmarking tool.

Built-in explainability tools.

Quick Start

Talk is cheap, show me the code some examples.

Below is an example of character-level text generation in the style of Shakespeare.

First, install the Julia language by running the following command and following the installation instructions:

curl -fsSL https://install.julialang.org | sh

In the first terminal window , run the following command to train the model over multiple epochs:

julia -O3 -t auto examples/TEXT/train.jl

In the second terminal window , run the following command after each training epoch to observe how the quality of the generated text evolves from one epoch to the next:

julia -O3 examples/TEXT/sample.jl

After 400+ epochs, you should see output similar to the following:

ROMEO:<br>The father's death,<br>And then I shall be so;<br>For I have done that was a queen,<br>That I may be so, my lord.

JULIET:<br>I would have should be so, for the prince,<br>And then I shall be so;<br>For the princely father with the princess,<br>And then I shall be the virtue of your soul,<br>Which your son,--

ESCALUS:<br>What, what should be particular me to death.

BUCKINGHAM:<br>God save the queen's proclaim'd:<br>Come, come, the Duke of York.

KING EDWARD IV:<br>So do I do not know the prince,<br>And then I shall be so, and such a part.

KING RICHARD III:<br>Shall I be some confess the state,<br>Which way the sun the prince's dead;<br>And then I will be so.

How To Use

Here is a quick "Hello, World!" example of a typical use case with the Tsetlin Machine.

Importing the necessary functions and the MNIST dataset:

using MLDatasets: MNIST<br>using .Tsetlin: TMInput, TMClassifier, train!, predict, accuracy, save, load, unzip, booleanize, compile, benchmark

x_train, y_train = unzip([MNIST(:train)...])<br>x_test, y_test = unzip([MNIST(:test)...])

Booleanizing input data (2 bits per pixel):

x_train = [booleanize(x, 0, 0.5) for x in x_train]<br>x_test = [booleanize(x, 0, 0.5) for x in x_test]

Hyperparameters

This implementation introduces some differences compared to the Vanilla Tsetlin Machine:

L — limits the number of included literals in a clause.

LF — new hyperparameter that sets the number of literal misses allowed per clause.

CLAUSES = 20 # Number of clauses per class<br>T = 20 # Voting threshold<br>S = 200 # Specificity<br>L = 150 # Maximum literals per clause<br>LF = 75 # Allowed failed literals per clause

EPOCHS = 1000 # Number of training epochs

Train the model over 1000 epochs and save the compiled model to disk:

tm = TMClassifier(x_train[1], y_train, CLAUSES, T, S, L, LF, states_num=256, include_limit=240)<br>train!(tm, x_train, y_train, x_test, y_test, EPOCHS, shuffle=true, index=false)<br>save(compile(tm), "/tmp/tm_last.tm")

Loading the compiled model and evaluating accuracy:

println">tm = load("/tmp/tm_last.tm")<br>accuracy(predict(tm, x_test), y_test) |> println

Benchmarking the compiled...

tsetlin machine text train epochs window

Related Articles