/pysurprisal

Calculate surprisal for words in text.

Primary LanguagePythonMIT LicenseMIT

pysurprisal

version downloads license python-version

An implementation of surprisal in Python.

Installation

pip install pysurprisal

Usage

from pysurprisal import Surprisal

text = "..."

# calculate surprisal for each word in the text
data = Surprisal(text)
surprisals = data.calculate_surprisals()

# get the top 10 most surprising words
top_k = data.get_top_k(10)

# print dictionary of all surprisals
# key = word, value = surprisal
print(surprisals.surprisals)

Contributors

  • capjamesg

License

This project is licensed under an MIT license.