/unidecode

sanitize - convert Unicode text into safe ASCII strings - Emacs Lisp package

Primary LanguageEmacs LispGNU General Public License v2.0GPL-2.0

unidecode.el

Functions for transliterating Unicode text to ASCII:

  • (unidecode STRING)
  • (unidecode-region BEG END)
  • (unidecode-sanitize STRING)
  • (unidecode-sanitize-region BEG END)

This package was derived from the Unidecode module for Python, which was in turn derived from the Text::Unidecode package for Perl.

Example use

(require 'unidecode)

(unidecode "Déjà vu")
;=> "Deja vu"
(unidecode "北亰")
;=> "Bei Jing "
(unidecode-sanitize "Déjà vu, all over again")
;=> "deja-vu-all-over-again"

How this package was made

This package uses a number of data files (see the data directory), which were converted from the corresponding files in the Python Unidecode module.

The scripts used to perform this conversion are tools/unidecode-convert.el and tools/unidecode_convert.py.

To perform the conversion yourself, load unidecode-convert.el and run the unidecode-convert command (e.g. M-x unidecode-convert RET). When prompted, enter the path to Unidecode’s source directory as the first argument, and where you want the files to be saved as the second argument. It uses the Python script to convert the Unidecode files to JSON, then uses Emacs’s json package to load them, and finally prints them as read-able Lisp data as stored in the data directory.

License

This project is free software, available under the terms of the GPL, version 2 or later. See COPYING for details.