RaphaelaHeil/strikethrough-removal-cyclegans

Strikethrough Removal From Handwritten Words Using CycleGANs

Raphaela Heil✉️, Ekta Vats and Anders Hast

Code and related resources for the ICDAR 2021 paper Strikethrough Removal From Handwritten Words Using CycleGANs

Table of Contents

1. Code
   1. Strikethrough Removal
   2. Strikethrough Classification
   3. Strikethrough Identification
   4. Running the Code
2. Data
3. Citation
4. Acknowledgements

Code

Each of the following subdirectories contains the code that was used in the context of this paper. Additionally, Python requirements and the original configuration(s) are included for each. Configuration files have to be modified with local paths to input and output directories before running.

Model checkpoints are attached in the release of this repository.

Strikethrough Removal

• code for training various forms of CycleGANs to remove strikethrough from handwritten words
• the CycleGAN code is based on https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

  @inproceedings{CycleGAN2017, title={Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkss}, author={Zhu, Jun-Yan and Park, Taesung and Isola, Phillip and Efros, Alexei A}, booktitle={Computer Vision (ICCV), 2017 IEEE International Conference on}, year={2017} }

Strikethrough Classification

• code to train a DenseNet121 to classify a struck-through word image into one of seven types of strikethrough

Strikethrough Identification

• code to train a DenseNet121 to identify whether a given word image is struck-through or not (i.e. 'clean')

Running the Code

Train

In order to train any of the three models, run:

python src/train.py -configfile <path to config file> -config <name of section from config file>

If no configfile is defined, the script will assume config.cfg in the current working directory. If no config is defined, the script will assume DEFAULT.

Test

For testing, run:

python src/train.py -configfile <path to config file> -data <path to data dir>

• configfile should point to the config file in an output directory of a train run (or one of the checkpoint config files)
• data should point to a directory containing struck and struck_gt sub-directories, e.g. one of the datasets presented in Data
• an additional flag -save can be specified to save the cleaned images, otherwise only performance metrics (F₁ score and RMSE) will be logged

Data

• Synthetic strikethrough dataset on Zenodo: https://doi.org/10.5281/zenodo.4767094
  • based on the IAM database
  • multi-writer
  • generated using https://doi.org/10.5281/zenodo.4767062
• Genuine strikethrough dataset on Zenodo: https://doi.org/10.5281/zenodo.4765062
  • single-writer
  • blue ballpoint pen
  • clean and struck word images registered based on:

    J. Öfverstedt, J. Lindblad and N. Sladoje, "Fast and Robust Symmetric Image Registration Based on Distances Combining Intensity and Spatial Information," in IEEE Transactions on Image Processing, vol. 28, no. 7, pp. 3584-3597, July 2019, doi: 10.1109/TIP.2019.2899947. (Paper, Code)

Citation

ICDAR 2021

@InProceedings{10.1007/978-3-030-86337-1_38,
author="Heil, Raphaela
and Vats, Ekta
and Hast, Anders",
editor="Llad{\'o}s, Josep
and Lopresti, Daniel
and Uchida, Seiichi",
title="Strikethrough Removal from Handwritten Words Using CycleGANs",
booktitle="Document Analysis and Recognition -- ICDAR 2021",
year="2021",
publisher="Springer International Publishing",
address="Cham",
pages="572--586",
}

Acknowledgements

• R.Heil would like to thank Nicolas Pielawski, Håkan Wieslander, Johan Öfverstedt and Anders Brun for their helpful comments and fruitful discussions.
• The computations were enabled by resources provided by the Swedish National Infrastructure for Computing (SNIC) at the High Performance Computing Center North (HPC2N) partially funded by the Swedish Research Council through grant agreement no. 2018-05973.
• This work is partially supported by the Riksbankens Jubileumsfond (RJ) (Dnr P19-0103:1).