A pytorch implementation of the paper A Few-shot Learning Approach for Historical Ciphered Manuscript Recognition and its extension Few Shots Are All You Need: A Progressive Few Shot Learning Approach for Low Resource Handwriting Recognition. The proposed model can be used for low resource handwriting recognition in a few-shot learning scenario.
clone the repository:
git clone https://github.com/dali92002/HTRbyMatching
cd HTRbyMatchingCreate your evironment, with the file htrmatching.yml, then activate it.
conda env create -f htrmatching.yml
conda activate fewdetDownload the desired weights. These are following the training that was done in A Few-shot Learning Approach for Historical Ciphered Manuscript Recognition.
| Training Dataset | Fine Tuning Dataset | URL |
|---|---|---|
| Omniglot | -- | model |
| Omniglot | Borg | model |
| Omniglot | Copiale | model |
| Omniglot | Vatican | model |
Check this instructions file to prepare your data for training:
https://docs.google.com/document/d/1c9waNt2vNq_WJiRLBU81hqCcTEaPjQ0-I8xybyyJ8Uc/edit?usp=sharing
Then, you canbtrain the model from scratch or fine tune it by taking the omniglot model weights from the table above. Here, I am training for the cipher named runic my training data path is ./few5/ my validation data path is ./data_validation , I am training with a batch_size of 5 and a threshold of 0.4 (it is recommended to keep it 0.4) and here I am doing a fine tuning over the model trained on omniglot so I specify train_type as "fine_tune". If you want to train from scratch specify the train_type as "scratch".
python train.py --cipher runic --data_path ./few5/ --val_data_path data_validation --batch_size 5 --shots 5 --alphabet alphabet --thresh 0.4 --train_type fine_tuneDownload the desired pretrained weigts from the section Models. Then run the following command. Here we are choosing to recognize the lines of the cipher "borg", in a 1 shot scenario, with the model finetuned on the borg (as will be stated in the testing model path). We specify the input data path: lines and alphabet. As well ass the desired output path, here we want in in a floder named output in this same directory and the threshold 0.4.
python test.py --cipher borg --testing_model /MODEL_WEIGHTS_PATH/ --lines /LINES_PATH/ --alphabet ./ALPHABET_PATH/ --output ./OUTPUT_PATH/ --shots 1 --thresh 0.4Please, check the folders named lines and alphabet to realize how you should provide your input data. After running you will receive the results in a 3 subfolders of your output folder.
This part is related to the paper Few Shots Are All You Need: A Progressive Few Shot Learning Approach for Low Resource Handwriting Recognition. ...
If you find this useful for your research, please cite it as follows:
@inproceedings{souibgui2021few,
title={A few-shot learning approach for historical ciphered manuscript recognition},
author={Souibgui, Mohamed Ali and Forn{\'e}s, Alicia and Kessentini, Yousri and Tudor, Crina},
booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
pages={5413--5420},
year={2021},
organization={IEEE}
}@article{souibgui2022fewShots,
title={Few Shots Are All You Need: A Progressive Few Shot Learning Approach for Low Resource Handwritten Text Recognition},
author={Souibgui, Mohamed Ali and Forn{\'e}s, Alicia and Kessentini, Yousri and Megyesi, Be{\'a}ta},
journal = {Pattern Recognition Letters},
volume = {160},
pages = {43-49},
year = {2022},
issn = {0167-8655},
doi = {https://doi.org/10.1016/j.patrec.2022.06.003}
}