Official Implementation of the paper "UTRNet: High-Resolution Urdu Text Recognition In Printed Documents"
The Poster:
- Python 3.7
- Pytorch 1.9.1+cu111
- Torchvision 0.10.1+cu111
- CUDA 11.4
- Clone the repository
git clone https://github.com/abdur75648/UTRNet-High-Resolution-Urdu-Text-Recognition.git
- Install the requirements
conda create -n urdu_ocr python=3.7
conda activate urdu_ocr
pip3 install -r requirements.txt -f https://download.pytorch.org/whl/torch_stable.html
- Training
python3 train.py --train_data path/to/LMDB/data/folder/train/ --valid_data path/to/LMDB/data/folder/val/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --exp_name UTRNet-Large --num_epochs 100 --batch_size 8
- Testing
CUDA_VISIBLE_DEVICES=0 python test.py --eval_data path/to/LMDB/data/folder/test/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth
- Character-wise Accuracy Testing
- To create character-wise accuracy table in a CSV file, run the following command
CUDA_VISIBLE_DEVICES=0 python3 char_test.py --eval_data path/to/LMDB/data/folder/test/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth
- Visualize the result by running
char_test_vis
- Reading individual images
- To read a single image, run the following command
CUDA_VISIBLE_DEVICES=0 python3 read.py --image_path path/to/image.png --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth
- Visualisation of Salency Maps
- To visualize the salency maps for an input image, run the following command
python3 vis_salency.py --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth --vis_dir vis_feature_maps --image_path path/to/image.pngE
- Create your own lmdb dataset
pip3 install fire
python create_lmdb_dataset.py --inputPath data/ --gtFile data/gt.txt --outputPath result/train
The structure of data folder as below.
data
├── gt.txt
└── test
├── word_1.png
├── word_2.png
├── word_3.png
└── ...
At this time, gt.txt should be {imagepath}\t{label}\n
For example
test/word_1.png label1
test/word_2.png label2
test/word_3.png label3
...
- UTRSet-Real
- UTRSet-Synth
- IIITH (Updated) (Original)
- UPTI (Source)
- UrduDoc - Will be made available subject to the execution of a no-cost license agreement. Please contact the authors for the same.
The text detection inference code & model based on ContourNet is here. As mentioned in the paper, it may be integrated with UTRNet for a combined text detection+recognition and hence an end-to-end Urdu OCR.
The UTRSet-Synth dataset was generated using a custom-designed robust synthetic data generation module - Urdu Synth.
This tool was developed by integrating the UTRNet (https://abdur75648.github.io/UTRNe) with a text detection model (YoloV8 finetuned on UrduDoc) for end-to-end Urdu OCR.
The application is deployed on Hugging Face Spaces and is available for a live demo. You can access it here. If you prefer to run it locally, you can clone its repository and follow the instructions given there - Repo.
Note: This version of the application uses a YoloV8 model for text detection. The original version of UTRNet uses ContourNet for this purpose. However, due to deployment issues, we have opted for YoloV8 in this demo. While YoloV8 is as accurate as ContourNet, it offers the advantages of faster processing and greater efficiency.
- 01/01/21 - Project Initiated
- 21/11/22 - Abstract accepted at WIDAFIL-ICFHR 2022
- 12/12/22 - Repository Created
- 20/12/22 - Results Updated
- 19/04/23 - Paper accepted at ICDAR 2023
- 23/08/23 - Poster presentation at ICDAR 2023
- 31/08/23 - Webtool made available
- 31/01/24 - Updated Webtool (with YoloV8) made available via HuggingFace here
- This repository is based on deep-text-recognition-benchmark
- We acknowledge the Rekhta Foundation and the personal collections of Arjumand Ara for the scanned images and Noor Fatima and Mohammed Usman for the dataset and the manual transcription.
- We also thank all the members of the Computer Vision Group, IIT Delhi for their support and guidance.
This is an official repository of the project. The copyright of the dataset, code & models belongs to the authors. They are for research purposes only and must not be used for any other purpose without the author's explicit permission.
If you use the code/dataset, please cite the following paper:
@InProceedings{10.1007/978-3-031-41734-4_19,
author="Rahman, Abdur
and Ghosh, Arjun
and Arora, Chetan",
editor="Fink, Gernot A.
and Jain, Rajiv
and Kise, Koichi
and Zanibbi, Richard",
title="UTRNet: High-Resolution Urdu Text Recognition in Printed Documents",
booktitle="Document Analysis and Recognition - ICDAR 2023",
year="2023",
publisher="Springer Nature Switzerland",
address="Cham",
pages="305--324",
isbn="978-3-031-41734-4",
doi="https://doi.org/10.1007/978-3-031-41734-4_19"
}