/BoundaryNet

BoundaryNet - A Semi-Automatic Layout Annotation Tool

Primary LanguagePythonMIT LicenseMIT

BoundaryNet

An Attentive Deep Network with Fast Marching Distance Maps for Semi-automatic Layout Annotation

To appear at ICDAR 2021 [ORAL PRESENTATION]

[ Paper ] [ Website ]
Figure : BoundaryNet architecture from different abstract levels: Mask-CNN, Anchor GCN.
We propose a novel resizing-free approach for high-precision semi-automatic layout annotation. The variable-sized user selected region of interest is first processed by an attention-guided skip network. The network optimization is guided via Fast Marching distance maps to obtain a good quality initial boundary estimate and an associated feature representation. These outputs are processed by a Residual Graph Convolution Network optimized using Hausdorff loss to obtain the final region boundary.

Dependencies and Installation

The BoundaryNet code is tested with

  • Python (3.5.x)
  • PyTorch (1.0.0)
  • CUDA (10.2)

Please install dependencies by

pip install -r requirements.txt

Usage

cd CODE

Initial Setup:

  • Download the Indiscapes dataset - [Dataset Link]
  • Place the
    • Dataset Images under data directory
    • Pretrained BNet Model weights in the checkpoints directory
    • JSON annotation data in datasets directory

Training

BoundaryNet

  1. MCNN:
bash Scripts/train_mcnn.sh
  1. Anchor GCN:
bash Scripts/train_agcn.sh
  1. End-to-end Fine Tuning:
bash Scripts/fine_tune.sh
  • For all of the above scripts, corresponding experiment files are present in experiments directory.
  • Any required parameter changes can be performed in these files.

Baselines

Refer to the Readme.md under the configs directory for modified baselines - CurveGCN, PolyRNN++ and DACN.

Inference

Test Set

To perform inference and get quantitative results on the test set.

bash Scripts/test.sh

Check the qualitative results in visualizations/test_gcn_pred/ directory.

Custom Images

  • Add Document-Image path and Bounding Box coordinates in experiments/custom_args.json file.
  • Execute -
 python test_custom.py --exp experiments/custom_args.json

Check the corresponding instance-level boundary results at visualizations/test_custom_img/ directory.

Fine Tuning on Custom Dataset

  1. Add dataset images in data folder and Json annotations in datasets/data_splits/.

  2. Fine Tune MCNN

  • Modify parameters in experiments/encoder_experiment.json file
  • Freeze the Skip Attention backbone
bash train_mcnn.sh

Check the corresponding instance-level boundary results at visualizations/test_encoder_pred/ directory.

  1. Train AGCN from scratch
  • From new MCNN model file in checkpoints
  • Modify the MCNN model checkpoint path in models/combined_model.py
bash train_agcn.sh

Check the corresponding instance-level boundary results at visualizations/test_gcn_pred/ directory.

Citation

If you use BoundaryNet, please use the following BibTeX entry.

@inproceedings{trivedi2021boundarynet,
    title = {BoundaryNet: An Attentive Deep Network with Fast Marching Distance Maps for Semi-automatic Layout Annotation},
    author = {Trivedi, Abhishek and Sarvadevabhatla, Ravi Kiran},
    booktitle = {International Conference on Document Analysis Recognition, {ICDAR} 2021},
    year = {2021},
}

Contact

For any queries, please contact Dr. Ravi Kiran Sarvadevabhatla

License

This project is open sourced under MIT License.