AssertionError: [5654. 5655.] at dewarp.py", line 181, in connect_nearby_grid_group
Closed this issue · 5 comments
First of all, thank you for open-sourcing oemer! I'm a developer on the Kiselgof-Makonovetsky Digital Manuscript Project where we're trying to digitize hundreds of handwritten historical musical manuscripts. Currently we're managing a crowdsourced transcription effort, and the possibility of using free music OCR software could really help us!
I installed oemer as per instructions, and when I ran it on one of the manuscript page images, I got the following error:
2021-12-17 23:07:38 No checkpoint found in ~/test/venv2/lib/python3.9/site-packages/oemer/checkpoints/unet_big/model.onnx
1st_model.onnx: 100% 70767752/70767752
1st_weights.h5: 100% 70977288/70977288
2nd_model.onnx: 100% 38448467/38448467
2nd_weights.h5: 100% 38570576/38570576
2021-12-17 23:08:00 Extracting staffline and symbols
2348 1565
2021-12-17 23:11:48 Extracting layers of different symbols
2348 1565
2021-12-17 23:17:48 Dewarping
Traceback (most recent call last):
File "~/test/venv2/bin/oemer", line 8, in <module>
sys.exit(main())
File "~/test/venv2/lib/python3.9/site-packages/oemer/ete.py", line 257, in main
extract(args)
File "~/test/venv2/lib/python3.9/site-packages/oemer/ete.py", line 135, in extract
coords_x, coords_y = estimate_coords(staff)
File "~/test/venv2/lib/python3.9/site-packages/oemer/dewarp.py", line 267, in estimate_coords
new_gg_map = connect_nearby_grid_group(gg_map, grid_groups, grid_map, grids)
File "~/test/venv2/lib/python3.9/site-packages/oemer/dewarp.py", line 181, in connect_nearby_grid_group
assert len(grid_id) == 1, grid_id
AssertionError: [5654. 5655.]
Details on my system:
- Python 3.9.9
- Tensorflow 2.7.0 (I tried using Onnxruntime but they didn't have a distribution for my system)
- MacOS 10.15.7 with an Intel Iris Pro 1536MB (if the GPU matters)
Thanks again for open-sourcing this project!
Could you provide the input image? This seems like your image quality is far too low than the system can handle, or the stafflines aren't very clear.
Thanks for the interest in this project! It's really inspiring to hear that!
Sure, here's the image. I have a higher-quality version of the same image I can try, I'll give that a shot too.
Good news! Oemer successfully processed the higher-resolution image. Bad news, it didn't extract much of value. I guess it's not really trained on handwritten scores.
Out of curiosity, what would it take to get oemer to work on handwritten scores like these? Would it just require retraining, or do you think the work would be more involved? How large of a training corpus would we need?
Okay, that piece looks pretty hard for oemer 😂 Indeed, oemer is designed to recognize typeset notations, not for handwritten scores. It is built up with lots of assumptions that exist in modern typeset scores. Handwritten scores have too many uncertainties that is hard to make assumptions to make the process easier. As a result, the whole algorithm, the pipeline, and the deep learning models need to be re-designed to make it able to recognize handwritten scores.
Makes sense. Thanks anyways, and again, thanks for sharing oemer with the world!