Issue with de-warp - strange result

Question

Issue with de-warp - strange result

Opened this issue 5 years ago · 22 comments

Hi Martin,
we met shortly in Bonn.
You have explained the de-warping which was very interesting for me.
I have tried out a bit - and after some environment issues, I could run it.
Unfortunately, the results (visible as image in OCR-D-IMG-DEWARP) looks very strange.
Maybe I have used wrong parameters.
I have used the model which is decribed in anybaseocr/models/readme.md:
https://cloud.dfki.de/owncloud/index.php/s/3zKza5sRfQB3ygy
I have used the following parameter sets:
{
"gpu_id":0,
"pix2pixHD":"/home/stefan/ocrd_all/pix2pixHD",
"model_name":"/home/stefan/ocrd_all/pix2pixHD/models/"
}
You can download the image examples here:
ftp://ftp.ccs-gmbh.net (directory example1)
User: OCRDExamples
Password: OcRd%123!

Answer 1 · 2020-02-17T11:01:24.000Z

@stefanCCS that ftp server seems to be down. And it would be way easier if you just pasted the images here. Github then takes care of hosting them and you can immediately see them when browsing the issue.

Answer 2 · 2020-02-17T12:15:31.000Z

@stefanCCS that ftp server seems to be down. And it would be way easier if you just pasted the images here. Github then takes care of hosting them and you can immediately see them when browsing the issue.

I just have tried the FTP-access - and it has worked fine:
wget --user=OCRDExamples --password='OcRd%123!' ftp://ftp.ccs-gmbh.net/example1/*
--> Please try once more.

Answer 3 · 2020-02-17T12:55:51.000Z

--> Please try once more.

Working now, though only via wget, not FileZilla for some reason. Unless there is a privacy problem (looks like dutch newspaper?), I'd also appreciate uploads to GitHub.

Answer 4 · 2020-02-17T13:22:11.000Z

Ok, understood ;-)
Here you can find attached the cropped image, which was taken as input to the dewarp.
(I could not load the source image here, as it is too big).

Answer 5 · 2020-04-14T08:30:27.000Z

Any update from somebody possible?

Answer 6 · 2022-03-20T16:20:34.000Z

Hard to say but I think #89 should fix or at least improve the issue. @stefanCCS feel free to reopen if you still encounter strange results with the newest release.

Answer 7 · 2022-03-24T07:36:39.000Z

Hi @kba ,
just have tried out (with update done via ocrd_all at 22.03.2022.
I still get strange results (Image are very "blurry").
See e.g. this excerpt:

I cannot re-open this issue: Will you do it? Should I create a new one?

Answer 8 · 2022-03-24T11:02:24.000Z

@stefanCCS as I explained here, you have to set a different resize_mode and resize_width for better quality (resolution), esp. with such large images. Also see improved help text.

Answer 9 · 2022-03-24T13:28:20.000Z

btw: I get this warning here - relevant ?

/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/torchvision/transforms/transforms.py:288: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "

Answer 10 · 2022-03-24T14:05:22.000Z

Tried out with

resize_mode none
==> Result: One example works better, the other one gets "out-of-memory" with more than 8GB RAM to try to allocate
resize_mode resize_and_crop (default) and resize_width 2048 (instead of default 1024)
==> Result: Both examples are still very blurred
resize_mode resize_and_crop (default) and resize_width 4096 (instead of default 1024)
==> Result: One example works better, the other one gets "out-of-memory" with more than 8GB RAM to try to allocate

All done on full page level.
Next try will be on region level ...

Answer 11 · 2022-03-24T15:58:16.000Z

I have tried out on region level (-P operation_level region)
Then, I got the following exception
(looking in the output folder, the just 1 region-dewarped-image was created, and this filename does not have any substructure - expected would be something linke *_TR-1, which I get e.g. when using ocrd-cis-ocropy-binarize.)

  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/bin/ocrd-anybaseocr-dewarp", line 8, in <module>
    sys.exit(cli())
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd_anybaseocr/cli/ocrd_anybaseocr_dewarp.py", line 210, in cli
    return ocrd_cli_wrap_processor(OcrdAnybaseocrDewarper, *args, **kwargs)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd/decorators/__init__.py", line 88, in ocrd_cli_wrap_processor
    run_processor(processorClass, ocrd_tool, mets, workspace=workspace, **kwargs)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd/processor/helpers.py", line 88, in run_processor
    processor.process()
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd_anybaseocr/cli/ocrd_anybaseocr_dewarp.py", line 169, in process
    prepare_data(self.opt, region_image), region, region_xywh, region_image.size, input_file)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd_anybaseocr/cli/ocrd_anybaseocr_dewarp.py", line 202, in _process_segment
    file_grp=self.output_file_grp,
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd/workspace.py", line 983, in save_image_file
    force=force)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd/workspace.py", line 365, in add_file
    ret = self.mets.add_file(file_grp, **kwargs)
  File "/home/ocrdadmin/ocrd_all/venv/sub-venv/headless-tf2/lib/python3.6/site-packages/ocrd_models/ocrd_mets.py", line 307, in add_file
    raise Exception("File with ID='%s' already exists" % ID)

Answer 12 · 2022-03-24T22:34:11.000Z

btw: I get this warning here - relevant ?

no, not relevant. (string vs int mapping of interpolation types does work.)

Answer 13 · 2022-03-24T22:36:30.000Z

Result: One example works better, the other one gets "out-of-memory" with more than 8GB RAM to try to allocate

Yes, it's memory-intensive for sure. That's by design, unfortunately. And it is a huge page.

Out of curiosity: have you tried on GPU as well?

Answer 14 · 2022-03-24T22:42:21.000Z

raise Exception("File with ID='%s' already exists" % ID)

thanks for the report – can you please try again with newest #91?

Answer 15 · 2022-03-25T07:51:09.000Z

Result: One example works better, the other one gets "out-of-memory" with more than 8GB RAM to try to allocate

Yes, it's memory-intensive for sure. That's by design, unfortunately. And it is a huge page.

Out of curiosity: have you tried on GPU as well?

No GPU available - work on getting one ...

For us this is not a huge page, it is a normal one - we sometimes get A1 newspapers with 600dpi - this is huge ;-)

Answer 16 · 2022-03-25T08:54:07.000Z

For us this is not a huge page, it is a normal one - we sometimes get A1 newspapers with 600dpi - this is huge ;-)

I see. I would say this simply is not possible at the moment with our architecture – unless you invest insane amounts of RAM.

Answer 17 · 2022-03-25T09:04:24.000Z

Maybe it works ok (in the meaning of RAM) on region level - I will try out ...

Answer 18 · 2022-03-28T08:32:53.000Z

Have tried out on Region-Level.
Works ok with newest version 1.8.
"ok" means

no issue with amount of RAM
no issue with blurred results
and some dewarping done
==> in general ok (but very slow on CPU).

It is ok for me to close this issue now (shall I do this?)

Answer 19 · 2022-03-28T08:58:06.000Z

Hi again,
last (for now) comment from my side.
I have tried out dewarping of a pretty strong warped region like this example here:

Unfortunately, the result is not better (somehow worse) - see here:

I would guess that the training has not seen this kind of strong warped examples.
Maybe something to be improved?

Anyway, issue can be closed as no software bug is open.

Answer 20 · 2022-03-28T11:01:01.000Z

I would guess that the training has not seen this kind of strong warped examples.
Maybe something to be improved?

Yes, that's exactly what I would assume. Unfortunately, the developers never documented what they used for training.

I guess we'll just have to write/wrap parameteric dewarpers...

Answer 21 · 2022-03-28T11:05:18.000Z

This brings me back to the idea, that ocrd-kraken-segment should learn "Regions-level". And with this lines, the dewarping should work ok.
What do you think?

Answer 22 · 2022-03-28T11:22:05.000Z

This brings me back to the idea, that ocrd-kraken-segment should learn "Regions-level". And with this lines, the dewarping should work ok.
What do you think?

You mean the line-level dewarping? Yes, I guess it would be better, but for extreme cases like this, the Ocropy dewarper currently is not working well – it only follows a certain amount of slope. (You'll probably have to set -P smoothness 0.1 or so.) In the future, it will directly follow the annotated polygon (or even baseline), instead of detecting the centerline via the old lineest mechanism.

But to get good textlines, even in the presence of high line warp, for print you can also use ocrd-cis-ocropy-segment on region level.