aws-samples/amazon-textract-response-parser

Rotation correction does not work as expected

MrinalJain17 opened this issue · 2 comments

MrinalJain17 commented

Are there plans to fix this issue:

amazon-textract-response-parser/src-python/trp/trp2.py

Line 595 in cd24097

# FIXME: add dimension. the relative scale messes up the new coordinates, have to use the actual image scale

I've noticed that the corrections applied align well when the document/image is rotated at right angles (90, 180, ...). But otherwise, the transformed geometries don't line up.

schadem commented

It is not scheduled yet, but gladly accept PRs that handle the rotation better.

MrinalJain17 commented

I can give it a shot, but will need some help.

As far as I understand, this is the issue:

  • The current geometry rotation (bounding box and polygons) applies the rotation along some center.
  • The document (image) when rotated generally is allowed to "expand". This rotated document image is now a plane that is misaligned with that of the geometry (bounding boxes, polygons).
  • This only occurs when the rotation is performed at non-right angles.

I have been looking into ways to tackle this, but haven't found much literature. Maybe something like PyTorch is doing - https://github.com/pytorch/vision/blob/781f512b01bc2324d7fdd11f0901f60571fc476f/torchvision/transforms/v2/functional/_geometry.py#L883

@schadem Do you happen to know if there exists some blog or something that can explain the potential solution in a better way?