Different layout with different sensor

[eabdollahian]

Describe the bug
I used three types of images for the setup layout. One real sheet from pdf, the second printed and scanned and the third captured by a camera.
I want to handle all of them but every image gives me a different layout.
Also If I get multiple images by Camera the layout does not fit exactly.

To Reproduce
I set template.json and config.json and used --setLayout

Expected behavior
I set up template.json for image that got from pdf and then see the layout result in other images and expected to layout fit in all images but they are different

Screenshots
Captured by Camera:

Scan image:

Extracted from original pdf:

Desktop (please complete the following information):

• OS: Linux
• Python version: 3.10
• OpenCV version: 4.5.5.64

Additional context
template.json

{
  "pageDimensions": [
    2092,
    2096
  ],
  "bubbleDimensions": [
    35,
    53
  ],
  "preProcessors": [
    {
      "name": "CropOnMarkers",
      "options": {
        "relativePath": "omr_marker.png",
        "sheetToMarkerWidthRatio": 17
      }
    }
  ],
  "fieldBlocks": {
    "MCQBlock1": {
      "fieldType": "QTYPE_MCQ4",
      "origin": [
        1570,
        45
      ],
      "fieldLabels": ["q1","q2"],
      "bubblesGap": 34,
      "labelsGap": 56
    },
    "MCQBlock2": {
      "fieldType": "QTYPE_INT_5",
      "origin": [
        1495,
        48
      ],
      "fieldLabels": ["q3","q4"],
      "bubblesGap": 46,
      "labelsGap": 33
    }
  }
}

config.json

{
  "dimensions": {
    "display_width": 800,
    "display_height": 1000,
    "processing_width": 1641,
    "processing_height": 1333
  },
  "threshold_params": {
    "MIN_JUMP": 30
  },
  "outputs": {
    "filter_out_multimarked_files": false,
    "show_image_level": 0
  }
}

[Udayraj123]

@eabdollahian the issue looks genuine. Since the template is large and relatively the bubbles are smaller, we anticipate such template shifts especially when the photos are taken from a Mobile camera(explained in sample3). It's recommended to try out the FeatureBasedAlignment plugin from sample6 (template_fb_align.json)

[eabdollahian]

@Udayraj123
I got worst result
I used FeatureBasedAlignment after CropOnMarkers. Is it right?
Case 1:

Case 2:

Case 3:

template.json

{
  "pageDimensions": [
    2092,
    2096
  ],
  "bubbleDimensions": [
    33,
    47
  ],
  "preProcessors": [
    {
      "name": "CropOnMarkers",
      "options": {
        "relativePath": "omr_marker.png",
        "sheetToMarkerWidthRatio": 17
      }
    },
   {
      "name": "FeatureBasedAlignment",
      "options": {
        "reference": "reference.png",
        "maxFeatures": 1000,
        "2d": true
      }
    }

  ],
  "fieldBlocks": {
    "MCQBlock1": {
      "fieldType": "QTYPE_MCQ4",
      "origin": [
        1530,
        275
      ],
      "fieldLabels": ["q1","q2"],
      "bubblesGap": 31,
      "labelsGap": 51
    },
    "MCQBlock2": {
      "fieldType": "QTYPE_INT_5",
      "origin": [
        1463,
        276
      ],
      "fieldLabels": ["q3","q4"],
      "bubblesGap": 41,
      "labelsGap": 30
    }
  }
}

config.json

{
  "dimensions": {
    "display_width": 800,
    "display_height": 1000,
    "processing_width": 1641,
    "processing_height": 1333
  },
  "threshold_params": {
    "MIN_JUMP": 30
  },
  "outputs": {
    "filter_out_multimarked_files": false,
    "show_image_level": 0
  }
}

And I used this as reference:

[eabdollahian]

@Udayraj123 Another question. What is the correct value for these?
processing_width, processing_height in config.json and
pageDimensions in template.json
My pdf-extracted image is 3508x2480
My scan image is 1754x1275
My camera image is 2891x2084
And do I need to set sheetToMarkerWidthRatio in CropOnMarkers?

[Udayraj123]

• Right, FeatureBasedAlignment is not working as expected possibly due to incorrect tuning. The debug output may help you set the right tuning for it. Instead if you can wait for a few months, we're trying to come up with a better solution for it, testing the solution in a different branch: feature/upgrade-crop-on-markers
• processing_width and processing_height are kept low for increasing speed. A healthy value is in the range 1500-2000
• pageDimensions are used for the detection. Since you have many questions in your layout, this value should also be in the range 1500-2000 (or more)
• Yes @eabdollahian the "sheetToMarkerWidthRatio" is needed for appropriate detection, although it seems to be already working in your case with the current value.

[Udayraj123]

Closing this for inactivity. @eabdollahian do try out above suggestions and feel free to reopen