MBARI MBB Supercategory annotation inconsistency?

[Jordan-Pierce]

Hi, I'm downloading the data used in the Super Category publication and downloading the training and validation set using the JSON file provided. I've noticed for about half of the images localizations are shifted by a considerable amount (see screenshot below) . Thought it was weird that half of them are perfect, the others shifted, if you go to the URL for this particular example and download the image, you can see that the width or height of the image is different from what is in the JSON file on zenodo.

This isn't a huge issue as I can use PIL to get the image sizes quickly and update annotations on my side, but just wanted you to know because whatever causes this might cause more problems down the line? I could have used the API to get the images and localizations with the UUID, but thought it would be faster this way; not sure if the API returns the correct values (and the publication JSON is wrong for some reason).

{
  "id": 2460789,
  "uuid": "d849c35a-7a48-498d-bff5-bba458ba4684",
  "url": "https://fathomnet.org/static/m3/framegrabs/Ventana/images/3133/00_34_54_08.png",
  "valid": true,
  "imagingType": "ROV",
  "depthMeters": 943.3200073242188,
  **"height": 486,**  <------------------------------------------------------------------------------
  "lastValidation": "2021-10-01T07:41:59.317264200Z",
  "latitude": 36.77669,
  "longitude": -122.041682,
  "altitude": null,
  "salinity": 34.481998443603516,
  "temperatureCelsius": 3.8340001106262207,
  "oxygenMlL": 0.6200000047683716,
  "pressureDbar": 953.9000244140625,
  "mediaType": null,
  "modified": null,
  "sha256": "cda85f06d28af95ea5daabde551e21339960366637b3bbb79b5a5d4984bcf38a",
  "contributorsEmail": "brian@mbari.org",
  "tags": [
    {
      "id": null,
      "uuid": "d1065090-5850-4882-bd63-40bf790bb006",
      "key": "source",
      "mediaType": "text/plain",
      "value": "MBARI/VARS",
      "createdTimestamp": null,
      "lastUpdatedTimestamp": null,
      "image": null
    }
  ],
  "timestamp": "2007-11-28T17:59:33Z",
  **"width": 720,**  <------------------------------------------------------------------------------
  "boundingBoxes": [
    {
      "id": null,
      "uuid": "210d1bb1-c09b-49f8-b5da-ce0a7ef61f6b",
      "userDefinedKey": "5667b4e2-a3b5-4b81-1f6f-806bd965ab1e",
      "concept": "Lycodapus fierasfer",
      "altConcept": null,
      "image": null,
      "groupOf": null,
      "height": 120,
      "occluded": null,
      "observer": "linda",
      "truncated": null,
      "width": 212,
      "x": 393,
      "y": 270,
      "rejected": false,
      "verified": false,
      "verifier": null,
      "verificationTimestamp": null,
      "createdTimestamp": null,
      "lastUpdatedTimestamp": null
    }
  ],
  "createdTimestamp": "2021-09-29T21:25:58.974Z",
  "lastUpdatedTimestamp": "2021-10-01T07:41:59.324Z"
},

JSON file says 720 x 486, image when downloaded is 720 x 369, or am I missing something?

[Jordan-Pierce]

Simple fix on the user's side:

# Get the attributes for the image
img_path = r['Image Path']
height = r['Height']
width = r['Width']

# Get the actual dimensions of downloaded image
w, h = Image.open(img_path).size
x_off = (w - width) // 2
y_off = (h - height) // 2

# Bounding box w/ offset if necessary
xmin = r['xmin'] + x_off
ymin = r['ymin'] + y_off
xmax = r['xmax'] + x_off
ymax = r['ymax'] + y_off

[hohonuuli]

@Jordan-Pierce Thanks for letting us know about that. At one point, we ran some automated code to remove black letter-boxes from some images. (The letterboxed images are not-so-good for ML training). That code should have adjusted the images sizes And the bounding boxes. I'll look into this further, but I think you're running into a case where the image was cropped but the boxes were not correctly shifted.

[Jordan-Pierce]

@hohonuuli I think you're right. If you want I can send you an updated train_images.json and val_images.json file with the corrections I made; otherwise it's a simple fix using the above code snippet. Cheers.