Sidecar JSON not being used for some files from Takeout

[juliancarrivick]

Hi! Thanks for this tool, it has made my transition from Google Photos so much easier 😃

I am running v0.22 on a Google Takeout archive. There are a few photos that I've noticed are failing to correctly set the capture date in Immich. They seems to be photos that I've saved from messenger or have been sent to me without any capture date in the EXIF fields. Some example file names are:

• 64749066_665197307240365_1070194369992065024_o.jpg
• USER_SCOPED_TEMP_DATA_MSGR_PHOTO_FOR_UPLOAD_15(1).jpeg
• MSGR_PHOTO_FOR_UPLOAD_1522315122420.jpg_152231.jpeg

In all of these cases, there is a sidecar JSON file, with a capture date (links erased):

{
  "title": "64749066_665197307240365_1070194369992065024_o.jpg",
  "description": "",
  "imageViews": "7",
  "creationTime": {
    "timestamp": "1561256963",
    "formatted": "23 Jun 2019, 02:29:23 UTC"
  },
  "photoTakenTime": {
    "timestamp": "1561256644",
    "formatted": "23 Jun 2019, 02:24:04 UTC"
  },
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "url": "https://photos.google.com/photo/xxx",
  "googlePhotosOrigin": {
    "webUpload": {
      "computerUpload": {
      }
    }
  }
}

And there isn't anything in the EXIF data that I can see:

➜ exiftool 64749066_665197307240365_1070194369992065024_o.jpg | grep Date
File Modification Date/Time     : 2024:08:31 17:41:58+01:00
File Access Date/Time           : 2024:09:06 18:19:48+01:00
File Inode Change Date/Time     : 2024:09:06 18:18:09+01:00
Profile Date Time               : 2009:03:27 21:36:31

However, on upload, the current datetime is used instead of the value in the JSON:

INFO | uploaded | file="Takeout/Google Photos/Photos from 2019/64749066_665197307240365_1070194369992065024_o.jpg" "capture date"="2024-08-31 16:36:56.714 +0000 UTC" time="2024-09-06T06:09.01 BST"

I have noticed that all the filenames seem to have numbers in it, perhaps that is confusing the heuristic where the capture date is taken from the name? Happy to assist with providing some example files, more logs etc.

Thanks again!

[simulot]

The difficulty is to associate each JPEG with a JSON.
Google did something wired when storing the images with very long names...

Could you find in the takeout the pairs of JPG - JSON files?

[juliancarrivick]

I haven't checked every one (there's 300+) but all the ones I've checked they're sitting side by side like so:

[simulot]

I assume you have used the -google-photos option.

I have noticed that all the filenames seem to have numbers in it, perhaps that is confusing the heuristic where the capture date is taken from the name? Happy to assist with providing some example files, more logs etc.

You're right. Long file names are shortened and de-duplicated by adding a sequence number. The associated JSON contains the full original photo name.

There a hand full of wired rules. and some are conflicting.

Have you kept the log file? You can share it using my discord account @simulot

[juliancarrivick]

... Now I feel silly. No, I wasn't using -google-photos. That'll teach me for skimming the documentation - I got to the upload command and glazed over the specific Google Photos documentation. I'll reimport and repoen this issue if it's still a problem (it's detecting the sidecar files now which it wasn't before).

I think a little "! Important" box under the "Command upload" heading would have saved me here. Even better a automated check if immich-go detects a takeout-*.zip file and checking if -google-photos is used would be grand. The first I can open a PR for easily, the second we'll see how we go! (assuming you're open to these changes 🙂 )