"Warning: Detected unknown filetype (descriptor with ext)"

Question

"Warning: Detected unknown filetype (descriptor with ext)"

steilerDev opened this issue 2 years ago · 28 comments

The list of supported file types need to be hardcoded.

There is no logic behind supporting additional extensions, besides knowing how the filetype (as returned by the Apple backend) is mapped to a file extension.

If you should come across an error with root cause Unknown filetype descriptor <...>, like this one:

[1/4/2023, 2:04:23 PM] ERROR Error-Handler: SyncError (FATAL): Sync failed caused by LibraryError (FATAL): Error while converting asset caused by LibraryError (FATAL): Unknown filetype descriptor public.mpeg (with potential extension .mpg)

This means that you have an asset in your library whose filetype is not yet supported.

Please report this filetype descriptor in this thread to include it in future releases. In the meantime you can add the filetype descriptor as key and the file extension as value in the above mapped file of your local icloud-photos-sync installation.

In the above example, the required key/value pair would be 'public.mpeg: 'mpg'.

Answer 1 · 2023-01-08T15:41:34.000Z

Hello,

Thank you for this promising app !

Experienced fatal error at 1/8/2023, 4:38:23 PM: SyncError (FATAL): Sync failed caused by LibraryError (FATAL): Error while converting asset caused by LibraryError (FATAL): Unknown filetype descriptor public.avi (with potential extension .avi) (Error Code: 39997bc9-7aed-4162-b2ff-244658dee643)

I tried 'com.microsoft.avi': 'avi', in file-type.ts without success :(

i'm stuck and don't know how to find and remove this file from icloud.

Any idea ?

Thanks,

Answer 2 · 2023-01-08T15:43:49.000Z

Try 'public.avi': 'avi' in the file-type.ts, this should fix it :)

I'll include 'avi' in the next one :)

Answer 3 · 2023-01-08T15:56:14.000Z

ok the filetype descriptor was in the error message🤦‍♂️
It's ok now, but now I get a new error

Experienced fatal error at 1/8/2023, 4:51:00 PM: SyncError (FATAL): Sync failed caused by iCloudError (FATAL): Unable to fetch records for album 'All photos' caused by Request failed with status code 500 (Error Code: f6cf423a-61d6-4682-8f0d-75c9bb463f3a)

Answer 4 · 2023-01-08T16:07:58.000Z

I have the impression that the error on the status code 500 is temporary, I come across the unknow error again

Experienced fatal error at 1/8/2023, 5:05:12 PM: SyncError (FATAL): Sync failed caused by LibraryError (FATAL): Error while converting asset caused by LibraryError (FATAL): Unknown filetype descriptor public.avi (with potential extension .avi) (Error Code: 1aefd397-dc4b-42f0-be2e-0f7586d85bd8)

I modified the file-type.ts file to add 'public.avi': 'avi', then relaunched the container, is that enough?

Answer 5 · 2023-01-08T16:13:29.000Z

You'd need to modify the file located at /opt/icloud-photos-sync/bin/lib/photos-library/model/file-type.js within the container. A relaunch should be sufficient, but I'm pretty sure that the change will not persist between runs, since you are modifying parts of the container and not a volume (this is the area, where I'm not 100% sure how Docker behaves :D )

Could you try installing it through npm directly and modifying it in your local install dir?

Answer 6 · 2023-01-08T16:20:46.000Z

You're right, I was modifying the file locally, not the one in the container, I mounted ./app on /opt/icloud-photos-sync and it's ok now.

Work is in progress

Thank you !

Answer 7 · 2023-01-08T16:30:58.000Z

@Laffreux-dev nightly Docker image contains change for avi support

Answer 8 · 2023-01-08T16:57:21.000Z

Great, thank you !

Answer 9 · 2023-04-24T11:15:13.000Z

Hello, I've got :

Experienced fatal error at 4/24/2023, 1:12:49 PM: APP_SYNC (FATAL): Sync failed caused by SYNC_CONVERSION (FATAL): Error while converting asset caused by LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor, see GH issue 143 (dyn.ah62d4rv4ge8022pwq7w1k4dzrzvg62pmru (with potential extension .largeThumbnail))

Is there any way to skip those files?

Answer 10 · 2023-05-03T22:01:32.000Z

Just received the error for com.olympus.or-raw-image (with potential extension .ORF)

EDIT: As in the above, adding this to file-type.ts fixed the issue.

Answer 11 · 2023-05-20T21:58:52.000Z

Just received
Unknown filetype descriptor, see GH issue 143 (com.canon.cr3-raw-image (with potential extension .CR3)) (Error Code: b19e770a-dcae-4f19-898d-9d91a9a14a8e)

fixed by adding

'com.canon.cr3-raw-image': `cr3`,

to file-types.js as described.

Answer 12 · 2023-06-14T16:54:37.000Z

I had to also add these two:

'com.adobe.illustrator.ai-image': `ai`,
'public.avchd-mpeg-2-transport-stream': `m2ts`,

I did find a potential source for a more comprehensive list:
https://gist.github.com/rmorey/b8d1b848086bdce026a9f57732a3b858

I tried parsing the output of the frameworks library differently and found that tags: holds the corresponding file extension but it is a messy dump file with other things in it that wouldn't make sense. Maybe if I have some more time I'll try to figure out how to generate a comprehensive list.

It would also be nice if this tool didn't abort, but just skipped any files that didn't match a known one.

Answer 13 · 2023-07-07T20:08:59.000Z

hello
please add
'com.dji.mimo.pano.jpeg' : jpg

Answer 14 · 2023-07-08T20:36:28.000Z

Here's another:
'public.mpo-image': MPO

Answer 15 · 2023-07-09T22:28:51.000Z

MPO and jpg (and all others until so far) added in v.1.1.1-nightly.14

Answer 16 · 2023-08-04T01:48:16.000Z

Hi!

Please add AVIF:
public.avif: 'avid'

Experienced fatal error at 8/4/2023, 11:43:03 AM: APP_SYNC (FATAL): Sync failed caused by SYNC_CONVERSION (FATAL): Error while converting asset caused by LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor, see GH issue 143 (public.avif (with potential extension .AVIF)) (Error Code: 2be417bf-d969-419c-a7c7-c17b646159be)

Answer 17 · 2023-08-06T11:26:52.000Z

Added to 1.2.0-nightly.6

Answer 18 · 2023-08-06T11:58:13.000Z

1.2.0-nightly.7 no longer fatally fails when there is an unknown file type descriptor, but ignores the asset without file type support. If --enable-crash-reporting is enabled the extension will be reported using the crash handler.

Answer 19 · 2023-08-14T16:35:01.000Z

I use Unraid to store/backup too and got an issue with unraids hidden .fuse* files.

LIBRARY_INVALID_FILE (WARN): Found invalid file (.fuse_hidden0016736000000ccc) caused by LIBRARY_UNKNOWN_FILETYPE_EXTENSION (FATAL): Unknown filetype extension, see GH issue 143 ()

Answer 20 · 2023-08-14T21:52:57.000Z

Added files matching /^.fuse_hidden.*$/ to ignore list with 1.2.0-nightly.11, however this seems to have broken my E2E tests - need to take a look tomorrow/later this week what's happening (so proceed with caution).

Answer 21 · 2023-08-25T14:05:21.000Z

Hi, I have just received the following error for Canon Raw Images (CR3)

Experienced fatal error at 8/25/2023, 4:03:07 PM: APP_SYNC (FATAL): Sync failed caused by SYNC_CONVERSION (FATAL): Error while converting asset caused by LIBRARY_UNKNOWN_FILE_TYPE_DESCRIPTOR (FATAL): Unknown filetype descriptor, see GH issue 143 (com.canon.cr3-raw-image (with potential extension .CR3)) (Error Code: d4469981-c522-4308-ba5c-04feab5bb97a)

Answer 22 · 2023-08-25T14:41:28.000Z

@mzaferyahsi this was already previously reported and added in v.1.1.1-nightly.14 - so going with the latest beta should resolve this.

Answer 23 · 2023-09-02T09:43:32.000Z

Hey, just had to add 'com.adobe.pdf': `.pdf`,

What about to report these as WARNING instead of Fatal errors? I don't care much if one pdf is missing amongst thousands of photos. Or allow to ignore these with an option.

Thank you 🙏

Answer 24 · 2023-09-02T09:57:51.000Z

@vguerci making this non-fatal has already been implemented in the latest beta :)

Answer 25 · 2023-09-11T14:15:47.000Z

Hey, I've also received the following warnings;

[2023-09-11T14:12:57.626Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.626Z] WARN RuntimeWarning: Error while loading iCloud asset 60AA40EE-A450-4530-BE69-B284CA6B35E4: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.627Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.627Z] WARN RuntimeWarning: Error while loading iCloud asset 7C23AF0A-19E8-4937-A498-4C82C298B3E4: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.628Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.628Z] WARN RuntimeWarning: Error while loading iCloud asset 52C23F77-2196-4DE3-B5B0-50CD28E59328: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.629Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.629Z] WARN RuntimeWarning: Error while loading iCloud asset ED0DF5BB-791D-456A-8E82-25F54598E0CC: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Error while loading iCloud asset BE472A0F-3E22-4FD4-B38A-2E2B83898DB8: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.630Z] WARN RuntimeWarning: Error while loading iCloud asset C64F9506-E473-4438-A4F9-0C5D8CDC9D24: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.631Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.631Z] WARN RuntimeWarning: Error while loading iCloud asset EB6B9522-10B9-416B-8FBD-652EEC023F0A: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.632Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.632Z] WARN RuntimeWarning: Error while loading iCloud asset 3E650C0A-6344-4E69-A181-97EFE1AE0A20: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))
[2023-09-11T14:12:57.633Z] WARN RuntimeWarning: Unknown file extension .HEIC for descriptor public.image
[2023-09-11T14:12:57.633Z] WARN RuntimeWarning: Error while loading iCloud asset E0FAFBE7-168C-448A-B04A-5C521C7141A2: LIBRARY_UNKNOWN_FILETYPE_DESCRIPTOR: Unknown filetype descriptor, please report in GH issue 143 (public.image (with potential extension .HEIC))

Answer 26 · 2023-09-18T12:53:25.000Z

public.image/HEIC will be included in the next release - interessting enough though, the lower case heic extension pops up as image.heic

Answer 27 · 2023-09-18T16:50:05.000Z

Got the following with 1.2.0:
Warning: Detected unknown filetype (undefined with MPO): Please report in GH issue 143

Answer 28 · 2023-09-18T18:33:25.000Z

undefined is problematic - seems I actually need to get rid of filetype descriptors...