Image widget does not support HEIC images
Sh1d0w opened this issue · 53 comments
Hello. It seems that the image widget does not support High Efficiency Image File Format , which is how by default iOS 11 now stores photos.
This seems like a big issue, as if we want to pick multiple images instead of directly passing the urls to the image, we have to process them and convert to JPG, which causes memory issues if the batch is too big.
- Try to load an image with .heic extension in the image widget
- You will get unsupported codec error
Logs
flutter analyze
Analyzing example...
No issues found! (ran in 17.7s)
flutter doctor -v
[✓] Flutter (Channel beta, v0.5.1, on Mac OS X 10.13.6 17G65, locale en-BG)
• Flutter version 0.5.1 at /Users/radoslav/Projects/flutter
• Framework revision c7ea3ca377 (3 months ago), 2018-05-29 21:07:33 +0200
• Engine revision 1ed25ca7b7
• Dart version 2.0.0-dev.58.0.flutter-f981f09760
[✓] Android toolchain - develop for Android devices (Android SDK 27.0.1)
• Android SDK at /Users/radoslav/Library/Android/sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-27, build-tools 27.0.1
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-915-b08)
• All Android licenses accepted.
[✓] iOS toolchain - develop for iOS devices (Xcode 9.4.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 9.4.1, Build version 9F2000
• ios-deploy 1.9.2
• CocoaPods version 1.5.3
[✓] Android Studio (version 3.0)
• Android Studio at /Applications/Android Studio.app/Contents
✗ Flutter plugin not installed; this adds Flutter specific functionality.
✗ Dart plugin not installed; this adds Dart specific functionality.
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-915-b08)
[!] IntelliJ IDEA Community Edition (version 2018.1.4)
• IntelliJ at /Applications/IntelliJ IDEA CE.app
✗ Flutter plugin not installed; this adds Flutter specific functionality.
✗ Dart plugin not installed; this adds Dart specific functionality.
• For information about installing plugins, see
https://flutter.io/intellij-setup/#installing-the-plugins
[!] VS Code (version 1.25.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension not installed; install from
https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[!] Connected devices
! No devices available
Any workaround for this? Pretty much makes working with images impossible on iOS as most of them will be HEIC by default.
@lifenautjoe While we are waiting for a fix for this, I've created this plugin https://github.com/Sh1d0w/multi_image_picker which allows you to pick multiple images at once and properly returns them as JPEG. Give it a try.
Woah. Looks really nice Radoslav!
Will give it a try and thanks for the quick response.
@Sh1d0w P.S. Your buy me a coffee page link on the repo badge is broken 😱
@Ahmadre that’s why I’ve opened this issue, you could just thumbs up the original post instead of stating the obvious.
var imageData: Data? if let cgImage = image!.cgImage, cgImage.renderingIntent == .defaultIntent { imageData = UIImageJPEGRepresentation(image!, 0.8) } else { imageData = UIImagePNGRepresentation(image!) }
@Ahmadre How would this code run in Flutter? Sorry I'm new ..
@Ahmadre So far I didn't need to run native code, but for this I guess I will need to. I am trying to pick images from device and send to a Java backend. Works fine with jpegs and pngs. Apple has to always do things differently!
I don't know how to pass the image data from my flutter code to the native code. I get an Asset type (from multi_image_picker) once i have selected the image. How do I pass this to the native code? Also, how will I get it back (what data type)?
Appreciate any help!
@mohisham Did you read the multi_image_picker documentation? It is all well explained there https://sh1d0w.github.io/multi_image_picker/#/imagedata
You can ue either getByteData or getThumbByteData which will return a JPEG representation no matter what image type you pick.
@Sh1d0w Thanks for your reply. Ok great! Clearly I didn't read the documentation carefully enough!
@Sh1d0w If I understand correctly, when a .heic file is selected, it is converted to jpg and returned. But then why is the file extension still .heic? Shouldn't it be changed to .jpg?
Thanks.
@mohisham no, the file path was added recently and returns the real file path as is.
I met it, too
I also ran into this problem, which was temporarily solved by getThumbByteData
+1
For folks commenting +1 – please add a 👍 to the issue (at the very top) – much more helpful for our tracking!
+1
Also, would be great to have AV1 Image File Format (AVIF) decoder support along with HEIC decoder in HEIF image container.
Also, issue title and info should be updated to HEIF (the image format) instead of HEIC (the HEVC image codec) to generalize the issue for image codec support and not specifically for iOS's HEIC preference.
@lifenautjoe While we are waiting for a fix for this, I've created this plugin https://github.com/Sh1d0w/multi_image_picker which allows you to pick multiple images at once and properly returns them as JPEG. Give it a try.
Sh1d0w i have tried your plugin its pretty good, but when i am returning (sending) images, it sends in HEIC format . So how can i change extension to PNG or JPEG ?
@MuhammadAmin94 When you request the asset byte data with either getByteData or getThumbData it will always return (convert it) as JPEG.
@Sh1d0w thanks for response! But I got this (image above) when i send photo from IPhone, and received with HEIC. As you said I am requesting it getByteData. here is my source code, am i doing something wrong ?
```
for (var i = 0; i < images.length; i++) {
ByteData byteData = await images[i].getByteData();
List imageData = byteData.buffer.asUint8List();
http.MultipartFile multipartFile = http.MultipartFile.fromBytes(
'photo' + i.toString(),
imageData,
filename: images[i].name,
contentType: MediaType("image", "jpg"),
);
// add file to multipart
request.files.add(multipartFile);
}
} else {}
That's maybe because your file extension is .heic however the encoded data is actually a JPEG. Try renaming the file to .jpg and see if it will make any difference.
Sorry, I searched about how to change file extension, but could not find any proper info. How can I change a file extension while I am sending it ? Do you have any idea ?
All photos taken on devices running iOS 11+ are by default in the HEIF format and they cannot be rendered, currently making Flutter a non-viable choice for certain use cases.
It's been ~2 years since the issue was first raised, could someone from the Flutter team please let us know if this feature is going to be a priority?
@Sh1d0w how i can have the File object like from the ImagePicker.pickImage() from the Assets object ??
@ToledoNicola You can obtain the byte data as shown in the plugin docs and then save it to a file in the app dir.
@Sh1d0w ok thanks, can you give me an example? please
@ToledoNicola You can obtain the byte data as shown in the plugin docs and then save it to a file in the app dir.
@Sh1d0w Hi, thanks for your contributions no this topic. I would like to ask you for some help, if possible. I'm fairly new to Flutter, (this is my 2nd week learning).
I'm downloading a few image files and storing them on the App's Document Directory. It isn't my intention to obtain these files from a picker, but from a download request.
¿Is it possible to read HEIC files and display them as JPEG data with Flutter?
@Miyaguisan You can use flutter_image_compress to achieve this.
Sample code:
Future<Uint8List> getJPEGImage(File image) async {
return FlutterImageCompress.compressWithFile(
image.absolute.path,
format: CompressFormat.jpeg,
).then((converted) {
return Uint8List.fromList(converted);
});
}You can then display it with Image.memory(getJPEGImage(heicImage)).
@liyuqian I have port libheif to skia, and make heic work on Android and IOS. Now only support still image.
Image url: https://nokiatech.github.io/heif/content/images/crowd_1440x960.heic
I suspect we will not be able to use libheif.
We'd also have to consider the binary size implications of supporting another image format.
@dnfield I see your point, but this is not some rare image format that is occasionally used, this is how by default iOS stores and presents images. Without a built in codec for HEIC images, we have to use workarounds to display images in our apps. This is nor developer friendly nor efficient. Is there any reasonable solution you can propose in that case?
I'm checking with some skia folks on this. We may be able to just use platform native decoders where available.
skbug.com/10369 (google only, sorry) is a Skia tracker for at least part of this.
/cc @LeonScroggins
I'm checking with some skia folks on this. We may be able to just use platform native decoders where available.
Android use this way to decode heif, it depends on android's libheif.so on Android 9+. But i can not sure whether flutter can use it directly.
I was able to get something simple wired in for iOS to support this. That should be achievable, but so far only for single frame images.
On Android the story is a little more complicated - the NDK is exposing API for this, but it's not going tobe available until R and onward. We could look at doing some JNI jumping to decode, but would need to benchmark that a bit because it probably won't perform too great.
We can't link against private API in the NDK. We can't use libheif at this time either.
https://docs.google.com/document/d/1G-5s6uoP1H_7f_ByLrvCVcoFXYZ80CixkMbbHMsMCag/edit#heading=h.pub7jnop54q0 (shorter link will be available soon at https://flutter.dev/go/fallback-image-decoding).
Opened a patch to support this on macOS, iOS, and Windows.
Android will be a bit tougher but doable.
Thanks, @dnfield !
@dnfield Are you still working on fixing this for Android as well? Or can you maybe share something about the "tough but doable" solution you were thinking of so we can help.
Is it possible at all to support this on Android before Android P?
I can hopefully get back to this this week. The basic idea will be to try to use NDK API via dlsym, and if it's not available, use JNI to have the Java side do it.
NDK API should work on Android R+.
JNI should work on O+.
Below Android O, you'd need to create a custom plugin. We don't intend to ship a HEIC decoder at this time, and Android <M has no HEIC capabilities.
Sorry, I said O+, it's really P+.
Sorry, I said O+, it's really P+.
I thought so already, but still looked for some magic API on O ;)
Quick update:
I talked with some Skia folks about making some existing Skia API work for the way Flutter is built and they're open to it. That will get us support on R+ pretty easily.
I have a almost there patch for P+ to fall back on as well. It won't be up today but sometime next week it should be ready to post. I'm going to bump the milestone out on this since I don't realistically see Android landing in the next week. Once Android is in, we can open separate bugs for additional platforms if needed.
I have a working PoC for Android, but it's using some less than desirable API to work around certain things being private in Skia. I've filed a feature request with the Skia team that should make this a bit better for us. No ETA yet on that.
https://github.com/flutter/engine/compare/master..dnfield:android_heic is the POC for this if anyone is interested in trying it out early or has early feedback.
Some of the API choices there could be vastly improved with some additional API from Skia.
Are there any improvements on this issue? If somebody could tell me how far we got with this that would be awesome! I'm absolutely down to help and code something to finally get HEIC support.
Any update on this?
Support for HEIC decoding on Android P+ is now in flutter/master: flutter/engine#26746
I think this issue was meant more for iOS than android. Shall it be reopened @bdero?
I think this issue was meant more for iOS than android. Shall it be reopened @bdero?
@droidpl iOS seems already implemented by flutter/engine#19989 .
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.