SpectrumKit not working with screenshot images in iOS 13
rahulvyas opened this issue · 16 comments
I have following version of spectrum
SpectrumCore (1.1.0)
SpectrumKit (1.1.0)
mozjpeg (3.3.2)
spectrum-folly (2019.01.21.00)
When I try to compress a screenshot I'm getting a plain white image. I'm not getting any error. I'm testing on iPhone XR iOS 13.1.2.
Attaching some input images and some screenshot of what I'm getting as output.
I'm not getting any kind of error when compressing these images just plain white image I'm getting
This is my image picking code
- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info {
NSString *mediaType = [info objectForKey:UIImagePickerControllerMediaType];
if ([mediaType isEqualToString:@"public.image"]){
UIImage *image = [info objectForKey:UIImagePickerControllerOriginalImage];
UIImage *transcoded = [self transcodeImage:image];
}
}
- (UIImage*)transcodeImage:(UIImage*)image {
FSPSpectrum *spectrum = [[FSPSpectrum alloc] initWithPlugins:@[[FSPJpegPlugin new]]
configuration:nil];
FSPEncodeRequirement *encodeRequirement =
[FSPEncodeRequirement encodeRequirementWithFormat:FSPEncodedImageFormat.jpeg
mode:FSPEncodeRequirementModeLossy
quality:70];
FSPConfiguration * configuration = [[FSPConfiguration alloc]init];
FSPTransformations *transformations = [FSPTransformations new];
transformations.resizeRequirement =
[[FSPResizeRequirement alloc] initWithMode:FSPResizeRequirementModeExactOrSmaller
targetSize:CGSizeMake(1024, 1024)];
FSPEncodeOptions *options =
[FSPEncodeOptions encodeOptionsWithEncodeRequirement:encodeRequirement
transformations:transformations
metadata:nil
configuration:configuration
outputPixelSpecificationRequirement:nil];
NSError *error;
FSPResultData *result = [spectrum encodeImage:image options:options error:&error];
if (result.result.didSucceed && result.data) {
UIImage* image = [UIImage imageWithData:result.data];
if (image) {
return image;
}
}
return image;
}
Does anyone knows any fix. I have to release the app on store and it's the only issue I'm stuck with.
I've tried same images on facebook and these are working perfectly.
Edit : Updating targetSize:CGSizeMake(2048, 2048) in FSPResizeRequirement produces the below image
Edit 2 : Updating targetSize:CGRectZero in FSPResizeRequirement always gives error and I'm not able to compress the image
Here is the error log
Printing description of error:
Error Domain=com.facebook.spectrum Code=255 "(null)" UserInfo={com.facebook.spectrum.error-name=scalingX > 0, com.facebook.spectrum.error-location=facebook::spectrum::core::proc::ScalingBlockImpl::ScalingBlockImpl(const image::pixel::Specification &, const image::Size &, const image::Size &):62}
Edit 3 : Same code works fine in iPhone 6 (iOS 12.4.6)
@wizh @cuva @diegosanchezr can you look into it ?
Are the contributors still active in this library ?
Hi @rahulvyas
We are sorry for the delay.
The issue is connected with the fact that the images are 16-bits per channel and we are able to reproduce it on your images, however we are getting into the error in this line
so not sure why you are getting a white screen instead.A way for you to mitigate this will be to use an image with 8-bit per channel instead.
We currently don't support images with different number of bits per component but we would be very happy to welcome contributions in form of PRs if you'd like to add that enhancement.
@zmroczek Thanks for the heads Up. However if you read my whole thread I'm getting different results based on TargetSize as I've mentioned. Also sometimes I get exception but it's not what you're telling. I request you to please go through the whole question again. As far as PR's , I am not much into image manipulations and C++. It would be great if someone experienced work on it. Isn't there any active contributor who can fix this ?
@zmroczek I wonder how Facebook able to compress the same image properly ? Isn't the Facebook iOS app uses spectrum itself ?
Hi @rahulvyas
First of all, I'd like to explain that we found why you are not getting an exception but a distorted result instead - it's because you're using an old version of Spectrum (1.1.0) which didn't have the code in place for throwing an error that I pointed out above. So if you switch to the newest version you will be able to catch the error as well instead of displaying those broken results.
The reason why those screenshots are causing problems might be connected to this specific iOS version saving screenshots as 16bits per channel instead of 8 and therefore you might have started getting the problem because of that.
Currently we are not supporting images with different number of bits per channel than 8 on Spectrum for iOS but we are investigating on fixing that.
@zmroczek Here is my podfile and podfile.lock enteries
Podfile
pod 'SpectrumKit/Plugins/Jpeg'
Podfile.lock
- spectrum-folly (2019.01.21.00)
- SpectrumCore/Base (1.1.0):
- spectrum-folly (~> 2019.01.21.00)
- SpectrumCore/Plugins/Jpeg (1.1.0):
- mozjpeg (= 3.3.2)
- spectrum-folly (~> 2019.01.21.00)
- SpectrumCore/Base (= 1.1.0)
- SpectrumKit/Base (1.1.0):
- spectrum-folly (~> 2019.01.21.00)
- SpectrumCore/Base
- SpectrumKit/Plugins/Jpeg (1.1.0):
- spectrum-folly (~> 2019.01.21.00)
- SpectrumCore/Plugins/Jpeg
- SpectrumKit/Base (= 1.1.0)
Is there any specific version you want me to upgrade to ? If yes please suggest.
Spectrum 1.2 was released back in June, and includes the check mentioned above.
@rahulvyas I'm suspecting that this is due to bytes per pixel in image specification not being passed correctly for those images when we were not throwing an error in the older version of Spectrum you're using:
Possibly this affects how the output image looked like but we need to dig deeper into that.
Thank you for providing this repro case, we will use it for investigating the solution for this enhancement. If you'd like to dig deeper nevertheless and contribute you are also very welcome to do that.
@rahulvyas I'm suspecting that this is due to bytes per pixel in image specification not being passed correctly for those images when we were not throwing an error in the older version of Spectrum you're using:
Possibly this affects how the output image looked like but we need to dig deeper into that.
Thank you for providing this repro case, we will use it for investigating the solution for this enhancement. If you'd like to dig deeper nevertheless and contribute you are also very welcome to do that.
Any updates on this ?
Hi @rahulvyas,
We're still looking into adding the support for images with other values in bits per component than 8 in the encoding function.
In the meantime to get the correct output, you could try using transcodeImage function rather than encodeImage by passing the input file directly there rather than decoded bitmap (as in the encoding function you're currently using). Let us know if this fixes the issue for you.
Thanks @zmroczek. I'll change the code as you've suggested. Thanks for your help.
Hello!
For anyone also having this issue I fixed this problem by converting the Images to have 8bit depth, using the Apple Accelerate Framework. This code works on IOS 13 and up, don't forget to import Accelerate.
private static func converTo8BitDepth(image: UIImage) -> UIImage? {
guard (image.cgImage?.bitsPerPixel != 32) else {
return image
}
guard let cgImage = image.cgImage,
let sourceImageFormat = vImage_CGImageFormat(cgImage: cgImage),
let destImageFormat = vImage_CGImageFormat(bitsPerComponent: 8, bitsPerPixel: 32, colorSpace: CGColorSpaceCreateDeviceRGB(), bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.first.rawValue), renderingIntent: .defaultIntent) else {
// Handle error.
return nil
}
guard let sourceBuffer = try? vImage_Buffer(cgImage: cgImage),
var rgbDestBuffer = try? vImage_Buffer(width: Int(sourceBuffer.width), height: Int(sourceBuffer.height), bitsPerPixel: destImageFormat.bitsPerPixel) else {
// Handle error.
return nil
}
defer {
sourceBuffer.free()
rgbDestBuffer.free()
}
do {
let toRgbConverter = try vImageConverter.make(sourceFormat: sourceImageFormat, destinationFormat: destImageFormat)
try toRgbConverter.convert(source: sourceBuffer, destination: &rgbDestBuffer)
} catch {
// Handle error.
print(error.localizedDescription)
}
if let cgImage = try? rgbDestBuffer.createCGImage(format: destImageFormat) {
return UIImage(cgImage: cgImage)
} else {
// Handle error.
return nil
}
}
This code works on IOS 13 and up, don't forget to
import Accelerate.
is it merged into main?