Issue with drawing on png images (and Livebook issues)

[kwando]

I've been playing around with Image in Livebook and I stumbled across some things :)

1. have to override kino version, since kino_vix and image cant agree on a version :)

Mix.install([
  {:image, "~> 0.16.0"},
  {:kino_vix, "~> 0.1.0"},
  {:kino, "~> 0.8.0", override: true}
])

2. Drawing on png's doesn't use the specified color:

Notice the fill color below is white (or maybe it is transparent)

3. Kino_vix explodes when creating a new image (think it is cause it tries to read the format of it)
   

4. Specifying color as a hex value doesn't work. (to be fair I'm not sure if this is a bug or not since the manual doesn't say I can do this). However the error message suggests it has converted my color into a color list at least, so not sure what to make out this.

It works with the color, `[0, 102, 119]`, I find in the stacktrace.

[kwando]

Whoops, didn't know image had is own Kino thingy, but I find some issues with that one too.

5. Still doesn't render images created in memory

6. Images that is missing metadata is throwing errors

[kwando]

Hmm that issue seems to be with vix though, now when I'm actually reading the stacktrace 🙈

Piping into Image.Kino.show() which is defined in image does work.

[kipcole9]

Wow, that's not a great experience for sure! Let me go through these one-by-one:

1. ....have to override kino version, since kino_vix and image cant agree on a version :)

Not familiar with kino_vix will look into it. But I suspect it means image might not be being resolved to 0.16.0 leading to the issue with #2 below.

2. Drawing on png's doesn't use the specified color:

This one I'm not seeing the issue. There was definitely a bug in resolving "#xxxxxx" colours but that is fixed, as best I can test, in 0.16.0 which your config shows you are using. So I am surprised. Does the output of the Mix.config cell show anything different?

3. No drawing on the crab

May I ask you to attach the crab image to the issue so I can see what is wrong here. I would expect there to be a circle as you indicate.

4. Specifying color as a hex value doesn't work.

Definitely should work in image 0.16.0 - there was a bug in earlier versions that show the same error you are reporting. Hence my concern that the image version you have resolved is not the correct one.

5. Still doesn't render images created in memory

Definitely a bug in the kino render code in Vix (of which I am the author). I will fix this today. Its clearly intolerant of missing metadata and it should be.

[kipcole9]

I'll also add ! versions of the drawing functions to make pipelines easier.

[kwando]

As you said, I probably didn't have 0.16.0 cause now I can't get the hex-color problem anymore.
kino_vix is basically just doing what vix itself is doing at the moment, so no need for that package anymore.

Drawing on the crab image is still an issue though, that file must be cursed somehow cause I cannot even it anymore 😅


Also if you gonna change kino stuff in vix, I would like to see the Image / Attributes tabs in the other order. So image is shown by default I suspect that is what you wanna see most of the time :)

Here is the crab image.

Let me know if there is anything I can do to help you out :)

[kipcole9]

Thanks for the crab image, will take a look now. I will also reverse the order of the tabs in Kino. The original order followed what @cocoa-xu did for evision but when she changed the order I didn't do so for Vix (I will today).

I've pushed d909708 that:

• Adds the ! version of the draw functions
• Updates the docs for the :color option to make clear that hex strings like #rrggbb are acceptable

When I work out what the issue is with the crab image I'll push a new release to hex and then work on the updates to vix for kino.

[kipcole9]

There were two issues with the crab. The main one was that the circle you were drawing completely eclipses the crab. Combined with the fact that the crab image has an alpha band and the color did not meant that the circle was masking the crab (default alpha of 0).

Here are some examples, reducing the size of the circle to show the issue better. Note that different colours that being passed for the circle fill:

I'm not sure what the sensible default should be here when the color supplied isn't the same number of bands as the image. The current default means the image has an alpha of 0 (transparent). Perhaps it should be 255 (opaque)? Perhaps it should be an error if the number of bands in the color doesn't match the number of bands in the base image?

[kwando]

Doh! I suspected I did something bad the last time I checked, kinda embarrassing to just paint out the crab like that. Sorry for that noise 🙈

Also thank you for explaining the color-bands issue. It wasn't obvious for me what was wrong there. I would expect the color to be opaque if didn't specify alpha. 255 in the case.

[kipcole9]

I have submitted a pull request for @akash-akya to consider that changes the tab order in Kino and removes the orientation data since that may not be present (and its also hard to understand).

[kipcole9]

I published image 0.17.0 with the following changelog entry:

Breaking change

• Change Image.orientation/1 to be Image.aspect/1 to better reflect its purpose. Orientation has a more specific meaning in image metadata and in Vips.Vix.Image.orientation/1.

Bug Fixes

• Fix Image.aspect/1 for exactly square images.

• Draw functions now default to opaque if the base image has an alpha layer (previously they would be transparent). Thanks for @kwando for the collaboration. Closes #32.

Enhancements

• Adds a :fit option to Image.thumbnail/3 that uses the same terminology as the CSS object-fit property. This option sets overwrites the options :crop and :resize to produce the specified outcome.

• Add Image.Draw.point!/4, Image.Draw.rect!/6, Image.Draw.circle!/5 and Image.Draw.line!/6.

Thanks for your patience and collaboration. Hopefully the PR for Vix will also be accepted so the Livebook experience is also smoother.

I'll keep this issue open for a while in case you spot anything else unexpected!

[kipcole9]

Closing now on the basis no further issues reported. Of course please open it again if I am mistaken.

[kwando]

Go ahead and close it :) once again thanks for this awesome library!