Responsive Images

[Mischback]

Though I have already created a tool to generate the required image variants to make responsive images work (ImP), this might not be sufficient / desired for this project.

Requirements

• determine required image sizes
  • 320px
  • 480px
  • 640px
  • 960px
  • 1280px
  • 1600px
  • 1920px
• integrate in the overall build process (pre-processing, Sphinx processing, post-processing)
• skip image processing during (local) development
• determine file formats
  • PNG (-> default and fallback format)
  • WebP(?)
  • Avif(?)
• control which image sizes / file formats will be generated by using a configuration file (or include this configuration in one of the existing configuration files conf.py or pyproject.toml )
• rely on libvips / pyvips
  • https://libvips.github.io/pyvips/index.html
• integrate into the layout
  • Images still should have horizontal margins in mobile layout, see admonitions for reference!

[Mischback]

Resources

• MDN: Responsive Images
• Google Guide having details of the difference between <img> with sizes and srcset vs. <picture> with <source>
  • one of the examples include the use case to include one file in different <source> elements hinting different pixel densities
  • this might be really useful:
    • Create all images as multiplies of the smallest dimensions
    • include the larger ones in all <source> elements of the smaller ones
    • Needs heavy testing
• Quite detailled guide, covering img aswell as picture, file formats and art direction
• (GERMAN) Including an example that covers resolution and format switching
• SmashingMagazine, covering different use cases
• Overview of CSSTricks, including other references
• Great introduction of sizes and srcset for img
• CLS in combination with images
• Responsive Images and LCP
• AVIF/WEBP vs JPEG/PNG
• Evaluate the quality of compressed images, including link to relevant Python library
• Includes code examples of SSIM

libvips

• Save image as webp
• save image as avif
• resize image
  • resizing works by providing a factor scale, which might be calculated as target_width/source_width
• faster alternative for resizing
• vipsthumbnail cmd line interface
• strip meta data programatically

Sphinx

• https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#images
• https://docutils.sourceforge.io/docs/ref/rst/directives.html#image
• https://www.sphinx-doc.org/en/master/extdev/index.html#dev-extensions
• https://www.sphinx-doc.org/en/master/development/tutorials/todo.html
• Image Collector
  • assuming this does the magic for automatically copying the images to the build directory
• docutils' guide on creating directives
• docutils' Image source
• docutils' visit_image() source
• Sphinx Application API
• Sphinx Photofinish source

Implementation

• the ImageDirective is not implemented / extended by Sphinx. It relies on the actual implementation in docutils (corresponding docs)
• the corresponding node class in docutils is basically empty; it just defines a method astext()
• visit_image()
  • Sphinx's HTML5Writer (see here)
  • docutils' Writer (visit_image())
• a custom implementation of EnvironmentCollector might be required to mark the image source files to be included into the build (see ImageCollector implementation)

[Mischback]

Implementation Idea

• Seperate image generation from markup
  • markup is handled by a dedicated Sphinx directive
    • something like :graphic: filename.png generates the whole <picture> markup, including different image resolutions and filetypes
    • filename.png might not even exist! But the generated actual filenames must then be included in the overall build process and their existence must be checked!
    • the directive must support specification of alt/title attributes
  • images are generated when they are included into the repository (see generation script below)
• image-processing script / generation script
  • provide mode to just generate compressed versions of a source file (e.g. a file sample-in.jpg will be stored as sample-out.jpg, sample-out.webp and sample-out.avif)
    • must support lossy and lossless compression, should be selectable but provide a sane default based on input file (jpg -> lossy, png -> lossless)
    • applies a default compression level per format, must be configurable
    • can determine deviation from the input file and adjust compression level dynamically
    • must strip all meta data from the files (e.g. EXIF, ...)
  • provide mode to make a source file responsive (e.g. a file sample-in.jpg will be processed to produce sample-out_large.jpg, sample-out_big.jpg, ..., sample-out_tiny.jpg, sample-out_large.webp, ..., sample-out_tiny.webp, sample-out_large.avif, ..., and sample-out_tiny.avif)
    • must the generated sizes are dependent on widths and are attached to a semantic name (like tiny) and are configurable
    • must be DRY: the resized images are then handed over to the compression mode (see above) for compression and storage

[Mischback]

Use responsive images while mitigating CLS

See this example @ MDN and this recommendation by Google

This might lead to rather complex HTML code, as all images are provided in

• a) different sizes
• b) different formats

Consequently, the following code might be generated:

<picture>
  <source srcset="foo-big.avif 1024w, foo-large.avif 2048w" media="(min-width: 1024px)" width="1024" height="512" type="image/avif">
  <source srcset="foo-big.webp 1024w, foo-large.webp 2048w" media="(min-width: 1024px)" width="1024" height="512" type="image/webp">
  <source srcset="foo-big.jpg 1024w, foo-large.jpg 2048w" media="(min-width: 1024px)" width="1024" height="512">

  <source srcset="foo-medium.avif" media="(min-width: 800px)" width="512" height="256" type="image/avif">
  <source srcset="foo-medium.webp" media="(min-width: 800px)" width="512" height="256" type="image/webp">
  <source srcset="foo-medium.jpg" media="(min-width: 800px)" width="512" height="256">

  <source srcset="foo-small.avif" width="256" height="128" type="image/avif">
  <source srcset="foo-small.webp" width="256" height="128" type="image/webp">

  <img src="foo-small.jpg" alt="Some description" width="256" height="128">
</picture>

(The source code basically replicates Google's recommendation and extends it using different image (compression) formats. This does result in rather bloated source code, as basically all images are provided in their own <source> element. On the other hand this directly enables art direction, assuming that changed image dimensions are dependent on the semantic size of the image.)

Taking this other Google guide into consideration, the code may be modified like this:

<picture>
  <source srcset="foo-big.avif 1024w, foo-large.avif 2048w" media="(min-width: 1024px)" width="1024" height="512" type="image/avif">
  <source srcset="foo-big.webp 1024w, foo-large.webp 2048w" media="(min-width: 1024px)" width="1024" height="512" type="image/webp">
  <source srcset="foo-big.jpg 1024w, foo-large.jpg 2048w" media="(min-width: 1024px)" width="1024" height="512">

  <source srcset="foo-medium.avif 512w, foo-big.avif 1024w, foo-large.avif 2048w" media="(min-width: 800px)" width="512" height="256" type="image/avif">
  <source srcset="foo-medium.webp 512w, foo-big.webp 1024w, foo-large.webp 2048w" media="(min-width: 800px)" width="512" height="256" type="image/webp">
  <source srcset="foo-medium.jpg 512w, foo-big.jpg 1024w, foo-large.jpg 2048w" media="(min-width: 800px)" width="512" height="256">

  <source srcset="foo-small.avif 256w, foo-medium.avif 512w, foo-big.avif 1024w, foo-large.avif 2048w" width="256" height="128" type="image/avif">
  <source srcset="foo-small.webp 256w, foo-medium.webp 512w, foo-big.webp 1024w, foo-large.webp 2048w" width="256" height="128" type="image/webp">
  <source srcset="foo-small.jpg 256w, foo-medium.jpg 512w, foo-big.jpg 1024w, foo-large.jpg 2048w" width="256" height="128">

  <img src="foo-small.jpg" alt="Some description" width="256" height="128">
</picture>

This implementation sacrifices the art direction feature, but allows serving suitable images for higher DPRs.

Question is: Is this relevant? If a user surfs the website with an iPhone (DPR > 1), is he really willing to download a significantly larger image file?

[Mischback]

There is a follow-up issue (#61)!

[Mischback]

Closed with 6012879 / #56