Support for storing fonts on flash memory
antirez opened this issue · 10 comments
Hello Peter,
this library is great, thank you! Playing with images rendering in the Pico and ESP32 MCUs, I noticed that to read the data from file and transferring them to the video memory is not slow at all, and saves a lot of memory. I wonder if there is any interest in doing this in this library: the font index and data could be stored on the file. When a character is requested, the index is checked directly in the file, then we seek in the right place and return the bytes() with a single read_into() call, and that's it. RAM is scarse but flash memory is a bit more available.
Of course, this is mostly relevant in case the font is not frozen inside MIcroPython, but to build MicroPython to include a file that can be just copied in the flash is something many folks are not willing to do.
There is a forthcoming MicroPython feature called romfs. The idea is that there will be a read-only mountable filesystem located in Flash. This means that files can be stored in Flash without the need to recompile. Fonts will be a prime target :)
@peterhinch thanks, I followed this issue in MP GitHub! It's a great thing, indeed :) I also saw that your system also supports font files target, sorry for not noticing when I opened this issue. However you are reporting that this system is slow, but I want to give it a try, maybe with mixed approaches like taking the index in memory and only the font on disk. Anyway really a great work. Now I'm going to explore your writer class as well. Closing for now, as there are little reasons to take this open now that all is clear. However if I get any decent result I'll ping back. Cheers.
Re speed there was a major improvement in rendering of color fonts when the framebuf.blit method acquired the palette arg. This enabled glyphs to be blitted with color mapping, rather than being rendered in Python one pixel at a time. If you do decide to develop your own alternative, I suggest you ensure you use blitting with a palette.
Text rendering in the various GUI libraries is visually fast. I think you would have to be rendering a lot of small text to a large display for a problem to be visible.
Have you actually encountered a visible problem, or have you another concern such as blocking time?
@peterhinch the idea that it could be slow comes from the README of this project actually. There is a mention about the availability of file targets instead of in-memory arrays, but it is mentioned that in this way rendering is very slow. About blitting, yep, I'm aware of it, but recently by using the Viper code emitter I was able to overcome most speed limits of MIcroPython even when the C-coded library does not implement certain things.
The reference to "slow" in the README specifically refers to storing binary fonts in a file, a technique I used nearly ten years ago with ePaper displays. This was probably before the time when frozen bytecode was available. I don't think there is any real application for binary fonts. Especially once romfs goes live.
The real focus of font_to_py.py is on creating Python font files which can be frozen. The lookup performance is very much faster than a seek in a binary file; in the case of normal (non-sparse) font files it is done by array indexing. The ordinal value of the character is an address of the index, which provides an address into the font array. I'm not sure how this could be made faster.
Sparse fonts add a level of indirection but it is still a fast, efficient lookup.
@peterhinch Hello again Peter! Yes, your reasoning makes sense, however I was able to obtain good results by storing fonts directly in the flash! Probably modern flash memories are much faster. What I do, however, is to perform the binary search in-memory of course. So the seek is just to retrieve the chars bitmaps.
All the details and the code here:
https://github.com/antirez/microfont
You may be potentially interested in the blitting code I wrote in Viper, that supports rotations, a feature that the framebuf library lacks. However I understand that it's not everyday that one needs rotations :) I hope I credited you appropriately. And let me say: thank you for your great work.
Impressive! Rotation support is remarkably efficient.
You state that RGB565 is the most common color format. This is true at the device level, but is ruinously expensive in framebuf RAM for large displays. My drivers such as ILI9341 use GS4_HMSB with Viper code mapping the 4-bit color values onto 16 bit RGB565 via a LUT when the data is output to the display. If you're interested in color on bigger displays you might want to consider this approach.
I don't think vertical mapping is worth supporting: I don't know a single display currently in production that uses it.
Looking back, the "slow" performance I experienced was with binary font files stored on an SD card accessed via the official sdcard.py driver. However I made no attempt to optimise this as the target display was ePaper which took about 5s to update.
Out of interest why does the text at 45° on the blue display appear to have missing pixels? The other display looks fine.
As an aside, storing fonts as Python source has other advantages apart from the ability to freeze them. A Python font includes its indices plus the code to access the font. This means that normal and sparse fonts provide the same API: sparse fonts work with applications that pre-dated their introduction.
Thank you! I used the same technique you are using in my ST77xx driver and indeed it works great! But in this case a monochrome framebuffer is used, for minimal memory usage. Apparently many of the recent ESP32-S3 are now implementing an on-board SPIRAM of 2MB, and MicroPython can use it transparently, so we will be able to drive very large displays with an in memory RGB565 framebuffer soon, but the problem with the Pico2040 is still here, and is becoming one of my preferred MCUs.
About the missing pixels, are you referring to the lateral pixel holes you can see in the question mark of "Test!"? Other than math rounding error, there is more fundamental issue: for instance in the case of a perfect rotation to 45 degrees, in theory there are pixels coordinates approximating very well a rotated straight line. However, what the library does is different: it is actually mapping a straight line in the corresponding rotated pixels, so we are getting pixels with non-perfect coordinates. It is clear if we think at two pixels aligned vertically, one on top of the other. They have no mapping rotated 45 degrees, can either translate to a single one, or two (incrementing the length from 2 to 2*sqrt(2), either ways is wrong).
To verify that this problem is not due to oversampling, I tried to generate the same image with and without oversampling. Images attached just for fun.
My comment was on the oblique "Test!" text in this image, which seems quite corrupt:
I guess something was amiss when you created that image as the ones above don't show the effect.
These are my take-aways from our interesting discussion:
- Rendering speed with your Viper optimisation is similar to my approach using blitting.
- My observation about accessing a random access binary file being "slow" is out of date and was probably influenced by the official SD card driver. On-chip Flash is evidently faster. Perhaps I should remove the comment from the docs and link to this solution.
- I haven't attempted text rotation but the fact that you can do it without a performance penalty is impressive.
Onboard SPIRAM has actually been around for some time on ESP32 but has only recently been widely available. Like you, I favour RP2. In my case this is because of its bare metal design: it supports hard ISR's unlike ESPx. I also like the fact that the Pico W uses a separate chip for WiFi. The second core is good, but tricky to use owing to the GIL-free MicroPython design. The super low cost is good too: I'm happy to solder them onto test boards unlike (say) Pyboard D!
My solutions aim to be cross-platform so limited RAM is something I have to accept. At the moment, even if romfs becomes available, I intend to stick with Python font files for the reason I gave and also because frozen bytecode may still have a role.
As a final comment I haven't attempted to optimise the Writer classes beyond the use of blitting with a palette. This is because, in its primary role in the various GUIs, the result is visually quick.
Thank you, @peterhinch, very interesting remarks! And I really wanted to hear the reasons for somebody with more experience than me in the embedded field about favoring the Pico VS ESP32: I look forward to test the Pico W in the next days, I ordered a few from UK, but the new borders setup make everything slow to ship. It is also super interesting how drivers and optimizations that software gets can change the game about disk VS memory (and now there is the not-so-new player, SPIRAM). I really enjoyed our exchange, and want to congratulate for the quality of your code, to modify the font conversion was straightforward because it was so well written. Great software is rare but in the embedded space perhaps even more so. Cheers!