Support @font-face on Linux

Question

Support @font-face on Linux

SimonSapin opened this issue 12 years ago · 36 comments

Today on IRC:

SimonSapin: Hi behdad. Does Pango have APIs to load a specific font file? (This is to implement web fonts in WeasyPrint.)
behdad: SimonSapin: no.
behdad: SimonSapin: however, you can subclass the FcFontMap and do it yourself.
behdad: that's what Firefox does.
behdad: don't ask me about the details... I don't quite know that myself either.
SimonSapin: behdad: I’ll look into it, thanks
behdad: well. that's not correct. I know. you can subclass PangoCairoFcFontMap
behdad: and override the create_font(), and figure it out from there
behdad: humm. something like that. don't remember the details.
behdad: the thing is, you need to work around the fontconfig layer

Now we only need to figure out how subclassing works in gobject. And all of fontconfig.

Answer 1 · 2016-09-12T15:26:53.000Z

Hi.
Do you have any updates or more information for this task?

Johannes

Answer 2 · 2016-09-13T13:38:42.000Z

@joeimer Wow. I've tried for the lulz to implement this solution on Stack Overflow, and… it works. As I've added almost all the features from fonts level 3, @font-face would be the icing on the cake.

Of course, this solution will only work on fontconfig-based backends (only Linux I think), but could be adapted for Uniscribe (Windows) and CoreText (OSX) by anyone intersted.

I'll push a new branch soon for testing.

Answer 3 · 2016-09-13T14:28:29.000Z

@liZe that was a fast answer. Thank you very much!
I think for the first part it is ok too only support it on linux.

Answer 4 · 2016-09-13T14:29:43.000Z

@joeimer I'm closing Kozea/tinycss#6 before creating the new branch 😉.

Answer 5 · 2016-09-14T08:16:53.000Z

Does that require writing the downloaded font to a temporary file so that fontconfig can find it? (Maybe that can be skipped for file: URLs?)

Answer 6 · 2016-09-14T08:27:05.000Z

Does that require writing the downloaded font to a temporary file so that fontconfig can find it?

Currently yes, giving an URL to FontConfig doesn't work (unsurprisingly).

Answer 7 · 2016-09-14T08:51:56.000Z

Yeah, to avoid temporary files I was thinking not of giving URLs to fontconfig but give it an in-memory buffer.

Answer 8 · 2016-09-14T09:01:37.000Z

In-memory would be perfect. In xhtml2pdf there was a problem with temporary files (permissions).

Answer 9 · 2016-09-14T16:31:32.000Z

Kozea/tinycss#6 is fixed, I have the @font-face rules in WeasyPrint ✌️.

It's really easy to add an external font into the list of available fonts. I'm now looking for a way to make pango find them with the name given in the @font-face rule.

In-memory would be perfect. In xhtml2pdf there was a problem with temporary files (permissions).

Good to know, thanks.

Answer 10 · 2016-09-14T17:26:01.000Z

Did you know that @font-face content is a list of descriptors, and that descriptors are not properties, and that their validators are almost the same, but not exactly the same? 😭.

Answer 11 · 2016-09-26T08:12:31.000Z

Since tinycss 0.4 has @font-face support, do you have a sense of a timeframe for a release of WeasyPrint with @font-face support? I've got something that needs fonts and I am looking at horrible workarounds at the moment.

Answer 12 · 2016-09-26T11:12:38.000Z

Since tinycss 0.4 has @font-face support, do you have a sense of a timeframe for a release of WeasyPrint with @font-face support?

No, I don't.

I've pushed to the font-face branch what I've done this week-end. Here is the current status:

@font-face rules are parsed,
if the src descriptor contains one URL, the font is downloaded as a temp file and added to the list of available fonts,
the font can then be used in the document using its real font family and properties (and not the ones given in the descriptors).

You can test the current version with a document like this one:

<html>
  <head>
    <style>
@font-face {
  font-family: 'Baloo Tamma';
  src: url(https://fonts.gstatic.com/s/balootamma/v1/M9jLCKQXJLpx_E5bTHjE-J0EAVxt0G0biEntp43Qt6E.ttf);
}

body {
  font-family: 'Baloo Tamma'
}
    </style>
  </head>
  <body>
    <p>Hello World!<p>
  </body>
<html>

What's missing:

managing multiple URLs in src (should be quite easy),
managing local fonts given by the local "function" (not that hard and not mandatory),
changing the family and the properties of the font in Fontconfig (see below),
cleaning the added font after use (quite hard but not mandatory),
masking local fonts (quite easy when point 2 is fixed and not mandatory),
adding the same feature for OSes using a backend different from Fontconfig, ie Windows and OSX (I won't do that myself).

What's blocking me is the Fontconfig part. I've already spent hours (days?) trying to find something more than the API documentation… If you have a real-life example explaining how to change the matching rules of a font, I'm interested. I've started to read the Firefox source code and I don't understand (yet) how it's working. If it's too hard for me, I'll ask The Great @behdad.

If anyone is interested, the current idea is to edit the added font's matching pattern. That's what's currently done in the code, but it doesn't work. I don't know if the edited pattern is just a copy, or if we need to rebuild the database, or… See the add_font_face function.

Answer 13 · 2016-09-26T11:37:27.000Z

I went over and asked Bedhad (we’re attending the same event for the next couple days) and he pointed me to:

https://mces.blogspot.com.es/2015/05/how-to-use-custom-application-fonts
Alternatively, subclass PangoFcFontMap to override the behavior of font selection.

Answer 14 · 2016-09-26T11:47:12.000Z

The first post looks wonderful, with links to font configuration modifications, in-memory fonts, synthetic bold and italic, and more. Thanks @SimonSapin and @behdad, have fun!

Answer 15 · 2016-09-26T18:17:59.000Z

Changing the family and properties of the font is done in fc5eb6c. This blog entry is really, really useful. I'll fix the multiple src, the masking of the local fonts and maybe the local function really soon.

@rokob You can test the branch if you want!

Answer 16 · 2016-09-27T10:45:25.000Z

Thanks @SimonSapin and @liZe. Note that subclassing PangoFcFontMap is what Firefox 2 or 3 did to support webfonts.

Answer 17 · 2016-10-01T19:22:33.000Z

I've added the support of multiple values for src and of the local function. It's been tested with multiple different configurations (including Google Fonts), and it seems to work very well.

Font selection is complete for now, the only descriptor not supported is unicode-range and it's not something I want to support now 😄. I'd like to add the @font-face font features before merging.

I'll add unit tests, for @font-face of course but also for different font features that couldn't be tested before (including the test of condensed fonts that was only working for lucky people). Having this feature is really good news for our the tests.

Some problems that I won't fix:

the unicode-range descriptor is not supported,
the font matching algorithm for @font-face is pretty bad,
the generated fontconfig configurations, the local fonts and the downloaded fonts are stored as temporary files (see the fontconfig bugs #78450 and #78452),
the local and dowloaded fonts are not cleaned (should they be removed when the Document object is deleted?).

Answer 18 · 2016-10-01T19:23:34.000Z

Oh, and we should create a fontconfig configuration for each different document.

Answer 19 · 2016-10-07T08:32:08.000Z

@liZe: Does your branch include support for specifying font files using data URIs? I tried checking out your branch, but it didn't work with the data URIs I'm using. (I think they're working in Chrome on the same page for me, but I probably need to verify that by uninstalling my local copy of the font first)

Answer 20 · 2016-10-07T09:26:25.000Z

Does your branch include support for specifying font files using data URIs?

Not yet, but that should be easy to fix!

Answer 21 · 2016-10-07T09:53:46.000Z

@liZe: Sorry, my memory failed me. The actual problem I ran into when using data URIs was that the process crashes with a SIGABRT:

python: fcmatch.c:779: IA__FcFontMatch: Assertion `result != NULL' failed.
fish: Job 2, “font-face-env/bin/python tools/…” terminated by signal SIGABRT (Abort)

Sadly, I haven't figured out how to actually make gdb catch the crash, so I don't really have any more debugging info at the moment.

Answer 22 · 2016-10-07T10:22:09.000Z

I have a minimal test case for the crash: https://gist.github.com/whitelynx/39938eb6f4a997094d305075d41b23ac

Answer 23 · 2016-10-07T12:03:48.000Z

@whitelynx Thanks a lot for reporting, that wasn't supported at all. I've used the real URL fetcher instead of Python's urlopen, it's much better.

I've also added the support of the font-variant and font-feature-settings descriptors.

The tests I've done seem to work. I somtimes get broken fonts in PDFs, don't know why and don't know how to reproduce.

Answer 24 · 2016-10-10T15:23:28.000Z

This works locally for me (on a Mac) but fails on my server (running Ubuntu). At the moment, I am assuming this is some server configuration error on my side rather than an issue with the code.

Answer 25 · 2016-10-10T15:28:48.000Z

This works locally for me (on a Mac)

On a Mac? Are you sure?

but fails on my server (running Ubuntu)

You can post logs here 😄.

Answer 26 · 2016-10-13T09:27:42.000Z

Maybe I should say, pdfs are generated in both cases. On the Mac I get the correct fonts in the output, on Ubuntu I see the fallback fonts. I will turn the logger back on and see what the errors are on both.

Answer 27 · 2016-10-26T17:27:50.000Z

Oh, and we should create a fontconfig configuration for each different document.

It's now done and tested, that was actually needed to use the same font family with different attributes set in different tests. I've blindly followed @behdad's post, I'm not sure to always understand everything 😉.

To make the tests reliable, I've created a font based on ahem with OpenType features.

The current implementation works well, but it relies on a global state (using pango_cairo_font_map_set_default). Fixing that is not hard, but it relies to have one more argument through the whole layout. @SimonSapin do you think it's time to put everything related to the document/html/css that is needed for the layout in one class? I think that we have at least style_for, get_image_from_uri, enable_hinting (currently set to True in some functions where it should be available) and now font_config.

Answer 28 · 2016-10-26T17:39:38.000Z

There’s already a LayoutContext class passed around. Would that work?

Answer 29 · 2016-10-27T16:09:59.000Z

There’s already a LayoutContext class passed around. Would that work?

Yes. I've cleaned a couple of functions to use that too instead of multiple arguments.

I've open a PR (#374), I'd loooooove to have nice testers!

Answer 30 · 2016-11-03T23:43:46.000Z

I've open #376 and #377 if anyone wants to support other OSes.

Answer 31 · 2017-07-16T16:46:44.000Z

It's strange on Archlinux x64, the fonts can not be loaded, i think "local" is not yet working right?

@font-face {
font-family: 'OpenSans';
font-style: normal;
font-weight: 400;
src: local('OpenSans'), local('OpenSans-Regular'), url("fonts/openSans.woff2") format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
}

127.0.0.1 - - [16/Jul/2017:18:40:16] "GET /styles/fonts/openSans.woff2 HTTP/1.1" 200 15530 "" "WeasyPrint 0.36 (http://weasyprint.org/)"
WARNING: Failed to load font at "http://shop.de:4444/styles/fonts/openSans.woff2"
WARNING: Font-face "OpenSans" cannot be loaded

Answer 32 · 2017-07-16T16:54:13.000Z

i think "local" is not yet working right?

local works, as long as the font is installed on your system and can be found by Pango with this name. It works for me (OpenSans is installed on my system). If it's not found, you should get these warnings:

WARNING: Ignored `unicode-range: U+0-FF, U+131, U+152-153, U+2C6, U+2DA, U+2DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000` at 7:1, descriptor not supported.
WARNING: Failed to load local font "OpenSans"
WARNING: Failed to load local font "OpenSans-Regular"
WARNING: Failed to load font at "file:///tmp/fonts/openSans.woff2" (URLError: <urlopen error [Errno 2] No such file or directory: '/tmp/fonts/openSans.woff2'>)
WARNING: Font-face "OpenSans" cannot be loaded

unicode-range is not supported (see #378). And I can't reach http://shop.de:4444/styles/fonts/openSans.woff2, that's maybe why you get the error about it.

Answer 33 · 2017-07-16T16:59:46.000Z

thx for the information, shop.de is only bound to localhost right now.

you can see it loads the woff file successfully:
127.0.0.1 - - [16/Jul/2017:18:40:16] "GET /styles/fonts/openSans.woff2 HTTP/1.1" 200 15530 ""

i have installed a google font package (e.g. i can select it in libreoffice) i guess they are installed

I replaced the woff file with ttf, now the font works

Answer 34 · 2017-07-16T17:22:54.000Z

i have installed a google font package (e.g. i can select it in libreoffice) i guess they are installed

If you get the local warnings (and I assume you do, because you wouldn't get the warning about the woff2 file otherwise), it means that the font can't be found on your system.

A common mistake is to install a font only for an user (the one you use for LibreOffice), but not for the user launching WeasyPrint. If you use WeasyPrint in a web app (probably launched by a dedicated user), I bet that it's your problem. If you want to be sure, you can install this AUR, it installs the font for the whole system, not just an user.

I replaced the woff file with ttf, now the font works

FreeType probably doesn't support the woff2 format yet. It's hard to find anything but this short thread about that.

Answer 35 · 2017-07-16T17:40:43.000Z

okay, i replaced the local with url location and pointed it to the ttf and it works perfactly fine, didn't know why i even put both the locale thing here. I also think that woff does not work. Thanks for the additional information =)

Answer 36 · 2018-12-24T14:03:54.000Z

If anyone uses Alpine images you should install ttf-ubuntu-font-family and cairo==1.16 packages.

RUN apk update && \
 apk --update --upgrade add ttf-ubuntu-font-family jpeg-dev zlib-dev libffi-dev  gdk-pixbuf pango-dev && \
 apk add cairo==1.16.0-r0  --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/main/ --allow-untrusted