[Builder] Is it possible to fix fonts without changing instances, and/or run more granular fix operations such as fix-gasp only?

Question

[Builder] Is it possible to fix fonts without changing instances, and/or run more granular fix operations such as fix-gasp only?

Closed this issue 4 months ago · 9 comments

I’m attempting to use the gftools builder to build a font, and the font source has instances that differ from the Google Fonts flavor of instances in two ways, which I’d like to preserve:

Style names have spaces. So, it’s "Semi Bold" rather than "SemiBold."
There is no Extra Bold instance in the font – it goes from Bold (700) to Black (900).

However, when running the gf builder, and include the fix operation in a custom recipe, the operation changes the instance names (removing spaces) and inserts a new instance of ExtraBold at wght=800.

I’d rather not discard the fix operation altogether, as I assume that it is genuinely useful to do things like adding the gasp and prep tables. However, I can’t use the builder if it adds and changes instances.

Is there a way to add args to the fix operation to limit name/instance changes? Alternatively, is there some way to use more granular operations in custom recipes, such as only running fix-gasp and fix-prep? I see these aren’t operations listed as available in the docs, and when I try to test them, the build fails.

If the option for this type of flexibility doesn’t yet exist, I do think the builder would be more widely useful if it did exist.

Thanks for your time!

Answer 1 · 2024-05-07T21:36:41.000Z

Hrm. gftools-fix exists to fix things up to Google Fonts specifications; the clue's in the name, and it's expected that it should be opinionated in certain ways. As it happens, there are scripts called gftools-fix-gasp, gftools-fix-hinting, etc. But they don't take a in-file and an out-file, which means they're not useable in builder operations yet.

I'm working on a better tie-up between gftools-fix and fontbakery, such that eventually you'll be able to fix things by fontbakery check ID, but we're a way away from that yet; maybe later in the year.

Answer 2 · 2024-05-08T15:28:48.000Z

Hey Simon, thanks so much for explaining that! Totally understandable that the GF fix script would exist to fix things to Google Fonts specs.

Looking again, I see there is an exec option...

exec: An escape hatch for arbitrary processing - executes the executable named in the exe argument with the arguments provided in args.

Am I correct in interpreting that this would be a possibility to use in pointing fonts towards a custom post-processing script, or arbitrary CLI commands? If so, it seems like this might solve my problem pretty perfectly.

However, I’m not yet having success, perhaps because I haven’t quite figured out how to feed in the font as an argument/target for the executable.

I’ve tried various versions of the below:

familyName: Familyname
buildVariable: false
buildWebfont: false
interpolate: true
autohintOTF: true
outputDir: ./fonts
sources:
  - sources/glyphs/Familyname.glyphs
recipeProvider: googlefonts
recipe:
  fonts/variable/Familyname[slnt,wght].ttf:
    - source: sources/glyphs/Familyname.glyphs
    - args: --filter ...  --filter FlattenComponentsFilter --filter DecomposeTransformedComponentsFilter
      operation: buildVariable
    - args: "--autofix"         # QUESTION: Is there a way to target the VF in these args?
      exe: "gftools fix-gasp"
      operation: exec
    - operation: rename
      name: Familyname Variable
    - args: "--src stat.yaml"
      needs: "fonts/variable/Familyname[slnt,wght].ttf"
      postprocess: buildStat

Is there a way to specify the font as a target in the args of the exec operation?

Answer 3 · 2024-05-09T09:28:25.000Z

Am I correct in interpreting that this would be a possibility to use in pointing fonts towards a custom post-processing script, or arbitrary CLI commands?

Yes... but:

gftools-builder sews together operations into a pipeline of commands, each of which act on intermediate files. It uses a bunch of temporary files, which it dreams up names for on the spot; each operation acts on a file called $in and is expected to generate a file called $out, and gftools-builder fills in these variables with the names of the intermediate files. Most of the time you don't see the $in and $out variables because it's implicit in the definition of the operation, but they're there. The $out of one operation becomes the $in of the next operation in the pipeline, until the final $out is the file you're trying to build:

But for this to work, operations must be commands which take a file and spit out a file: take a source and compile to TTF; take TTF, fix it, produce fixed TTF; and so on. However, something like gftools fix-gasp takes a file, fixes it, and writes back to the original file. There is no output file. So it's not quite suitable for use in an exec step, as I mentioned above.

I suppose you could probably force it to work with something like this:

    - exe: "gftools fix-gasp $in && cp $in $out"
      operation: exec

In other words, fix the file in-place, and then copy the fixed version to where the builder is expecting to find it.

Answer 4 · 2024-05-09T15:48:03.000Z

Thank you so much, @simoncozens! This is a really helpful explanation, and I appreciate both the caution and the example of how it could be done.

My wider motivation is to perhaps create modified version of gftools fix, and run that, but then I realized it might be more direct to use the individual gftools commands. I’ll think and probably experiment to see whether something makes sense.

Answer 5 · 2024-05-09T16:19:23.000Z

Yeah. My end goal is to have a command where you can go gftools fix -c some.fontbakery.check.id and it'll fix that particular problem for you. But again, that's going to take a while.

Answer 6 · 2024-05-14T14:44:15.000Z

Ohh that would be very cool. I’ll probably use the basic exec workflow for now, but keep my eye on the fontbakery-based fixes for the longer term!

Answer 7 · 2024-05-23T22:05:18.000Z

Okay, I’m just trying to apply this exec approach, and hitting a couple of bumps.

If I don’t include args in the exec operation, it fails with a message like this:

gftools builder Familyname/config.yaml
Traceback (most recent call last):
  File "/Users/stephennixon/.pyenv/versions/3.11.3/bin/gftools", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/stephennixon/.pyenv/versions/3.11.3/lib/python3.11/site-packages/gftools/scripts/__init__.py", line 91, in main
    mod.main(args[2:])
  File "/Users/stephennixon/.pyenv/versions/3.11.3/lib/python3.11/site-packages/gftools/builder/__init__.py", line 359, in main
    pd.walk_graph()
  File "/Users/stephennixon/.pyenv/versions/3.11.3/lib/python3.11/site-packages/gftools/builder/__init__.py", line 298, in walk_graph
    edge["operation"].validate()
  File "/Users/stephennixon/.pyenv/versions/3.11.3/lib/python3.11/site-packages/gftools/builder/operations/exec.py", line 13, in validate
    raise ValueError("No arguments given")
ValueError: No arguments given

If I do add an args item with a blank value, I can get past the above error, but I am still having trouble getting cp $in $out to work.

My ultimate goal, in the near term, is to adapt a version of the general GF fix script to do most of the same things, but without changing/adding instances.

For now, I’m just trying to get the gasp fix to work, to verify that the exec option is possible to use. But, that is failing, too. So, to test things even more simply, I’m just trying to run a basic test script to see whether the $in $out variables are working as needed.

I’ve written a simple test Python script that writes a txt file with the current time and the input args:

import sys
from datetime import datetime

args = sys.argv
current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")

with open("test_file.txt", "w") as file:
    file.write(f"Current time: {current_time}\n\n")
    file.write(str(args))

I’ve set this up in my config.yaml with this step:

    - args: ""
      exe: "python ../scripts/test.py $in $out && cp $in $out"
      operation: exec

This saves test_file.txt next to the config.yaml. However, it is outputting a file that appears to have no args beyond the basic python file (even though I am attempting to feed in the $in $out values:

Current time: 2024-05-23 18:03:24

['../scripts/test.py']

It then fails to run the cp $in $out command, so the build stops – I think because this command fails to output a new temporary font file for the next operation to work on.

[3/7] exec
FAILED: /var/folders/nb/mc9zt2n930vd_jkbg0kn2d680000gn/T/tmpskn698iv 
/Users/stephennixon/type-repos/Familyname/venv/bin/python3 -m gftools.builder.jobrunner python ../scripts/test.py   && cp    
python ../scripts/test.py
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
[4/7] exec
FAILED: /var/folders/nb/mc9zt2n930vd_jkbg0kn2d680000gn/T/tmpl4_2fiz0 
/Users/stephennixon/type-repos/Familyname/venv/bin/python3 -m gftools.builder.jobrunner python ../scripts/test.py   && cp    
python ../scripts/test.py
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
ninja: build stopped: subcommand failed.

I’ve also tried formatting the variables in the exe value like ${in} ${out}, but this doesn’t make a difference.

If I leave out the exec operation step altogether, the font build completes, and fonts are built. However, they are built without the necessary fixes, and I need some way to control this.

Am I still missing some part of the syntax?

Answer 8 · 2024-05-24T06:13:36.000Z

exe should be the name of the executable and args should be the arguments to it.

Answer 9 · 2024-05-28T14:48:17.000Z

Ohh haha, it is so simple, after all. Thank you for taking the time to spell that out for me.

In case it helps others, here’s the required item in the recipe:

    - args: " ../scripts/test.py $in $out && cp $in $out"
      exe: "python"
      operation: exec

This seems to work flawlessly for an arbitrary Python script.

Or, for my earlier goal of running the gftools fix-gasp operation, the recipe step looks like this:

    - args: "fix-gasp --autofix $in && cp $in.fix $out"
      exe: "gftools"
      operation: exec

For my purposes, it might be more readable and flexible to just add one recipe step for a python script to make necessary fixes, rather than figuring out the exact input/output args needed for various gftools scripts. But, I may change my mind on that, after trying both.

Thanks for your help here, Simon! I’m happy for this issue to be closed, unless you want to keep it open for some reason.