awalsh128/cache-apt-pkgs-action

Failing to correctly recover LaTeX packages

tueda opened this issue · 15 comments

tueda commented

I tried to use this action to cache LaTeX packages:

      - uses: awalsh128/cache-apt-pkgs-action@v1.1.1
        with:
          packages: texlive-latex-base  # and other LaTeX packages
          version: 1.0

An example repository is here. Storing cache (~61M) seems to work (log), but in the next commit, restoring the cache causes the subsequent step to fail (log):

Run pdflatex test.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.

kpathsea: Running mktexfmt pdflatex.fmt
mktexfmt: mktexfmt is using the following fmtutil.cnf files (in precedence order):
mktexfmt: mktexfmt is using the following fmtutil.cnf file for writing changes:
mktexfmt:   /home/runner/.texlive2019/texmf-config/web2c/fmtutil.cnf
mktexfmt [INFO]: writing formats under /home/runner/.texlive2019/texmf-var/web2c
mktexfmt [INFO]: did not find entry for byfmt=pdflatex, skipped
mktexfmt [INFO]: total formats: 0
mktexfmt [INFO]: exiting with status 0
I can't find the format file `pdflatex.fmt'!
Error: Process completed with exit code 1.

which sounds like some post-install processes might be omitted or performed in an incompatible way.

awalsh128 commented

One way we can see if the file is there statically is to look inside the package itself. Can you run this for me?

dpkg -c package_file.deb

You can also use the -e switch to extract the postinst.sh script. One of them should reveal the file. Otherwise it may be an artifact generated deeper down in the post install.

tueda commented

Thank you for your reply! I put

dpkg -c /var/cache/apt/archives/texlive-base_2019.20200218-1_all.deb

and

dpkg -e /var/cache/apt/archives/texlive-base_2019.20200218-1_all.deb
head -n 100000 DEBIAN/*

in the workflow (log).

I noticed that, for the successful case (without restoring cache, see the above log)

kpsewhich fmtutil.cnf
kpsewhich -engine=pdftex pdflatex.fmt

print

/usr/share/texmf/web2c/fmtutil.cnf
/var/lib/texmf/web2c/pdftex/pdflatex.fmt

but restoring apt packages from the cache causes them to print nothing (log), though /usr/share/texmf/web2c/fmtutil.cnf -> /var/lib/texmf/fmtutil.cnf-DEBIAN exists.

awalsh128 commented

Added a regression test for this and confirmed it fails on the CI as well (test result log). When I run the action locally (action_local.sh), the command prints a result which seems to indicate an artifact still gets left around even after an APT purge and autoremove.

I'll update the ticket once I have more.

tueda commented

Maybe a similar issue for MPI:

      - uses: awalsh128/cache-apt-pkgs-action@v1.1.1
        with:
          packages: libmpich-dev
          version: 1.0
BoboTiG commented

Gentle ping to notice we also have the LaTeX issue. Do you need more data to move forward?

Using that code:

      - uses: awalsh128/cache-apt-pkgs-action@latest
        with:
          packages: dvipng texlive-latex-extra texlive-fonts-extra texlive-science dictzip
          version: 1.0
  • 👍1
awalsh128 commented

Hi @BoboTiG , sorry for the lag here. Replicating the issue for yourself as noted above in comment would be helpful.

Really the main thing is what in that APT package is being omitted when it is backed up. That is what I am trying to find to see if we can modify this action to accommodate. I am thinking this is very similar to #44 (comment)

awalsh128 commented

@BoboTiG thanks for your help with the other ticket. I'll be able to dig more into this issue this weekend and will do a patch release with your most recent commit.

  • 👍1
awalsh128 commented

I was able to debug some of the install script behavior to work correctly in general but the regression is still failing. The execute script feature is in dev. You can run this if you want but I am not seeing any scripts to execute yet.

uses: awalsh128/cache-apt-pkgs-action@dev
with:
    packages: yourpackage
    version: 1
    execute_install_scripts: true

WARNING: The outputs are broken on the action. This shouldn't matter though if you don't use them.

I'll update the ticket once I fix some other breaks and can get around to debugging this with the new feature.

awalsh128 commented

I may be possibly hitting the reaches of what the action can do. The texlive-latex-base.postinst script requires that it is called within a maintainer script (see https://www.debian.org/doc/debian-policy/ap-flowcharts.html).

awalsh128@lithium:~/cache-apt-pkgs-action-ci/devtools$ sh -x /tmp/texlive-latex-base.postinst configure
+ set -e
+ update-texmf-config format map
dpkg-trigger: error: must be called from a maintainer script (or with a --by-package option)

Type dpkg-trigger --help for help about this utility.

The execute_install_scripts feature really only works for simple shell scripts that aren't dependent on the Debian package manager commands. To be able to execute this postinst script I would need to run sudo dpkg --configure texlive-latex-base. Although this requires that the package be registered in the package manager database. You see how this can be quite the rabbit hole.

I am going to let the feature stay but I think I have hit the limits of what this action can do. Open to ideas but without anything new I think I may just mark this as "won't fix". 😞

BoboTiG commented

No worries, it is already great to have such an action :)

awalsh128 commented

Published the latest version of the action. Going to close this out.

https://github.com/awalsh128/cache-apt-pkgs-action/releases/tag/v1.2.0

asmeurer commented

If anyone ever figures out the solution to the I can't find the format file pdflatex.fmt'!` problem when using this action with texlive please share. Reinstalling latex in each build is a huge pain.

  • 👍2
asmeurer commented

I guess the solution is to use a docker image. I was able to use https://github.com/xu-cheng/latex-docker, but in general you might have to build your own docker image with the packages you need.

koppor commented

One can use the action https://github.com/zauguin/install-texlive, which also supports caching and uses way less disk space.

koppor commented

@asmeurer A well-maintained docker image is https://gitlab.com/islandoftex/images/texlive. It is originating a team very close to overleaf and latex development in general.