gnu-octave/MinusForge

A pkg resolver for Octave Packages.

MinusForge

A server side pkg resolver for Octave Packages.

How does pkg install -forge work?

Since about the year 2010 one can install Octave packages from Octave Forge via pkg install -forge. This mechanism did not change ever since.

Under the hood, the pkg command web scraps the latest package version from a fixed URL, that is currently redirected to Octave Forge.

In the following example for the io-package:

## Redirected to https://octave.sourceforge.io/io/index.html
html = urlread ("https://packages.octave.org/io/index.html");
html(isspace (html)) = [];
pat = "<tdclass=""package_table"">PackageVersion:</td><td>([\\d.]*)</td>";
t = regexp (html, pat, "tokens");

During the Octave Forge redesign this pkg design decision required a workaround, as can be seen in the following figure:

The problem is compatibility with older Octave versions.

In a second step the latest package version is retrieved from another fixed url:

https://packages.octave.org/download/io-2.6.4.tar.gz

that redirects to the centralized Octave Forge download location

https://downloads.sourceforge.net/octave/io-2.6.4.tar.gz?download

How does pkg list -forge work?

A second less famous use case that might have to be covered by this project is listing Octave Forge packages with their latest version.

From bug #39479 it can be conclude that this feature has not been widely embraced by users, as listing those 71 packages with their latest version number could take up to minutes.

Under the hood, first https://packages.octave.org/list_packages.php is read to get a list of all package names. In old Octave pkg versions before 2019 (bug #39479), the above mechanism is used to retrieve each package version, while since Octave 6 another hacky solution web scrapes the SourceForge download directory https://sourceforge.net/projects/octave/files/Octave%20Forge%20Packages/Individual%20Package%20Releases to extract the version numbers.

About MinusForge

The idea of MinusForge is to build a small PHP based web service download.php deployed on octave.org that handles all three types of pkg queries in combination with .htaccess-rules:

• https://packages.octave.org/io/index.html
  • Redirects to Octave Packages (same workaround as Octave Forge).
• https://packages.octave.org/download/io-2.6.4.tar.gz
  • Real url is resolved by download.php.
  • Limitation: only .tar.gz can really be installed by pkg.
• https://packages.octave.org/list_packages.php

  • Still forwarded to Octave Forge: (1) For simplicity (2) Documentation explicitly says

    The "-forge" option lists packages available at the Octave Forge repository.

and replies in a way that even old Octave pkg versions can resolve most of the Octave Packages repository.

Try it yourself

To demonstrate MinusForge is ready to take over, please try yourself the following URLs from octave.org and the most updated package in Octave history statistics-bootstrap:

• https://packages.octave.org/statistics-bootstrap/index.html (URL that would be generated by pkg-tool, if hosted on octave.org)
• https://packages.octave.org/download/statistics-bootstrap-5.2.1.tar.gz (URL that would be generated by pkg-tool, if hosted on octave.org; Note that the package version in the example (5.2.1) must be the latest available version, check on https://packages.octave.org/statistics-bootstrap)
• https://packages.octave.org/list_packages.php (redirects to Octave Forge)

To observe a limitation, see for example:

• https://packages.octave.org/download/pkg-dev.tar.gz (redirects to a ZIP-file, this would not be installable by the pkg-tool)