generator: reconsider tmpfs usage

[lucab]

torcx currently mounts its own tmpfs for unpacking purposes.

There are a bunch things to be re-considered in that regard:

• moving out of tmpfs: concern expressed by @philips and @dm0-, as we go forward this will increse memory pressure (plus, CL doesn't setup swap). However, at early-boot (generator) we may not have yet FS set up
• tmpfs options: we should expose options to tune tmpfs parameters to limit resource usage, in a format compatible with mount/systemd-mount. This is a low-priority wishlist for the moment and can be easily exposed via config.json.

We were aware of tmpfs concerns while designing this so torcx runtime paths are still easily switchable to somewhere else, but I don't have a clean alternative solution relying on local-fs right now. Going in this direction may require:

1. fiddling with local-fs.target so that it is reached in initrd before pivot_root happens and generators are run. @crawford is looking into the feasibility of this.
2. moving part of torcx into initrd, and make all mountpoints available for initrd (similar to what ignition does). This is similar to the above and adds some more complexity, so we opted away from it.

I'm also not sure what are the implications of PXE-booting for such discussion (do we have any guarantees of a RW storage in there?).

[squeed]

What would it be like if torcx addons were squashfs images?

[euank]

I think mounting a squashfs in /run would be a quite neat solution. I can't think of any reason why that wouldn't work, other than whether we need be concerned with backwards compatibility here or not.

It would be possible to support both paths if we need backwards compatibility without too much difficulty as well.

@lucab, wdyt? Was that an option that was considered? Did it end up not getting considered due to the original desire to use an OCI image?

[lucab]

Was that an option that was considered? Did it end up not getting considered due to the original desire to use an OCI image?

For reference, we didn't consider squashfs initially.
Torcx groundwork happened around the time of OCI image-spec 1.0-rc4, and we were expecting to just adopt that. However, the archive format evolved into something completely different and unnecessarily complex (for our usage), took a long time to stabilize to 1.0 and OCI nowadays still lacks a standardized (trusted) distribution mechanism.
As such, in order to deliver the generator (and tectonic-torcx), we just kept using the same tgz archiving format without all of OCI layout, and switched to simple HTTP GET-ting.