Rust local compilation

[tcr]

We can generate an SDK while building OpenWRT which allows us to cross-compile applications to Tessel 2. These SDK bundles include GCC utilities for compiling and linking mipsel binaries, as well as shared libraries and headers for packages (e.g. libudev, libusb, etc.) These are versioned and thus 1:1 with openwrt-tessel builds, e.g. 0.0.16 for the latest build.

Since we've recently been able to build a macOS version of the SDK, we are now unblocked from having cross-compilation support for macOS and Linux. Windows support, while not possible yet, may be unblocked by the Linux subsystem or successfully compiling the SDK with Cygwin.

My proposal is we move to local compilation of Rust binaries on Linux and macOS:

1. This is a faster process than using the cross-compilation server.
2. It allows for locally linked crates outside of the current directory.
3. It allows users to select which nightly they want to cross-compile against.

The downside is that we will need to maintain the cross-compilation server for the near future until Windows support is solved, creating two projects to maintain.

The first step is adding support for SDKs to t2-cli. One proposal is this:

1. t2 install-sdk will download and install the latest SDK version. It will be stored at ~/.tessel/sdk/.
2. Because new OpenWRT builds require new versions of the SDK (~100M packed, ~250M unpacked), we should consider being prudent with new OpenWRT releases. Storing the delta between versions may be lower cost, but more complex.
3. We always expect you want the latest SDK version. Storing multiple versions of the SDK is wasteful; we can allow overrides with a --sdk parameter.
4. When compiling against a newer Tessel OpenWRT than your SDK, we can throw a warning but let you continue anyway. When your SDK is newer than your Tessel, you should update your Tessel (so code doesn't fail to run that relies on dependencies).

We will need a cross-compiled libstd for Rust. (TODO: Using xargo or rust-cross-libs.sh?)

1. In the install-sdk step, create the ~/.tessel/rust folder.
2. Add ~/.tessel/rust/tessel2.json as below to reference as target.json.
3. Inspect the current rustc version. If it's stable, download a cross-compiled libstd into ~/.tessel/rust/1.13.0/libstd (or whatever version).

These libstd versions should be generated for each stable version of Rust.

Next is refactoring our Rust code for t2-cli. --rustcc should continue to exist but only be defaulted on Windows. For other programs, we can do the following:

1. t2 install-sdk will install the OpenWRT SDK and the rust libstd.
2. t2 init --lang=rust will install the SDK if it is missing, then initialize a Rust directory.
3. t2 run Cargo.toml will check if the SDK exists. If it does not, it will fail and ask you to run t2 install-sdk first.
4. t2 run Cargo.toml will check for that you are using a stable rustc and that the libstd is installed. If not, it will fail and ask you to run t2 install-sdk first. If you are running a beta or nightly rustc, it will warn you that you need to cross-compile libstd (TODO: What are those instructions, do we need to provide them?) and say that std may not be found.
5. On success, it will cross-compile the binary, bundle the tarball, and proceed as our current code does.

Finally, we will need to add a --bin target to t2 run Cargo.toml to disambiguate between multiple binaries being run on Tessel.

User requirements: the will need the user to have cargo and rustc installed.

---

Target JSON tessel2.json:

{
    "arch": "mips",
    "cpu": "mips32r2",
    "data-layout": "e-m:m-p:32:32-i8:8:32-i16:16:32-i64:64-n32-S64",
    "llvm-target": "mipsel-unknown-linux-uclibc",
    "target-env": "uclibc",
    "features": "+mips32,+soft-float",
    "os": "linux",
    "vendor": "openwrt",
    "relocation-model": "pic",
    "target-endian": "little",
    "target-pointer-width": "32",
    "exe-allocation-crate": "alloc_system",
    "lib-allocation-crate": "alloc_system",
    "linker": "mipsel-openwrt-linux-gcc",
    "ar": "mipsel-openwrt-linux-ar",
    "dynamic-linking": true,
    "executables": true,

    "c-compiler": "mipsel-openwrt-linux-gcc",
    "c-flags": "-Wall -g -fPIC -mips32 -mabi=32 -O2",
    "c-link-flags": "-shared -fPIC -g -mips32",
    "c-triple": "mipsel-openwrt-linux"
}

References:

• Cross-compilation steps: https://gist.github.com/kevinmehall/16e8b3ea7266b048369d
• xargo: https://github.com/japaric/xargo

[kevinmehall]

Do we also need to version the libstd for Rust for a specific nightly version?

It needs to be compiled with the exact rustc you use, yes. This may be an argument for supporting stable rather than nightly.

[badboy]

Do we already have a target.json?
I have a hacky implementation of using a local rustc already here: https://github.com/badboy/t2-cli/tree/cargo-tessel, hopefully we can build the proper version on that.

[badboy]

It needs to be compiled with the exact rustc you use, yes. This may be an argument for supporting stable rather than nightly.

If we still need the allocation lib workaround, then we need to use nightly.

[kevinmehall]

1. t2 run Cargo.toml will check for 1) a .cargo/config that lists the linker target being mipsel-unknown-linux-gcc

.cargo/config is unnecessary if the custom target specifies the right linker.

[kevinmehall]

@badboy target.json I've been using is here. Note that you can specify "exe-allocation-crate": "alloc_system", in target.json on stable.

[badboy]

Perfect, than we can target stable I guess.

[kevinmehall]

Also, we should switch that target.json to "llvm-target": "mipsel-unknown-linux-uclibc", "target-env": "uclibc" now that rustc supports a mipsel-unknown-linux-uclibc target.

[tcr]

@badboy @kevinmehall Rolled up the modified target.json into the issue, also reworked how install-sdk will pull down stable libstd and not require nightly.

[johnnyman727]

In terms of the CLI interaction, would it be possible to detect if it's possible to compile locally? Maybe checking to see if the SDK is installed in ~/.tessel/sdk and rustc and cargo are on the PATH? In any case, I think it would be great if we could first check if local compilation is possible and if not, resort to remote compilation with a visible warning that you should be prepared to install requirements or local compilation (unless user is on Windows, then no warning). More formally:

1. Check if local Rust compilation is possible
2. If it is, do that using instructions above
3. If not, send bundle to remote compilation server
4. Print instructions for installing dependencies for local compilation and why

The benefit here is that folks can quickly go from zero to blinking LEDs without having to worry about downloading a 100MB+ package and all the other dependencies. I think that will help keep the user experience fast and smooth which people have come to expect from Tessel.

I don't see any mentions of rustup here, would this cross compilation not need it if we use xargo? I don't quite understand how the process works if we don't use @kevinmehall's cross compilation script or rustup.

[kevinmehall]

You would still use rustup to install rustc and cargo. The t2 install-sdk would install a libstd that enables that rustc to compile for T2.

[johnnyman727]

@kevinmehall it's already possible to link to an alternative stdlib with rustc?

[kevinmehall]

It's not an alternative libstd, it's the default one, compiled to a new target. All rustup target add does is drops some files in ~/.rustup, so t2 install-sdk can do the same.

[johnnyman727]

It's not an alternative libstd, it's the default one, compiled to a new target

I understand that, but how do you tell rustc to use the libstd compiled for Tessel instead of the libstd that you probably already have for your host computer (and vice versa when you're not building for Tessel)?

[kevinmehall]

cargo build --target=tessel2 looks for a tessel2.json on RUST_TARGET_PATH and a libstd (and dependencies) in its sysroot path compiled for the same tessel2 target.

[johnnyman727]

👍 thanks for explaining that.

[tcr]

See the status of local compilation here: #46

[tcr]

Merged 👏