Toit programming language

This repository contains the Toit language implementation. It consists of the compiler, virtual machine, and standard libraries that together enable Toit programs to run on an ESP32.

Jaguar: Live reloading for the ESP32

You can use Toit to develop, update, and restart your ESP32 applications in less than two seconds through Jaguar. Jaguar is a small Toit program that runs on your ESP32. It uses the capabilities of the Toit virtual machine to let you update and restart your ESP32 applications written in Toit over WiFi whenever your source files change. Once set up, it is as easy as:

jag watch examples/hello.toit

It is also straightforward to install extra drivers and services that can extend the core functionality of your device. Add automatic NTP-based time synchronization without having to write a single line of code:

jag container install ntp examples/ntp/ntp.toit

You can watch a short video that shows how you can experience Jaguar on your ESP32 in less two minutes:

Community

Use this invite to join our Discord server, and follow the development and get help. We're eager to hear of your experience building with Toit.

We also use GitHub Discussions to discuss and learn and we follow a code of conduct in all our community interactions.

References

The Toit language is the foundation for the Toit platform that brings robust serviceability to your ESP32-based devices. You can read more about the language and the standard libraries in the platform documentation:

Contributing

We welcome and value your open source contributions to the language implementation and the broader ecosystem. Building or porting drivers to the Toit language is a great place to start. Read about how to get started building I2C-based drivers and get ready to publish your new driver to the package registry.

If you're interested in pitching in, we could use your help with these drivers and more!

Licenses

The Toit compiler, the virtual machine, and all the supporting infrastructure is licensed under the LGPL-2.1 license. The standard libraries contained in the lib/ directory are licensed under the MIT license. The examples contained in the examples/ directory are licensed under the 0BSD license.

Certain subdirectories are under their own open source licenses, detailed in those directories and the files they contain. These subdirectories are:

The subdirectory lib/font/matthew_welch/
Every subdirectory under packages/
Every subdirectory under lib/font/x11_100dpi/
Every subdirectory under src/compiler/third_party/
Every subdirectory under src/third_party/
Every subdirectory under third_party/

Installation

The instructions in this section don't cover the IDE integration. Follow the instructions below to set up Toit support for your editor.

Arch Linux

For Arch Linux (or variants such as Manjaro) use your favorite AUR helper to install the toit or toit-git package.

For example:

yay -S toit

Other platforms

We're actively working on simple installation steps for other platforms. For now, please use the build instructions below or download the .tar.gz files from the release page.

Building

Dependencies

Build system

To build Toit and its dependencies the build host requires:

GNU Make
CMake >= 3.13.3
Ninja
GCC
Go >= 1.16
python-is-python3: on Ubuntu machines
glibc-tools: optional and only available on newer Ubuntus

If you are using a Linux distribution with apt capabilities, you can issue the following command to install these:

sudo apt install build-essential cmake ninja-build golang

For builds targeting ESP32 hardware additional requirements might be in effect depending on the build host's architecture, see paragraph ESP32 tools.

For builds targeting RISC-V, ARM32, or ARM64 hardware, see the Other platforms README.

ESP-IDF

The Toit VM has a requirement for the Espressif IoT Development Framework, both for Linux and ESP32 builds (for Linux it's for the Mbed TLS implementation).

We recommend you use Toitware's ESP-IDF fork that comes with a few changes:

Custom malloc implementation
Allocation-fixes for UART, etc.
LWIP fixes

The fork's repository has been added as a submodule reference to this repository, so doing a recursive submodule init & update will establish everything nedded:

git submodule update --init --recursive

If the submodule update step fails with:

Submodule path 'esp-idf/components/coap/libcoap': checked out '98954eb30a2e728e172a6cd29430ae5bc999b585'
fatal: remote error: want 7f8c86e501e690301630029fa9bae22424adf618 not valid
Fetched in submodule path 'esp-idf/components/coap/libcoap/ext/tinydtls', but it did not contain 7f8c86e501e690301630029fa9bae22424adf618. Direct fetching of that commit failed.

try following the steps outlined here. It is an issue in the upstream ESP-IDF repository caused by the tinydtls component having changed its remote URL.

To use the offical ESP-IDF, or any other variation, you need to add the Toit specific patches first.

Then make sure it is available in your file system and point IDF_PATH to its path instead before building.

export IDF_PATH=<A_DIFFERENT_ESP_IDF>

ESP32 tools

If you want to build an image for the ESP32, install the ESP32 tools.

On Linux:

$IDF_PATH/install.sh

The default location of $IDF_PATH is under ./third_party/esp-idf

For other platforms, see Espressif's documentation.

Remember to update your environment variables:

source $IDF_PATH/export.sh

Build for host machine

Make sure the required build tools are installed as described in dependency sections ESP-IDF and Build system above.

Then run the following commands at the root of your checkout.

make all

NOTE

These instructions have been tested on Linux and macOS.

Windows support is still preliminary, and the build instructions may differ for Windows. Let us know on the discussions forum how we can improve this README.

This builds the Toit VM, the compiler, the language server and the package manager.

You should then be able to execute a toit file:

build/host/sdk/bin/toit.run examples/hello.toit

The package manager is found at build/host/sdk/bin/toit.pkg:

build/host/sdk/bin/toit.pkg init --project-root=<some-directory>
build/host/sdk/bin/toit.pkg install --project-root=<some-directory> <package-id>

Debugging

See https://github.com/toitlang/toit/wiki/Debugging.

IDE integration

Toit has a VS Code extension. You can either use the published extension or build it yourself from the sources.

In the VS Code extension (version 1.3.7+) set the toitLanguageServer.command setting to ["PATH_TO_SDK/bin/toit.lsp", "--toitc=PATH_TO_SDK/bin/toit.compile"], where PATH_TO_SDK is the path to your build/host/sdk/ folder in the Toit repository.

This makes the extension use the language server that was compiled in the build step.

Other IDEs

The Toit language server is independent of VSCode and can be used with other IDEs. It can be started with:

build/host/sdk/bin/toit.lsp --toitc=build/host/sdk/bin/toit.compile

See the instructions of your IDE on how to integrate the language server.

There are syntax highlighters for VIM and CodeMirror in the ide-tools repository.

Build for ESP32

Make sure the environment variables for the ESP32 tools are set, as described in the dependencies section.

Build firmware that can be flashed onto your ESP32 device. The firmware is generated in build/esp32/firmware.envelope:

make esp32

If you want to use esptool.py to flash the generated firmware on your device, you need to extract the firmware.bin file and pass it to esptool.py. Assuming your device is connected through /dev/ttyUSB0 you can do this:

build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    extract --format=binary -o firmware.bin
python third_party/esp-idf/components/esptool_py/esptool/esptool.py \
    --chip esp32 --port /dev/ttyUSB0 --baud 921600 \
    --before default_reset --after hard_reset write_flash -z \
    --flash_mode dio --flash_freq 40m --flash_size detect \
    0x001000 build/esp32/bootloader/bootloader.bin \
    0x008000 build/esp32/partitions.bin \
    0x010000 firmware.bin

By default, the image boots up but does not run any application code. You can use your own entry point by installing it into the firmware envelope before extracting the firmware.bin file:

build/host/sdk/bin/toit.compile -w hello.snapshot examples/hello.toit
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install hello hello.snapshot
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    extract --format=binary -o firmware.bin

Alternatively, you can also specify the entry point through the ESP32_ENTRY make variable and let the Makefile handle the flashing:

make flash ESP32_ENTRY=examples/hello.toit ESP32_PORT=/dev/ttyUSB0

Adding multiple containers

You can add more containers before you extract firmware.bin, so you firmware envelope can have any number of containers.

build/host/sdk/bin/toit.compile -w hello.snapshot examples/hello.toit
build/host/sdk/bin/toit.compile -w ntp.snapshot examples/ntp/ntp.toit

build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install hello hello.snapshot
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install ntp ntp.snapshot

You can list the containers in a given firmware envelope:

build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container list

The listing produces JSON output that can be processed by other tools:

{ "hello": {
    "kind" : "snapshot",
    "id"   : "f0b7e859-9188-52d9-8be3-856bd0e75919"
  },
  "ntp": {
    "kind" : "snapshot",
    "id"   : "6efefb4b-aa91-5600-ba7d-f76a8dc0ac01"
  }
}

Adding container assets

Containers have associated assets that they can access at runtime. Add the following code to a file named assets.toit:

import system.assets

main:
  print assets.decode

If you run this on an ESP32, you'll get an empty map printed becase you haven't associated any assets with the container that holds the code.

To associate assets with the container, we first construct an encoded assets file and add this README.md file to it.

build/host/sdk/tools/assets -e encoded.assets create
build/host/sdk/tools/assets -e encoded.assets add readme README.md

Now we can add the encoded.assets to the assets container at install time:

build/host/sdk/bin/toit.compile -w assets.snapshot assets.toit
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install --assets=encoded.assets assets assets.snapshot

If you update the source code in assets.toit slightly, the printed information will be more digestible:

import system.assets

main:
  readme := assets.decode["readme"]
  // Guard against splitting a unicode character by
  // making this non-throwing.
  print readme[0..80].to_string_non_throwing

You'll need to reinstall the container after this by recompiling the assets.toit file to assets.snapshot and re-running:

build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install --assets=encoded.assets assets assets.snapshot

NOTE

To access the device /dev/ttyUSB0 on Linux you probably need to be a member of some group, normally either uucp or dialout. To see which groups you are a member of and which group owns the device, plug in an ESP32 to the USB port and try:

groups
ls -g /dev/ttyUSB0

If you lack a group membership, you can add it with

sudo usermod -aG dialout $USER

You will have to log out and log back in for this to take effect.

Configuring WiFi for the ESP32

You can easily configure the ESP32's builtin WiFi passing it as configuration when you extract the firmware.bin file:

echo '{ "wifi.ssid": "myssid", "wifi.password": "mypassword" }' > config.json
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    extract --format=binary -o firmware.bin --config config.json

The Makefile also has the ESP32_WIFI_SSID and ESP32_WIFI_PASSWORD make variables to support this, if you prefer flashing through make:

make flash ESP32_ENTRY=examples/http/http.toit ESP32_WIFI_SSID=myssid ESP32_WIFI_PASSWORD=mypassword

This allows the WiFi to automatically start up when a network interface is opened.

pedrominatel/toit

Toit programming language

Jaguar: Live reloading for the ESP32

Community

References

Contributing

Licenses

Installation

Arch Linux

Other platforms

Building

Dependencies

Build system

ESP-IDF

ESP32 tools

Build for host machine

Debugging

IDE integration

Other IDEs

Build for ESP32

Adding multiple containers

Adding container assets

Configuring WiFi for the ESP32