ix-plugin-hub/iocage-plugin-index

esphome

duffel90 opened this issue ยท 23 comments

esphome is not working.

esphome 1.20.1

ERROR: Failed building wheel for cryptography
error: can't find Rust compiler

after installing rust and installing cryptography it seems to work
so pleas adding pkg install rust to the skript

ok esphome is running. But when I will install an update I get this error.

Processing esppower (board: esp-wrover-kit; framework: arduino; platform: platformio/espressif32@3.2.0)

Platform Manager: Installing platformio/espressif32 @ 3.2.0
Platform Manager: espressif32 @ 3.2.0 has been installed!
The platform 'platformio/espressif32@3.2.0' has been successfully installed!
The rest of the packages will be installed later depending on your build environment.
Tool Manager: Installing platformio/toolchain-xtensa32 @ ~2.50200.0
Error: Could not find the package with 'platformio/toolchain-xtensa32 @ ~2.50200.0' requirements for your system 'freebsd_amd64'

Sorry, I meant to have ESPHome removed from the plugin list a few months ago.

It's been awhile since these toolchains were included with platformIO, so it was only a matter of time before the plugin broke. There's some links to FreeBSD sources, if you want to try compiling the missing toolchains yourself.

I made several attempts awhile back but was never successful at getting this work

Thanks for the link @trombik (and hints about using portshaker)

I had another go at this and I'm happy to say I manages to get it compiled this time.

It did take almost an hour to compile so I don't think it's will be realist to build during the plugin install (or update)

It could also be possible to provide the pre-compile pkg with the plugin but I'm not sure that's something I can continuously maintain on my own.

@tprelog what would be a feasible solution?

  1. build the package in a part of CI of ix-plugin-hub/iocage-plugin-index
  2. download the package from somewhere, but NOT from the official package repository, include the package, and provide it with the plugin

what would be a feasible solution?

@trombik -- I think, strictly speaking for this ESPHome plugin, having the pre-compiled toolchains available somewhere online may suffice. Then the plugin could download and install them as needed

For right now, I'm going to include the toolchains I compiled yesterday, with the plugin.

Truth be told, this is over my head but if I was a little smarter, I would attempt to build the both, xtensa-esp32 and esp-quick-toolchain, using the (cirrus?) CI and somehow provide those built packages automagically to a github repo.

@tprelog building packages on cirrus-ci is what I'm trying now. esp-quick-toolchain is easy, but xtensa-esp32-elf is tricky due to limited time in a task (two hours at max). just made the first successful run for 13.0-RELEASE. will publish packages by tomorrow.

for those who are curious, the CI script can be found at:

https://github.com/trombik/xtensa-esp32-elf/blob/ci/.cirrus.yml

for simple ports, see .cirrus.yml in esp-quick-toolchain.

at the least, the build process works, but looks like artifacts, or the library behind it, has a bug which prevents the task from keeping the packages for tests in later tasks.

cirruslabs/cirrus-ci-docs#913

before someone bumps into an issue, i should mention that the port has two FLAVORs and depending on the esp-idf being used, you have to choose either idf4 or idf3. FLAVOR corresponds to esp-idf version, idf4 for 4.x, idf3 for 3.x.

this is quite important if you are indirectly using esp-idf, i.e. arduino projects.

By now the updated requirements have made it to the plugin manifest for the 12.2-RELEASE, and I've also updated the plugin to use the toolchains I compiled from @trombik ports. A clean install of the ESPHome plugin should (at least for now) compile both ESP8266 and ESP32 firmwares out of the box.


For an existing plugin, running a Plugin UPDATE from the TrueNAS UI should also work - but may require some additional steps. If the ESPHome service fails to start after a Plugin UPDATE, you'll need re-create the ESPHome virtualenv. This can easily be done from the jail's console menu.

Use number 7 to remove

image

When the virtualenv is removed, number 7 will change to install

image

@tprelog the packages are now available in CI.

CI page
https://cirrus-ci.com/github/trombik/xtensa-esp32-elf

in a build page, you can see a list of all tasks.

https://cirrus-ci.com/build/5994087745585152

each CI build has generate_repo task.

https://cirrus-ci.com/task/5629361672945664

the task page has (if the build is successful) an artifact, repos. in the artifact, packages for releases are available. you can download a file, or all files as a zip. the packages are not yet signed.

ideally, there should be a GitHub Pages that publishes packages in repositories, and users should be able to just add a pkg.conf(8) to use the repository. however, due to long build time (and quirks in cirrus-ci), that feature will not be implemented soon.

for esp-quick-toolchain, the same.

i'm updating the toolchains. new FLAVOR includes idf43, idf42, and idf41. idf4 FLAVOR has been removed. there is no idf5 for the master branch yet. please see the progress at trombik/xtensa-esp32-elf#29

Can anyone please help with the ESPHome plugin?

My only experience with FreeBSD was using jails on FreeNAS / TrueNAS CORE and I really only figured out as much as I needed to know in order to make things work. Now I am no longer using TrueNAS CORE or any iocage jails.

Previously I had just compiled @trombik ports and added them to the plugin's artifact repo.... Now these toolchains need updated again, and I think the correct way forward is to make the plugin, if possible, download the packages already built from @trombik cirrus task mentioned above.

Is this even possible? I don't even know where to start and we are entering a busy months at work. Unfortunately, I don't have enough free time to invest trying to figure this out by myself. Any help or even links to what I should be reading would be highly appreciated.

@tprelog recently, I noticed that marcos GitHub Actions runner supports virtualization (I'm guessing they are not VM, but physical machines). that means it's possible to run a FreeBSD virtual machine on it, to install build system and the ports tree, to create a complete set of packages, and to publish them on GitHub release page as a zip file. my concern is, the maximum build time is limited to six hours. I'm not sure all the builds finish in six hours on VM.

I'll give the idea a try. in the meantime, have a look at GitHub Actions documentation, especially sections about artifacts.

cirrus CI is great but there are limitations, and my build script is more complicated than I hoped.

@tprelog here is my initial implementation:
https://github.com/trombik/iocage-esp-idf-toolchains

please follow the instructions in README.md. you need to create the following key, see Signing key.

2564a06764db03728eca2a0d511fe02b917bf36263a624b54d1db6ed0dd23d2e

the total build process takes 11 hours on my T480. however, you don't have to build everything. when the first build task finishes (takes half an hour), then you may terminate the build by CTRL+C. the rest of the build process is known to work.

with trombik/iocage-esp-idf-toolchains@64bd54a you should be able to build all packages necessary for esp-idf on both FreeBSD 13.0 amd64 and FreeBSD 12.3 amd64.

Thank you @trombik! There is certantately a lot I need to read / learn about.

For now (due to lack of time on my part) I've just compiled the toolchains again using portshaker. To make this a little easier, I just have another TrueNAS plugin that installs portshaker and builds only the toolchain versions specifically for the ESPHome plugin. I'm installing all the requirements from the package repo so it is only the two toolchains that are being compiled. Since I'm only building one version of each toolchain used by the ESPHome plugin, total build time is under two hours for both toolchains


I'm really in over my head here -- Right now it's toolchain-xtensa causing grief.

I compiled and installed esp-quick-toolchain-gcc48-3.0.5 but it still complains about an incorrect version

Tool Manager: Installing platformio/toolchain-xtensa @ ~2.100300.0
Error: Could not find the package with 'platformio/toolchain-xtensa @ ~2.100300.0' requirements for your system 'freebsd_amd64'

I also noticed you dropped esp-idf v3 from xtensa-esp32-elf because it's EOL - I tried V4 several months ago and it did not work with ESPHome, hopefully they update it soon. For now, I already have the correct version compiled.

I compiled and installed esp-quick-toolchain-gcc48-3.0.5 but it still complains about an incorrect version

gcc48 FLAVOR is for arduino 2.x. the project you are compiling is probably using arduino 3.x. install gcc103 FLAVOR.

esphome had major updates in esp32 around Oct 2021. there were bugs in its components, but as far as i know, the build process works fine now.

well, the latest esphome updated the recommended version (the default version) of arduino/esp8266 from 2.x to 3.x. some components work fine with 3.x, but others don't.

by the way, the ports officially supports overlay. with poudriere-devel, portshaker is not necessary.

The ESPHome plugin has been removed from TrueNAS CORE -- This issue can be closed