SmittyHalibut/EleksTubeHAX

Use VSCode and platformio to simplify whole build

hallard opened this issue · 32 comments

hallard commented

Thanks @SmittyHalibut for this amazing job.

I'm planning to port this one using visual studio code and platformio so end user can build easily getting rid of checking all library version in arduino IDE and changing #define in source code, just doing change in platformio.ini file

If you agree and plan to merge this new pull request I will do from here, else I will fork and make available on my github, but I prefer contribute to yours.

humm looks like this fork already done the hard job :-)
https://github.com/judge2005/EleksTubeHAX

Let me know.

Charles

aly-fly commented

Hi Charles, I have made quite a few updates to the code on my fork (https://github.com/aly-fly/EleksTubeHAX) but due to some conflicts I was not able to merge it here... In the mean time also this repo received a few updates, so it has become quite a task to merge both repositories together, before porting to Platformio (This branch is 32 commits ahead of, 19 commits behind SmittyHalibut/EleksTubeHAX:main.)
Unfortunately time is quite limited, that's why I left this on the backlog...
Otherwise I have good experience with Platformio and would like to see the ported code sooner or later.

aly-fly commented

Also, this guy already made a port to Platformio:
main...judge2005:EleksTubeHAX:main

SmittyHalibut commented

And if I'm being honest, this is abandonware for me. I haven't put any work into this for (crikey!) years now. @aly-fly is basically the owner now, despite it being in my account.

Should we officially move the repo to you now?

aly-fly commented

Should we officially move the repo to you now?

It was officially your project (+ Frank's if I remember correctly?) so lots of kudos for the first developers.
If you are in for the move, I will be glad do accept and maintain it :)

SmittyHalibut commented

@Frank-Bemelman You have an opinion here? @aly-fly is right, you put in as much effort as anyone else here.

For me personally, it was a 4-week project for me to do while I had some time off work, that I no longer have time to maintain. I'm happy to hand primary ownership off to someone else for on-going maintenance, and to my observation, @aly-fly has been most active in that.

Frank-Bemelman commented

@SmittyHalibut - well, I didn't add much. Fix one little bug with mqtt as far as I remember. Not much kudos for my contributions, really. It's like you say, at some point you find yourself fiddling with projects like this, and once it works - well, it works. I am the last person to claim any ownnership, and it gets moved to a new owner, why not. I moved it to platformio myself, which was not that complicated. But never bothered to adjust my fork since I didn't change more than a few lines of code. I better remove my fork, just to avoid confusion with all those forks.

hallard commented

Woo I was not expecting so much animation here, nice :-)

Another solution is that @SmittyHalibut give permission on this repo so that defined contributors (@Frank-Bemelman , @aly-fly) can merge or publish (same or new branch) here like that, owner stays owner but other can works and do modifications seen directly from here, and other are not confused with too much forks, may be a good deal.

whatever I can do that but can't set a timeline because looks like everyone there 24H/day is not enough :-)

SmittyHalibut commented

Done. @aly-fly you're now a contributor. I don't object to transferring ownership to you if you'd prefer. Let me know.

Also, I knew I forgot someone. @RedNax67 was the other person I was thinking of who contributed a lot early in the process. Sorry for forgetting you!

  • 👍1
SmittyHalibut commented

To address the ACTUAL ask of this Issue! :-)

I have personally moved to PlatformIO and VScode for all my development as well, so I certainly don't object to migrating this project to that platform. My only concern is the additional support burden of helping new folks program their hardware.

Though, from the looks of the other tickets we've received, I don't know if this hardware platform is even still available. As it is, there won't really be any new users anyway. At least, not without a fair bit of work to the code to abstract out the different hardware versions and make it easy to add new ones.

In any case. As far as I'm concerned, please do migrate it to PlatformIO. Arduino IDE (at least the v1.8 branch) is less than ideal. (I haven't used v2 yet, so have no opinions on it.)

frankcohen commented

Hi Smitty, +1 on anything you want to do with the project. AND, right on time, the CEO at EleksMaker today replied to my post on Reddit, inviting a response to his email. Anything you or anyone wants me to reply to them? -Frank

aly-fly commented

Done. aly-fly you're now a contributor. I don't object to transferring ownership to you if you'd prefer. Let me know.

I received and confirmed the status. Thank you. For now, I think the status is perfectly fine as is, because this repo is linked on many internet sites. Will take time in following few weeks to merge and test all changes from my branch into this one.

aly-fly commented

Arduino IDE (at least the v1.8 branch) is less than ideal. (I haven't used v2 yet, so have no opinions on it.)

I tried Arduino v2. Many projects don't compile. Also the Data Upload tool is not available. It's almost impossible to migrate from v1.8 to v2. PlatformIO will be the way to go. @hallard Did you check out that fork that was already migrated? Is it useful?

aly-fly commented

AND, right on time, the CEO at EleksMaker today replied to my post on Reddit, inviting a response to his email.

Found it: https://www.reddit.com/r/arduino/comments/mq5td9/comment/ki7r3u2/?utm_source=reddit&utm_medium=web2x&context=3

Well, I am not sure if the author is happy that their design is being hacked and also copied by Chinese guys. And then we support all sorts of hardware copies that reduce author's profit. :)
At the moment I don't have any questions for the EleksMaker...

hallard commented

@aly-fly before I found the vscode version I tried with this specific fork version https://github.com/bitrot-alpha/EleksTubeHAX/ may be time of my search I tough it was the most "up to date" and I succeeded to fix my clock ,but as library management is a mess (don't blame any authors, it's Arduino way), it's only after vscode come into my mind
Anyway only after I took the idea to migrate this one to vscode and then found other contributors already done, so no I did not try any with vscode yet. But for sure I will as soon as it is in this repo or let me know the fork to use, I can give a try.

I also plan to re org the digits, having all in a folder with both (bpm, jpg or png) AND .clk so it's easier to see in explorer or finder and have a script to select the one to put in data folder easily without having to rename file name because we all prefer different digits from each others.

frankcohen commented
  • 👍1
aly-fly commented

Alight, I took whole weekend and merged together more or less everything useful from many different forks out there :D Also tested compilation of different options. And functionality on my hardware.
There are two main tasks still open: merging manual timezones from RedNax67 and analyzing all sorts of updates from wfdudley.

Looking at Platformio migration made by judge2005 - this doesn't seem to be very difficult. We might just merge ideas from there :)
@hallard If you have experience doing this, you can proceed using existing code here, create a pull request and then we will continue development using Platformio.

p.s. I made a backup of this repository before this mass update, if anyone needs it, it's on my disk.

frankcohen commented
SmittyHalibut commented

@aly-fly said:

p.s. I made a backup of this repository before this mass update, if anyone needs it, it's on my disk.

And, ya know, git revision history. ;-)

I've just unburied my clock recently. Once you're done, I'll probably pull this all down and install it, just to see how things have changed.

aly-fly commented

And, ya know, git revision history. ;-)

I know, but better safe than sorry. I sleep better this way knowing I can manually revert anything If I made a big mistake somewhere.

aly-fly commented

I added a Web UX to control the EleksTube IPS clock. The code is here: https://github.com/frankcohen/EleksTubeIPSHack

Thanks for this, will be useful to look at. That is in the plan to add, if we will have enough space. Goal at the moment is to pack many features into 1 MB and keep 3 MB free for as many clock faces as possible loaded at the same time. It is likely we will not have hardware resources available to implement HTTPS server and certificates.

Good job on your wrist watch, wend through the project, looks impressive!

hallard commented

Nice job here, really sorry guys, I'm currently out of time with huge project to get some spare time right now. But will test ASAP.

frankcohen commented
aly-fly commented

Hi everyone, I am getting back to the original topic: Migration to PlatformIO.
I am playing with some of my other small projects (CO2 monitor + clock) to gain experience in PlatformIO.

Steps:

  1. updated all libraries in Arduino 1.8 IDE to latest version, including ESP core.
  2. migrated the project with as little modifications as required.

Results:
Compilation and running success.
BUT - code size is significantly larger! About + 280 kB, what I really don't like.

Sketch uses 1040501 bytes (99%) of program storage space. Maximum is 1048576 bytes.
Global variables use 46412 bytes (14%) of dynamic memory, leaving 281268 bytes for local variables. Maximum is 327680 bytes.

RAM:   [==        ]  16.0% (used 52424 bytes from 327680 bytes)
Flash: [======    ]  63.2% (used 1325369 bytes from 2097152 bytes)

Arduino: 1040501 bytes.
PlatformIO: 1325369 bytes.
(Ignore the % values as I had to adjust partition sizes.)

I am trying to find out default compiler settings for optimization. Does anyone has insights what are default -O settings for both Arduino and PlatformIO?

Edit: Found the build logs for Arduino. It sets -Os to optimize for size.
After lots of googling (not in the actual project) it seems that same setting is used in PlatformIO by default.
This makes me pretty unhappy that PlatformIO makes bigger build size that eats into our precocious space for clock images..

Edit 2: Tested two empty projects on both Arduino and Platformio - both yielded < 1% difference in code size. Both ~230 kB. Investigating further..

aly-fly commented

Alright, I successfully migrated the project to PlatformIO.
Using manual file copying instead of "import arduino" function. Kept libraries in the same folder (not moved into .\lib). Maybe some of this contributed to build size differences in my previous post. Now build code size is about the same as before. Yay!
Set up manual partitions as under Arduino (No OTA, 1M app, 3M SPIFFS).
Kept "_arduino" folder as a backup; can be deleted later.

Please download, test and report back!

Issue that I noticed is configuring the pins for the TFT display. You can either manually overwrite the config file in the library (as before) or define pins in platformio.ini. Currently project is set-up using the latter approach. Still investigating some other approaches.

hallard commented

@aly-fly awesome thanks.

For PIO, a idea is do define the differents boards into platformio.ini and then set pinout for each board, like that you have all paremeters in one file and can be different for each board, just need to select the board you want to use on VSCode

Here a quick and dirty example I made some time ago for LVGL boards
https://github.com/hallard/Makerfabs-LvglWidgetsDemo/blob/main/platformio.ini

  • 👍1
aly-fly commented

That is one way, yes, with different build environments.
I was looking into a solution to make a pre-build .py script that before every compilation rewrites the TFT config file User_Setup_Select.h to include the global_defines.h into the library and this takes away the probability of the user to incorrectly select the environment on the PIO startup. User config is then fixed for every user locally in the _user_config.h file.

judge2005 commented

Whichever approach is taken, I would prefer that it not modify any GIT files as this makes it hard to fork the repo and issue pull requests.

I took the platform.ini approach in https://github.com/judge2005/EleksTubeIPS as I'm building releases for others to use as well as myself. I have already accidentally flashed the wrong image!

aly-fly commented

I made a .py script that copies config files into the TFT library folder.
Could be a bit more polished (using system variables for file location) but I spent too much time failing to get it working, and finally gave up and reverted to hard coded paths. If anyone is more fluent in this, please correct it.
This way all settings are together in a sinlge file _user_defines.h and no need to switch environments.

aly-fly commented

Hi all,
Did anyone had time to test this PIO project?
Are build paths for copying files same on Linux as on Windows?
Is everything working fine and can we delete the Arduino folder?

SmittyHalibut commented

Sorry, I haven't had a chance to try it yet. Lots going on at work right now.

Martinius79 commented

Hello!

I was able to build and upload the firmware with the PlatformIO extension/IDE in VS Code using the files from the "EleksTubeHAX_pio" folder in the main branch.
Just needed to make the "normal" changes to the code (select model, rename user-defined header file, set up WiFi). I didn't have any "prerequisites" installed before, just VS Code.

VS Code detected the need for PlatformIO and C/C++ extensions and installed them.
PlatformIO then checked and downloaded the needed libraries.

Just two minor problems occurred:
The first line "Import (env)" from the little Python helper script needed to be commented out, and I needed to install Python (executable missing).

And because I have a variant of the NovolLife clock (SPECTRUM-V2.0), I need to reduce the serial port speed to 512000 baud, but this has nothing to do with the PlatformIO migration. Maybe I can commit some pictures of it over the weekend.

The firmware itself is working fine!

My setup:
Windows 11 Pro 23H2 + VS Code 1.87.0 + fresh GIT clone from repo.
Actually, Arduino 1.8.19 is also installed, so I didn't do a completely "clean build" yet on a vanilla Windows in a fresh VM, but don't think that is needed.

Great work so far @aly-fly :)

I guess the Markup files with the instructions need to be created/renewed, and a bit of "fine-tuning" is possible, like using different build environments (to preselect the settings for the different variants and versions of the clock), as @hallard mentioned in his post and showed in his platformio.ini.

I don't know if I will find the time to do any of these tasks in the near future, but I will try...

Some questions from my side are left:
What about the SPIFFS upload? I didn't check how to do it with PlatformIO yet.
Should be possible to use the esp32fs jar call variant, I guess? -> OK, found the SPIFFS building stuff :)
Which versions of the precompiled firmware should be included in the repo in the future? the Arduino build ones or the PlatformIO ones? Or both? Any naming convention for them?

Bye
Martinius

aly-fly commented

Hi Martinius, thanks for the detailed report! Glad it works fine for you.
Not sure if build environments are needed, as user must set other things too, besides the hardware type.
Will add a note in the Readme regarding baud rate and SPIFFS upload tool.
Arduino folder is already behind on the changes, I will delete it now, as you are confirming that PIO files are all good.