Updates for Responsive web UI

[fvanroie]

Is your feature request related to a problem? Please describe.
The current web UI is clunky and looks old fashioned.

Describe the solution you'd like
Implement modern technologies like CSS variables, JavaScript and responsive document queries.
Use a minimalistic approach and include the sources as a gzipped document in the firmware.
These static CSS/JS files can be cached in the web-browser, leading to smaller/faster responses for each page.

Describe alternatives you've considered
Using standard templating engines would bloat the firmware size with unnecessarily big JavaScript libraries, or we would have to link to them on the internet, which would not work offline or in a restricted network environment.

Additional context

• Add style.css and vars.css. Gzipped in firmware which can be overridden in flash.
• Add script.js. Gzipped in firmware which can be overridden in flash.
• Update forms to use simple html code, leaving the styling to the CSS.
• Use JavaScript to render JSON objects in the browser instead of serving full HTML from the MCU.
• Test web UI configuration in AP-mode.

[nagyrobi]

Some results (with dark theme from user_config_override):
On mobile:

(needs buttons wider+centered)

(GPIO settings need brushing)

(same on desktop)

[fvanroie]

This is mostly complete.

GPIO forms need some work, but I will do these when updating that module with sensors later.
AP-mode has not yet been tested...

[kquinsland]

This came from a discussion on Discord.

I am currently working on a tool that - eventually - should be able to connect to a device and get a dump of all of it's state: every detail! What GPIO(s), syslog settings, is the telnet server on... mqtt user/pass/host ... etc.

The ultimate goal is to flash a device with openHasp and configure WiFi via UART before re-assembling / deploying the device in its intended location.

Once the device has an IP address, I should be able to run a simple tool that will:

• set mqtt info
• turn off Telnet
• turn off mdns
• set the syslog endpoint
• adjust brightness / timeouts
• adjust the default start page
• configure GPIOs
• sets the default theme
• ... etc.
• upload misc jsonl and cmd files
• Save all the changes and reboot

Basically automate all the clicking I'd do normally and ensure that the declared state and the actual device state match always.

I was having a miserable time trying to get this via telnet... at least partly because the telnetlib that python ships with does not appear to easily support "interactive" usecases like mine. I pivoted to HTML as scraping it does offer a predictable payload that can be parsed to get device info and device state - including files - can be managed this way.

I am able to get most of the device information with basic regex parsing but a more machine readable format would be simpler.

so instead of having to:

wifi_cfg_raw = urllib.get(http://1.2.3.4/config/wifi)
wifi_creds = re.match(.wifi_cfg_raw, r'...' )

I could just do this

dev_cfg_doc_str = urllib.get(http://1.2.3.4/api/config.json)

dev_cfg = json.parse(dev_cfg_doc_str)

// set new wifi name
dev_cfg['wifi']['ssid'] = 'someNetwork'

# or POST or whatever
urllib.patch(http://1.2.3.4/api/config.json, dev_cfg)

Depending on exactly how the web UI is revamped, it could be possible to have HTML make PATCH calls to /config.json or similar rather than the current method:

<form method="POST" action="/config">...</form>

It may also be easier to implement an API that works a bit more like the telnet interface: rather than PATCH calls to a singe file, a series of endpoints is implemented to 'mirror' the telnet:

to get/set GPIO config:

$ telnet 1.2.3.4
<...>
Prompt > config/gpio
[15:58:34.999][65524/115604 43][42552/42872  1] MSGR: config/gpio
[15:58:35.022][65524/114740 42][42552/42872  1] CONF: {"config":[197658,263456,329249,655628,655886,656155,0,0]}
[15:58:35.048][65524/115208 43][42552/42872  1] MQTT PUB: config => {"gpio":{"config":[197658,263456,329249,655628,655886,656155,0,0]}}

Prompt > config/gpio {"gpio":{"config":[1,1,1,1,1,1,0,0]}}

So the /config/gpio page would load some bootstrap HTML/JS which would then send a GET to http://1.2.3.4/api/config/gpio and would get back a simple JSON doc like:

{"gpio":{"config":[197658,263456,329249,655628,655886,656155,0,0]}}

When the user saves the GPIO config, instead of POST the form directly to the endpoint, a small bit of JS would POST a body to http://1.2.3.4/api/config/gpio that looked like {"gpio":{"config":[1,1,1,1,1,1,0,0]}}

---

As of right now I have found that there is a http://1.2.3.4/config.json endpoint which is very helpful but it appears that there are a few issues with using that:

• could go away/move
• The only way to set a password for MQTT/HTTP/WiFi is via POST or telnet (or, once MQTT password is set, i guess it and the telnet interface behave the same so you could then set a HTTP password once MQTT connection works...)
• There is no programmatic way to restart the device w/o first overriding the content of config.json with whatever values exist in memory. This means that I can't adjust the gpio:[] array in config.json and then upload a new config.json to the device and expect it to persist after calling /reboot I need to emulate the browser and send a POST to the config/gpio endpoint to get the new values in memory and then call /reboot to get the changes saved
• Files (online.cmd, for example) still need to be uploaded via /edit endpoint... But this is not so critical as the file editor and /edit endpoint already appear to be largely independent/distinct from the rest of the web UI as it is.

in short: (almost) all device state can be fetched by fetching the http://1.2.3.4/config.json file but any change to device state must be done by emulating the web browser or telnet/mqtt. For the purposes of my tool, am 'ignoring' telnet and MQTT because:

• telnetlib was not playing nice
• I would expect that 90% of openHasp users will only ever use HTTP and HTTP is 'default enabled' w/o credentials so it's almost a guarantee that my tool can work after the device has connected to WiFI

Tl;DR: making web/telnet behave identically would be very helpful :).

[nagyrobi]

• upload misc jsonl and cmd files

all files supported should be included in this list, like png or other image formats and zip.

[kquinsland]

• upload misc jsonl and cmd files

all files supported should be included in this list, like png or other image formats and zip.

Ahh good point. I am just mimicking the upload button on the web editor so these file types will probably "just work"

[dgomes]

http://www.bx.psu.edu/~nate/pexpect/pexpect.html

[nagyrobi]

The ultimate goal is to flash a device with openHasp and configure WiFi via UART before re-assembling / deploying the device in its intended location.

Note that for this kind or deployment you can also rely on the built-in user_config_override.h functionality, which also brings the advantage of predefining your own factory settings for the plate. This way even if for any reason at the location a hardware settings reset happens (flash storage data corruption for example), the factory default settings will still be yours, even with config.json lost.

I've learnt this the hard way using Tasmota: they have implemented a kind of weird recovery to factory defaults by power-cycling the device a few times within a time window. I've ran into a power outage caused by the energy company which fell exactly into this pattern - and found myself with about 30 IoT devices built in walls all returned to factory defaults.

[kquinsland]

Note that for this kind or deployment you can also rely on the built-in user_config_override.h functionality, which also brings the advantage of predefining your own factory settings for the plate.

Absolutely! I am trying to avoid that route specifically to make the tool accessible to other people that don't have a build env set up though. The 'idea' is that I'll have a small colleciton of yaml files that I can keep in git. A change to any of the files will run this tool w/ that file. Inside the file is all the information needed to connect to the device and configure the mqtt/gpio and render out / upload any files to the device.

You can disable the brownout -> reset functionality with Tasmota: https://tasmota.github.io/docs/Commands/#setoption65

[nagyrobi]

You can disable the brownout -> reset functionality with Tasmota: https://tasmota.github.io/docs/Commands/#setoption65

Yes I know. Already done that...
I migrate everything to ESPHome as their architecture design is way more adequate for operating multiple dozens of devices. It doesn't really need to store any settings/config at runtime...

[fvanroie]

Endpoints proposed:

endpoint	GET	POST / PUT	DELETE
/api/info/	✅
/api/config/	✅
/api/config/hasp/	✅	✅
/api/config/wifi/	✅	✅
/api/config/mqtt/	✅	✅
/api/config/http/	✅	✅
/api/config/gui/	✅	✅
/api/config/debug/	✅	✅
/api/config/time/	✅	✅
/api/config/ota/	✅	✅
/api/config/gpio/	🔲	🔲	🔲

[kquinsland]

I just wanted to leave this here: https://stackoverflow.com/questions/31089221/what-is-the-difference-between-put-post-and-patch

I don't think there is a ton of value in doing a fully pedantic implementation and arguing over which verb should be used for what and if you are updating an existing resource or if the server can only have a single resource of a given type so there is no such thing as "update" only "set the resource to this state" semantics.

TL;DR: I could totally understand removing the POST and DELETE column and just using GET for "give me" and POST for "use this" operations. NO need to complicate things beyond that... even if it's not a "technically correct" implementation :/

[nielsnl68]

Hello beatagerger,

Sorry for bumping in, it is not that we dont want to help you there where a couple of issues with it in your first post that did not part of this discussion. The second thing is that your post pointed a couple off things at once, that while reading where not understandable. Also using CAPS is in most cases used for shouting and that is not wanted here.

I understand you have some problems with openHASP. But take a moment, read your first message, go to the discussion section.
Create a new topic there and explain what you are using as display and what steps you have tried and where not able to figure out.
I agree with you that the documentation could be better, but it is all done in spare time. And it helps when you explain where you get stuck.

Let's stop this overheated discussion here, it is already to warm outside.