How to execute string instead of word?
Closed this issue · 23 comments
I have cforth embedded into my app now. I also written makefiles for my app which can regenerate the rodict. And I have successfully added a word to rodict.
Next I have some forth text in a string which will define a new word. I don't see any API for running it. How do I do that? I suspect the answer is that I have use the file system. But my embedded system is not going go to have a file system so I guess I have to build a fake one? Then put my string into the fake file system and use execute_word() to load the fake file?
Jos, I redid everything using the WIP branch. The component repository is here:
https://github.com/jonsmirl/forth_component
The component is completely self contained except for the host build of three things: makeccalls, forth, makebi. The component could be extended to build those three executables. I believe there is a way to build host tools in the ESP IDF, I just haven't figured it out yet. The code in that repo works using spiffs. Just call forth() from your main app and you will get a prompt on the UART.
The main difference in making cforth into a component is to allow the whole component to be build using the ESP IDF. That picks up support for esp32, esp32s2, esp32s3, esp32c3, esp32c6, etc... Putting cforth into a component lets the various Espressif compilers access it via 'idf.py set-target xxxxx'. All of the files in the component were copied out of cforth with minimal change. For example adjusting paths in the forth files and some API changes between ESP IDF 3.x and 4.x. This process could be scripted to keep everything in sync.
Another benefit is that extend.c can access ESP IDF header files now. I don't have a final opinion yet, but I am leaning towards extend.c containing no code other than ccalls[]. The timer code is living in both worlds, I would like to remove the "esp_timer.h" include from extend.c.
I did end up having to redo all of my changes. In the first attempt I made some error which caused the ESP to reboot whenever I started forth(). I spent two days looking for the error and could not find it. So I went back and started from the beginning being very careful to test after each change and do lots of commits.
Edit: Making cforth into a component also means that it can now access all of the APIs in my app so I can add custom words to call my own code.
Edit: maybe I can define my own custom words to load my strings of forth code? I'll look at doing it that way so that I don't have to tear up the existing file system support.
This string you want to execute - I presume that it never changes because you are planning to put in in FLASH somewhere. If so, can't you just put it in a word in app.fth like
: foo " : doit 1 . 2 . 3 . 4 . cr ;" eval ;
But why do it that way? Why not just put doit in app.fth?
If you want to be able to change it later without re-downloading, the way to go is a small partition with a LittleFS filesystem. You haven't said why that is objectionable.
I already have a key-value store my app is using (the ESP NVS store). I didn't want to burn more flash making a partition for spiffs since I am running out of flash space. For the moment I turned my OTA2 update area into spiffs, but now I can't OTA.
So there is a single OTA image for all devices. Then the Forth I want to add is unique to each device. There is a cell phone app which lets you design the screens. It will generate the Forth snippet and send it to the device.
When the device updates using OTA the key-value store is preserved. The Forth snippets of code are used to customize the device to the user's preferences. This is not simple preferences, they can have custom designed screens and the Forth is used to create the screen display. That's the whole reason I am adding Forth -- to support custom defined screens.
So can you fetch an NVS string item and pass its value to evaluate ?
Yes, NVS item is arbitrary length binary array. So anything can be stored.
I am looking into how to use evaluate to define words.
The API only allows a single word. So why does this work from the command line:
S" : MYVAR4 ( -- ) CREATE 0 , ; " EVALUATE
but when I do it in two steps, it fails.
S" : MYVAR4 ( -- ) CREATE 0 , ; "
EVALUATE
I think that S" leaves the string in the input buffer, which is the current line. When you split is across lines, that input buffer is overwritten. Forth strings have a lot of restrictions.
I'm not sure what you mean by "the API only allows a single word".
https://github.com/MitchBradley/cforth/blob/master/src/cforth/forth.c#L1326
Isn't it just searching for a single word to execute?
I'll try it with a string of words and see what it does.
: foo
bar
oof
fred
derf
;
Then use execute-word on foo
Or
: execute-nvs " startup" $nv-get evaluate ;
" execute-nvs" execute-word
Why not just define the word "app" to do what you want - some canned startup stuff then execute-nvs per above - then use execute_word("app", up); as with the various tmain.c files.
I was wanting to allow the loading of new words without restarting everything. That way the user can be on phone making changes in the editor, then push their updated screen to the device to test it. The push would write it to nvs and load it into forth. But maybe I can just restart forth on a push. I'd have to be sure I can do that without leaking memory.
Can I push a string parameter (or an integer) somehow using execute_word()?
execute_word("name-of-function"); //push the function name
execute_word("execute-nvs");
And then nv-get() uses the function name to get the function from nvs?
I tried pushing constants, but this fails:
execute_word("5", callback_up);
nv-get() doesn't have to use a string, I can load the functions by number too.
I have 128KB free at the end of my flash chip, maybe spiffs will run in it. Not sure how spiffs does directories; it may chew up too much with a FAT table. I'll give it a try and see if spiffs will boot.
Edit: forth and spiffs booted with 128KB partition. I will need to test now and see what kind of file granularity spiffs has. I'm fairly sure it won't let me make 10,000 one character files. The NVS key-value storage would let me do that. If I can make 100 small spiff files that should be plenty.
Use LittleFS not SPIFFS. SPIFFS is not stable. You do not need anywhere near 128K for a LittleFS partition. 16K would probably work.
The reason you cannot do execute_word("5"..) is because 5 is not a word, it is a literal. Words are things that have explicit entries in the Forth "symbol table". It would be impractical to have an entry for every possible number.
I think I am finally starting to see where you are coming from. I think that, instead of running Forth as an interactive interpreter that repetitively reads input from a user and executes it, you want to call into Forth, have it execute something, and return to an external "main" routine. The piece you are missing is spush(cell n, cell* up).
char * foo = "5 . 6 . 7. cr";
spush(foo, up);
spush(strlen(foo), up);
execute_word("evaluate", up);
In the NVS context, it might look like
uint8_t nvval[MAXNVVAL];
size_t nvsize;
err = nv_get_blob("mynvhandle", "forth_code", nvval, &nvsize);
// Handle errors
spush(nvval, up);
spush(nvsize, up);
execute_word("evaluate", up);
Yes, you are correct. This is an event driven UI. Touches to the touch screen drive the execution of the forth snippets. So when you touch a button to go to the next screen, it will execute the snippet that draws the next screen.. The events all have unique numbers which can index into the key-value store to load the the correct forth snippet and run it. Use nvs_load in forth to run code that associates the various button pushes with the actions you choose. So the snippet for event-131 might run action-12, action-27, action-39. The actions are coded in C and forth wires them together is various ways. Plus forth gives me the ability to do computations or loops in addition to stringing together simple actions.
After I get the basic events going the next step is to use forth plus sockets to read/write from Google's real-time db. This is not hard to do because it has an HTTPS interface. I will need TLS working which ESP has. I will drop down into C to handle complex things like the JWT security tokens.
I was far down the path using Micropython to do all of this until I noticed that Micropython has a 1.2M image footprint and I don't have that much flash to work with.
Later on I can trim forth down even further by eliminating the interactive environment, but I'm keeping it right now so that I can test my snippets using the UART.
BTW, I'm one of the early Linux device tree people along with Grant Likely.
Good to know re the device tree. I suspect that the device tree architecture is the part of my legacy that has the longest "legs".
So if you just want to do it with numbers, you would write
spush(n, up);
execute_word("do-action-number", up);and then do-action-number would pop n from the stack and construct the NVS key name and ...
( n ) " nv-event%d" sprintf nv-load
That is my demo showing that I could extend the library. Or if you want to use lvgl, add the espressif lvgl component to your main app. https://lvgl.io/
I removed it and pushed a new version without it.
I managed to get cforth to draw a button...
init_io(0, (char **)0, (cell *)callback_up); // Perform platform-specific initialization
callback_up = (void *)init_forth();
//execute_word("app", callback_up); // Call the top-level application word
char *cmd = "\
decimal \
variable screen \
lv-scr-act screen ! \
screen @ \
variable button \
lv-btn-create button ! \
25 25 button @ lv-obj-set-pos \
100 100 button @ lv-obj-set-size \
";
spush(cmd, callback_up);
spush(strlen(cmd), callback_up);
execute_word("evaluate", callback_up);
When I run the "app" word there is something in there which is breaking my LCD. I've deleted a lot of the stuff from app.fth but my LCD is still broken. Most likely something is messing with a GPIO used by the LCD. It is parallel LCD so it uses 20 GPIOs.
https://github.com/jonsmirl/forth_component/blob/main/components/forth/regen/app.fth has
: app
banner hex
interrupt? if quit then
['] load-startup-file catch drop
quit
;
The only IO that touches is the console serial port and whatever the file system uses, which is typically the same FLASH that code is executing from. Lacking information about exactly which GPIOs you are using, it is hard to say.
Jos, I removed lv_scr_act1(). I copied this component out of my much larger project. The version in my project has a lot of extended calls added.