[FEATURE REQUEST]ESP3DLib 3.0
luc-github opened this issue · 21 comments
Here features that will be covered by ESP3DLib 3.0 :
- Webui
2.13.0 support - Move to SSDP asyncudp library
- Move ESP32 core to 2.0.2 / 2.0.3
- Move from SPIFFS to LittleFS
- AP mode setup reviewed
- New embedded page
- Add Telnet server support
- Add Notifications support
- Add MKS Cura pluggin support for file upload
- Move SD support from Marlin SD Lib to SDFat 2.0 for better performance (TBC),
Note: Marlin itself will still use Marlin SD Lib as only ESP3DLib will be affected.
Testing boards:
1 - MRR_ESPA V1.3 without display
2 - FYSETC V1.0 + MKS TFT32_L V4.0 (With BTT FW)
3 - MKS TinyBee V1.0 + MKS Mini12864 V3.0
4 - Panda Zhu V1.2
Marlin Code base should be 2.0.9.3 bugfix 2.0.9.4? (TBD)
Some update:
1 - Use ESP3D 3.0 as code base is almost done - what is not yet implemented is the SD upload
2 - Use of SSDP asyncudp library is implemented but I got some random (refering to esp_timer) crash and also some Watchdog reset on ESP3DLib task , even without the SSDP library so I need to dig in
using disableCore0WDT(); in Esp3DLib seems mitigating the issue for the moment but need to investigate
3 - Moving Marlin to latest ESP32 core git version is done (2.0.9.3+) - I did some change to make it support it in Marlin sources
4 - Moving from SPIFFS to LittleFS is also implemented
5 - Telnet is implemented
6 - Notifications are implemented as part of ESP3D 3.0 code but not yet tested
I have currently only tested with MRR_ESPA V1.3 without display - I still need to adjust code with display and with serial display
WebUI is finally the 3.0 as even it not yet finished it will be better to focus on latest versions
since use of latest Marlin git seems supporting long name I will also spend time to add it
#define LONG_FILENAME_HOST_SUPPORT // Get the long filename of a file/folder with 'M33 <dosname>' and list long filenames with 'M20 L'
#define LONG_FILENAME_WRITE_SUPPORT // Create / delete files with long filenames via M28, M30, and Binary Transfer Protocol
That said, I need to see how to handle Marlin with Long name support and Marlin without long name support in WebUI 3.0 refactoring
So it should save time to stay on WebUI 3.0 and ESP3DLib may directly jump to 3.0 as final version
Need to spend some time on stability issues before move to SD support then WebUI Update
Here the base for the display message output
Some screenshots
I spent all week on random crash and I think I worked around it, I still need to test AP mode in long run but I do not have anymore issue in STA mode so far (TBC)
For some reason having webserver running on task on Core0 is randomly triggering WDT - I have disabled the watchdog for the ESP3DLib task but it still randomly trigger WDT on core 0, one solution seems to disabling watchdog when handling webserver
disableCore0WDT();
_webserver->handleClient();
enableCore0WDT();
So I will use it as workaround from now, and continue porting
next step is Display output : direct and serial, then SD support, once done I will go back to WebUI rewrite to implement missing panels.
so far so good ^_^
Ok final workaround is now validated instead of using task wdt feeder I use global wdt feeder and it seems ok after one week of long run test
Display output is now properly managed
Now working on SD Support - I wish I can use the SDFat instead of Marlin SDLib which is very slow for upload - it seems Marlin support now long name in command but no more subdirectory files for print command: luc-github/ESP3D-WEBUI#221 (comment)
Seems use current ESP3D V3 code using SDFat V2 fit with existing Marlin SDLib
>[ESP740]
Directory on SD : /
<DIR> overlays
<DIR> System Volume Information
<DIR> Archive
<DIR> New folder
bcm2708-rpi-b-plus.dtb 13.94 KB
COPYING.linux 18.25 KB
LICENCE.broadcom 1.46 KB
bcm2709-rpi-2-b.dtb 15.00 KB
bcm2708-rpi-b.dtb 13.68 KB
bcm2708-rpi-cm.dtb 13.64 KB
issue.txt 145 B
CUBE.GCO 205.31 KB
CUBE01.GCO 1.99 MB
SUPPORT2.GCO 4.40 MB
bootcode.bin 17.51 KB
cmdline.txt 136 B
config.txt 1.62 KB
fixup.dat 6.47 KB
fixup_cd.dat 2.48 KB
fixup_db.dat 9.52 KB
kernel.img 3.94 MB
kernel7.img 4.04 MB
fixup_x.dat 9.52 KB
start_cd.elf 619.66 KB
start_x.elf 3.73 MB
LICENSE.oracle 18.53 KB
bcm2710-rpi-3-b.dtb 15.62 KB
bcm2710-rpi-cm3.dtb 14.99 KB
24 file(s), 4 dir(s)
Total 62.95 MB, Used 24.63 MB, Available: 38.31 MB
ok
>M20
Begin file list
CUBE.GCO 210240
CUBE01.GCO 2089832
SUPPORT2.GCO 4613256
ARCHIVE/CUBE-C~1.GCO 210240
NEWFOL~1/SUPPORT2.GCO 4613256
End file list
ok
So now need to see how to avoid conflict in usage and allow only to access SD at once - but so far so good
EDIT: current test use SdFat 2.0.6 and now I would like to bump to SdFat 2.1.2 =>need to check upgrade first
Upgrade to SDFat 2.1.2 is now done and tested UTF file name also:
when it is good in ESP3DLib - even with latest bugfix and enabling
#define UTF_FILENAME_SUPPORT
the output seems incorrect
>[ESP740]
Directory on SD : /
<DIR> overlays
<DIR> System Volume Information
<DIR> Archive
<DIR> New folder
bcm2708-rpi-b-plus.dtb 13.94 KB
COPYING.linux 18.25 KB
LICENCE.broadcom 1.46 KB
bcm2709-rpi-2-b.dtb 15.00 KB
bcm2708-rpi-b.dtb 13.68 KB
bcm2708-rpi-cm.dtb 13.64 KB
issue.txt 145 B
CUBE.GCO 205.31 KB
CUBE01.GCO 1.99 MB
SUPPORT2.GCO 4.40 MB
LEBOSSE.pdf 0 B
bootcode.bin 17.51 KB
cmdline.txt 136 B
config.txt 1.62 KB
fixup.dat 6.47 KB
fixup_cd.dat 2.48 KB
fixup_db.dat 9.52 KB
kernel.img 3.94 MB
kernel7.img 4.04 MB
fixup_x.dat 9.52 KB
VGMIL-6001.jpg 1.69 MB
start_cd.elf 619.66 KB
start_x.elf 3.73 MB
LICENSE.oracle 18.53 KB
bcm2710-rpi-3-b.dtb 15.62 KB
Transfert.pdf 57.88 KB
bcm2710-rpi-cm3.dtb 14.99 KB
610203146-ddt.pdf 7.30 MB
盧克test.GCO 4.40 MB
6102031462-ddt.pdf 7.30 MB
30 file(s), 4 dir(s)
Total 62.95 MB, Used 49.46 MB, Available: 13.48 MB
ok
>M20 L
Begin file list
CUBE.GCO 210240 CUBE.GCO
CUBE01.GCO 2089832 CUBE01.GCO
SUPPORT2.GCO 4613256 SUPPORT2.GCO
ARCHIVE/CUBE-C~1.GCO 210240 Archive/CUBE - Copy.G
NEWFOL~1/SUPPORT2.GCO 4613256 New folder/SUPPORT2.GCO
TEST~1.GCO 4613256 __test.GCO
End file list
ok
still need to test print command but seems not able to print sub folder file... TBC
WebUI is working as expected :
still need to handle possible sd access conflict in ESP3DLib
I am planning to do a SD release when ESP3DLib access SD and SD Mount when operation is done -> it will mimic the share SD behaviour => To be tested
The sharing SD code have been updated as well as the new format of ESPXXX output as explained here : luc-github/ESP3D-WEBUI#157 (comment)
Still have FTP/WebDav SD access is not updated - I want to do the webui update first with updated output support and see how to handle the UTF / long name support in webUI as it seems it is still incomplete in Marlin : luc-github/ESP3D-WEBUI#221 (comment)
Ok here we are - Webui V3 is now usable for Marlin-embedded but will still be updated : https://github.com/luc-github/ESP3D-WEBUI/blob/3.0-rewrite/dist/Printer3D/Marlin-embedded/index.html.gz?raw=true
to use ESP3DLib V2 with latest Marlin bug-fix and latest ESP32 core (2.0.3rc) you need currently to use this fork / branch : https://github.com/luc-github/Marlin/tree/ESP3DLibV2
here the board and status that I will updated during my tests
| Board | Status | notes | notes 2 |
|---|---|---|---|
| MRR_ESPA |  |
No display | No IO expander |
| MRR_ESPE | |
No board on hand but compilation Ok | With IO expander |
| FYSECT_E4 | |
with MKS TFT32_L V4.0 serial TFT | No IO expander |
| PANDA Zhu V1.2 | |
No need UART patch | With IO expander |
| TinyBee V1 | |
with MKS Mini12864 V3 | With IO expander |
I have already starting to test and found some cases that I could not test on simulator 😉 so will update WebUI / ESP3DLib in parallele
I do not have time to write doc for the configuration / feature list at this moment if you want to help on it, ping me, so you can do it on Wiki
Issue with TinyBee MKS screen is now solved - issue was related to ESP3DLib code - so far so good
the webUI is also under test with ESP3D V3 luc-github/ESP3D#741
So I prefer to continue tests as ESP3DLib V2 and ESP3D V3 share same code
Finally WebDav is working as expected ^_^
I still have to review FTP server code in same way and should be ok
FTP server code is now updated - the number of connections must be limited to 1 as it does not support multiple request
Hi Luc. I have tried ESP3dLib v3.0 on my custom Board. But file gcode and the folder in which there is a gcode file not showing but if I check directly on the printer. gcode files and folders containing gcode files exist.
and controller fan for cooling main board when stepper or heater on can't function.
when i used esp3dlib v1 with marlin 2.0.9 not happen problem.
@agoy507 please this is status thread, not issue ticket do not hijack it, I have moved it to another ticket
@agoy507 please this is status thread, not issue ticket do not hijack it, I have moved it to another ticket
I'm sorry luc. I don't know. But i'm newbie on github
Because there was no realease for a while from Marlin, here the branch of latest bugfix-2.1.x with several fixes: https://github.com/luc-github/Marlin/tree/ESP3DLib-V3-bugfix-2.1.x that support latest ESP3DLib 3.0, FYI Marlin upstream is broken for ESP3DLib 1.0 #75
Current change log for reference:
-
Update PIO to 6.1.0 in esp32.ini ( esp32-arduino core v2.0.7)
-
features.ini
- Add ESP3D_WIFISUPPORT better discriminate needed libraries
- Update NEOPIXEL_LED library to 1.12.0 to support new esp32-arduino core
-
Update plateformio.ini
- Update upload baud rate to 115200 as default (suggested per vivian-ng for MRR_ESPA and MRR_ESPE)
- Update monitor speed to 115200 as default
- Add flash si8ze for each esp32 flasy that is not 4MB
-
Update ESP32 HAL
- fastio to handle extended pins
- Update ADC channels naming to match new esp32-arduino core
- Update config.auto_reload to match new esp32-arduino core
- Fix TMC2209 error with new esp32-arduino core
- Add SERIAL_PORT_3 feature support to ESP32
- Add a new define
CUSTOM_SD_ACCESSto allow concurrent SD access by external library - Update patch of u8g_esp32_spi.cpp to be more generic and no linked to ESP3D_WIFISUPPORT but to CUSTOM_SD_ACCESS
-
Fix Configuration_adv.h to restore broken support of ESP3D_WIFISUPPORT
-
Update cardreader.cpp to support a new define
CUSTOM_SD_ACCESSto allow concurrent SD access by external library -
Add
#define OLD_ADAPTIVE_MULTISTEPPINGto pins_MKS_TINYBEE.h to fix deadloop at start -
Fix SanitiCheck.h to restore broken support of ESP3D_WIFISUPPORT
-
Update pins_FYSETC_E4.h and MarlinCore.h to support
#define SKIP_TMC_SERIAL_BEGINto fix Serial initialization error if using TMC2209 in uart mode because of multiple Serial initialization that is already done in HAL -
Update .gitignore to ignore serial logs files
In same spirit of ESP3D-TFT I have listed all tested boards : https://esp3d.io/esp3dlib/v3.x/hardware/index.html
I did same for 1.0 https://esp3d.io/esp3dlib/v1.x/hardware/index.html
Feel free to let me know if need more data or I missed some boards
Hey @luc-github, first of all thanks for all the work in this project, it's looking great!
I'd like to help a bit with this, is there any blocker that would prevent having the changes mentioned a couple of comments ago in Marlin upstream? If not, is it okay with you if I start sending some of the changes there as PRs? I'm thinking of making small, isolated PRs so there's a better chance to have them merged.
Kind regards :)
Fermín.
I have to sync with latest ESP3D Code so that is first blocker for sure, second I need to test with latest changes and of course update my fork with latest Marlin to finalize tests so still several blockers
Why so hurry to push change upstream ? as you see code is still in alpha
isolated PR is not possible, just changing pio version is a breaking change, so esp3dlib V1 won't work anymore, need also to change esp3dLib, etc... it is a full interconnected changes
@fermino did you already tested my fork with the changes ?
What part of the PR do you need ?
It has been a long time I did not sync Marlin and ESP3DLIb - as ESP3D will be soon move to beta state it is time to do I think - I will work on it this week