0.4.0 questions
pellepl opened this issue · 37 comments
This issue is for asking on the upcoming 0.4.0 version.
0.4.0 is under construction. This is a full rewrite and will change the underlying structure. Hence, it will not be compatible with earlier versions of the filesystem. The API is the same, with minor modifications. Some config flags will be removed (as they are mandatory in 0.4.0) and some features might fall away until 0.4.1.
@igrr: Yes, the config options regarding file system structure are decided runtime. There is no need to typedef sizes for page indices and such in 0.4.0, this is determined compile time by spiffs itself (depending on file system size, block size, and page size). There will be compile time switches, but the goal is to keep these to functionality only, without affecting file system structure. The exceptions are file name size and optional meta data length - these must be compile time limited.
The images will be self-describing as a part of this also. The old SPIFFS_MAGIC* config option disappears in 0.4.0 and corresponding functionality is always on.
Will there be an option/tool to convert existing SPIFFSs to the new version?
If not, is the old 0.3.x branch still receiving updates for devices that already have SPIFFS deployed on them?
If requested
ok, let me put in a request then :)
we would like to stay on latest SPIFFS, but we have older builds of user firmware to consider during OTA. luckily, our OTA does not require the migration to be performed in place, so being able to mount old FS read-only to copy the data over to the new FS would be sufficient. i'm even fine with shipping 0.3.x and 0.4.x code together to use the right one as appropriate, provided there is a way to determine which one should be used.
Does SPIFFS 0.4.x support asynchronous operation?
With asynchronous I mean that you can e.g. ask for some data to be written and get a callback when the data is actually written.
This could be done e.g. by requiring a asynchronous Flash Driver...
so being able to mount old FS read-only to copy the data over to the new FS would be sufficient.
That's good. Still, you mean on target right?
yes, on target. OTA will come with new spiffs code and new fs image. new code boots and mounts the new fs, then it needs to mount the old FS (in r/o mode) and copy over some files.
yes, we reserve 2x the space for code and fs, to make sure we can revert in case of failed OTA.
Do you have anything in mind allowing for "background garbage collection" in a way that the erase (especially) as well as the preparation of erased pages is not done synchronously?
Using SPIFFS 0v3 we found out that GC takes place in the middle of some write process. As a result the write has excessive delays. Even calling the GC during the "idle phase" didn't help there for some weird reason. So we came to the conclusion it might be beneficial if one could kick GC early and let it run in background rather than waiting until we desperately need some memory. Of course this could introduce other issues like Flash is busy while one desperately wants to flush a page, but maybe it's worth some thinking about it.
This is maybe pushing it, but journaling?
@uniederer You can call in the garbage collector early by using SPIFFS_gc. Does this not work?
Directory support?
I'm missing an API function to truncate a file to a specific size without having to create a copy first.
@FrederikTB As i wrote in my original posting, calling the GC early (i.e. during the idle phase of the system) won't prevent SPIFFS from kicking the GC again upon the next write. The second Problem is the blocking nature of the GC run blocking everything in the System.
Is there an approximate release date for 0.4.0?
Additionally I think a common use case is appending files (for writing logs or CSV files ...), so this should be as fast as possible. Currently I have big issues with that.
Indeed some way to simplify logging would be great, such as functions to truncate a file and for fast apppending.
Currently SPIFFS_check() time appears to be quadratic on filesystem size. This makes it pretty slow on larger filesystems.
I guess this is not likely to change? My current plan is to make multiple small filesystems to get around that.
Very much looking forward to 0.4.0!
can V0.4.0 use less MACRO DEFINE? u can use functions instead of MACRO DEFINE, functions is more understandable, and save MCU flash.
Is there an approximate timeline for when this 0.4.0 would be released?
Looking forward to power failure safe mode.
Is this still coming ?
Sorry, I cannot say if nor when. Due to personal reasons all spiffs dev is on ice.
Sorry, I cannot say if nor when. Due to personal reasons all spiffs dev is on ice.
Don't you worry. We, plebs, will manage ! Thank you for your great work ! Get better !
Hey dude
are you still working on it?
You have done a great job,and we hope you can carry on~
Greetings from China.
Hi.
Currently the only way to copy a file from a PC to the SPIFF is via the Python utility, or to enable a full web server. This can cause issues when you want to update a file on the SPIFF from a PC, where the only connection is serial.
Could there be a function or protocol where you could open a file on the SPIFF and stream from the serial directly in to the file and then close it.
Many thanks
Hi.
Currently the only way to copy a file from a PC to the SPIFF is via the Python utility, or to enable a full web server. This can cause issues when you want to update a file on the SPIFF from a PC, where the only connection is serial.
Could there be a function or protocol where you could open a file on the SPIFF and stream from the serial directly in to the file and then close it.
Many thanks
Hi marcwolf1960,
of course there could be such a protocol. This would be very application specific and does not necessarily have to be part of a library. I implemented a protocol over UART for my application and it would not be much work for you to do the same. My protocol was a quick shoot. It does not need to be very stable and handle every possible error. It works as follows:
- Host sends a 16 byte GUID (for synchronization)
- Device sends 0x00 as ack
- Host sends 1 byte length of the file name
- Device sends 0x00 as ack
- Host sends the filename
- Device opens or creates the file and sends 0x00 as ack
- Host sends 4 bytes file size
- Device sends 0x00 as ack
- Host sends chunks of 1024 bytes (or remaining data if less than 1024 bytes)
- Device writes the data to the file and sends 0x00 as ack (repeat previous step if more data is to be received)
- when all bytes are received the device closes the file and waits for the next GUID
@hzm476172557 thanks for the kind words. No, not working on it now. My spare time is too sparse for this nowadays. Not saying I'll never return to it though.
You have done great service for the community, hope your are doing well and hear from you soon
Hi,
Any news about V0.4.0 ?
I'm asking because I'd like to translate your excellent work to another language (Ada). The translation process is quite hard and long. If V0.4.0 is ready, I'll switch to it directly.
Regards,
Nicolas
Hi,
OK. So, I'll go with the current version.
Thanks for your quick answer.
Regards,
Nicolas
Sorry but spiffs is dormant since years ago. Still have no plans of picking it up.
Perhaps you can update the README.md to reflect this? Particularly the "NOTICE" section, which says,
0.4.0 is under construction. ...