Download of LittleFS: Assertion in littlefs/lfs.c failing

Question

Download of LittleFS: Assertion in littlefs/lfs.c failing

Miq1 opened this issue 3 years ago · 15 comments

What I did:

put a file in the data folder
erased flash
built file system
uploaded file system
uploaded sketch (that should modify the file system by writing files)
tried to download the filesystem.

That resulted in

 *  Executing task: C:\Users\Micha\.platformio\penv\Scripts\platformio.exe run --target downloadfs --environment nodemcuv2 

Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino)
-------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 (4.0.1) > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif8266 @ 3.30002.0 (3.0.2)
 - tool-esptool @ 1.413.0 (4.13)
 - tool-esptoolpy @ 1.30000.201119 (3.0.0)
 - toolchain-xtensa @ 2.100300.210717 (10.3.0)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 52 compatible libraries
Scanning dependencies...
Dependency Graph
|-- DHT sensor library for ESPx @ 1.18.0
|-- eModbus @ 1.6.0+sha.5c2d656
|   |-- ESPAsyncTCP @ 1.2.2
|-- ESP8266WiFi @ 1.0
|-- ESPAsyncTCP @ 1.2.2
|-- ESP8266mDNS @ 1.2
|   |-- ESP8266WiFi @ 1.0
|-- ArduinoOTA @ 1.0
|   |-- ESP8266WiFi @ 1.0
|   |-- ESP8266mDNS @ 1.2
|   |   |-- ESP8266WiFi @ 1.0
|-- LittleFS @ 0.1.0
|-- ESP8266WebServer @ 1.0
|   |-- ESP8266WiFi @ 1.0
Building in release mode
Auto-detected: COM3
command_download_fs(["downloadfs"], [])
Entrypoint
Retrieving filesystem info for ESP8266.
FS_START: 0x300000
FS_END: 0x3fa000
FS_PAGE: 0x100
FS_BLOCK: 0x2000
Recognized LittleFS filesystem.
Parsed FS info: FS type FSType.LITTLEFS Start 0x300000 Len 1024000 Page size 256 Block size 8192 Tool: C:\Users\Micha\.platformio\packages\tool-mklittlefs\mklittlefs
Executing flash download command.
"C:\Users\Micha\.platformio\penv\Scripts\python.exe" "C:\Users\Micha\.platformio\packages\tool-esptoolpy@1.30000.201119\esptool.py" --chip esp8266 --port "COM3" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "C:\Users\Micha\Documents\PlatformIO\Projects\Dewair\downloaded_fs_0x300000_0xfa000.bin"
esptool.py v3.0
Serial port COM3
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 48:3f:da:7e:48:34
Uploading stub...
Running stub...
Stub running...
1024000 (100 %)
1024000 (100 %)
Read 1024000 bytes at 0x300000 in 92.5 seconds (88.5 kbit/s)...
Hard resetting via RTS pin...
Downloaded filesystem binary.
Download was okay: True. File at: C:\Users\Micha\Documents\PlatformIO\Projects\Dewair\downloaded_fs_0x300000_0xfa000.bin
Executing extraction command: "C:\Users\Micha\.platformio\packages\tool-mklittlefs\mklittlefs" -b 8192 -p 256 --unpack "unpacked_fs" "C:\Users\Micha\Documents\PlatformIO\Projects\Dewair\downloaded_fs_0x300000_0xfa000.bin"
Assertion failed!

Program: C:\Users\Micha\.platformio\packages\tool-mklittlefs\mklittlefs.exe
File: littlefs/lfs.c, Line 84

Expression: block < lfs->cfg->block_count
Unpacked filesystem.
Extracted 0 file(s) from filesystem.

My platformio.ini looks like:

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
board_build.filesystem = littlefs
monitor_speed=115200
monitor_filters = esp8266_exception_decoder, default
lib_deps =
    beegee-tokyo/DHT sensor library for ESPx @ ^1.18
    eModbus = https://github.com/eModbus/eModbus
    ESP8266WiFi
    ESPAsyncTCP
    ESP8266mDNS
    ArduinoOTA
    LittleFS
    ESP8266WebServer
build_flags =
    -DLOG_LEVEL=6
extra_scripts = download_fs.py

The upload looked good to me:

 *  Executing task: C:\Users\Micha\.platformio\penv\Scripts\platformio.exe run --target uploadfs --environment nodemcuv2 

Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino)
-------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 (4.0.1) > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif8266 @ 3.30002.0 (3.0.2)
 - tool-esptool @ 1.413.0 (4.13)
 - tool-esptoolpy @ 1.30000.201119 (3.0.0)
 - tool-mklittlefs @ 1.203.210628 (2.3)
 - tool-mkspiffs @ 1.200.0 (2.0)
 - toolchain-xtensa @ 2.100300.210717 (10.3.0)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 52 compatible libraries
Scanning dependencies...
Dependency Graph
|-- DHT sensor library for ESPx @ 1.18.0
|-- eModbus @ 1.6.0+sha.5c2d656
|   |-- ESPAsyncTCP @ 1.2.2
|-- ESP8266WiFi @ 1.0
|-- ESPAsyncTCP @ 1.2.2
|-- ESP8266mDNS @ 1.2
|   |-- ESP8266WiFi @ 1.0
|-- ArduinoOTA @ 1.0
|   |-- ESP8266WiFi @ 1.0
|   |-- ESP8266mDNS @ 1.2
|   |   |-- ESP8266WiFi @ 1.0
|-- LittleFS @ 0.1.0
|-- ESP8266WebServer @ 1.0
|   |-- ESP8266WiFi @ 1.0
Building in release mode
Auto-detected: COM3
Building file system image from 'data' directory to .pio\build\nodemcuv2\littlefs.bin
/config.html
Looking for upload port...
Using manually specified: COM3
Uploading .pio\build\nodemcuv2\littlefs.bin
esptool.py v3.0
Serial port COM3
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 48:3f:da:7e:48:34
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Compressed 1024000 bytes to 4952...
Writing at 0x00300000... (100 %)
Wrote 1024000 bytes (4952 compressed) at 0x00300000 in 0.5 seconds (effective 18166.2 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

Answer 1 · 2022-10-14T16:10:04.000Z

Block numbers and pagesize arguments look good, as well as the length. Address looks fine too, it reads from the same address that stuff was written to. Can you upload the C:\Users\Micha\Documents\PlatformIO\Projects\Dewair\downloaded_fs_0x300000_0xfa000.bin file for inspection?

Answer 2 · 2022-10-14T16:27:04.000Z

downloaded_fs_0x300000_0xfa000.zip
Here we go.

Answer 3 · 2022-10-14T17:09:19.000Z

Well that's weird, given that file, I can immediately list its contents..

"C:\Users\Max\.platformio\packages\tool-mklittlefs\mklittlefs.exe" -b 8192 -p 256  -l downloaded_fs_0x300000_0xfa000.bin
19270   /config.html    Thu Oct 13 08:44:23 2022
965     /set.js Thu Jan 01 00:00:00 1970
256     /settings.bin   Thu Jan 01 00:00:00 1970
Creation time:  Fri Oct 14 15:39:26 2022

But unpacking indeed fails with the same error message as yours. I'll try and understand why..

Answer 4 · 2022-10-14T17:20:29.000Z

If that is a hint: I looked into the downloaded .bin file with a hex editor and found a file that supposedly should have been altered in the meantime being still at the state of the first write to it, despite the fact I did a fresh download after deleting the previous.

Answer 5 · 2022-10-14T17:21:51.000Z

That may be natural for LittleFS, not doing an erase unless absolutely necessary to overwrite data. I'll keep it in mind though.

Answer 6 · 2022-10-14T17:44:41.000Z

Wow, I just downloaded the latest source code for mklittlefs from https://github.com/earlephilhower/mklittlefs, compiled in debug mode, ready to step-by-step debug the point where it fails, and what do you know, that latest version can unpack the binary just fine?!

Directory ./unpack_dir/ does not exists. Try to create it.
config.html      > ./unpack_dir/config.html     size: 19270 Bytes
set.js   > ./unpack_dir/set.js  size: 965 Bytes
settings.bin     > ./unpack_dir/settings.bin    size: 256 Bytes

unpack_dir.zip

I need to check what the difference between this and the PIO packaged version is..

My compiled version now is
mklittlefs.zip

Answer 7 · 2022-10-14T18:04:11.000Z

I seem to have stirred up something?

If that matters, the settings.bin file is written binary, so it may contain any byte value - could that cause problems?

Answer 8 · 2022-10-14T18:06:14.000Z

No that shouldn't matter at all. If you save binary files, then those must be preseverd exactly as is. Same for the html and js files. As referenced in the issue, although I can "extract" the files now, they don't match what I see in the littlefs .bin image.

>type unpack_dir\*

unpack_dir\config.html


���3�������3����3�������h��3��Yd������3����3�����3��e������3�
�!���3�60305�c�"r���3��������
���3�60305ML������3�x��3���$���������{;����������3�p��3�{;���3����3�P1;������3���3�PA��������������3�3��2��0��3�0����3�@\Registry\Machine\System\CurrentControlSet\Control\Nls\Sorting\Versions����T��0��3�Q��3������Q��3�0��3�l��3�h��3������T�� �����Wh����2����3�P��3����3������p��3���3������0��3���H��3������3���������3��B�������3�@F�T��000603xx�2�;����,��3�Q�������3����T���y���0F�T���1d�"r���3����2����3����T����Wh�������T��Ad�"r���3�����T�����3����3���������3����3�,��3�A�������3��������3�����T�� ����h��3�����3���������3�*����Z�"r���3����2�@��3��(
unpack_dir\set.js


��T������$�T����$�T��3�P��T���f�T��
����$�T�����    �� ��T���P��T�����)�T��P��T��H��T�������0��������T��#���B�T���B�T������������T���'�T��k��������T�����b������������T��@Y����T�����T�������0�
�}C����3���`p�C��%C������T��������3��0���������@�  ��@яC�p<���|C��b��������Zt�C��
unpack_dir\settings.bin


@�@яC�p<��3�
�|C��b��������Zt�C��
��3�@�����
�}C����3�0�C�

The first two cannot be correct when I clearly see human-readable HTML code in the .bin file.

Answer 9 · 2022-10-14T18:25:10.000Z

At least I am able to re-read the binary file obviously. I will try to open and read the set.js file as well to make sure it is nothing caused by my application.

Answer 10 · 2023-09-23T20:31:24.000Z

Use newer mklittlefs (not in pio Registry). The issue was that the needed setting -s was not given in the script. Newer mklittlefs does not need -s option anymore for extracting files.

Answer 11 · 2023-10-04T18:14:21.000Z

Uploaded a updated mklittlefs for esp32x MCUs. Extracting files does now work for me.
Before never got this going.
https://registry.platformio.org/tools/tasmota/tool-mklittlefs

Answer 12 · 2023-10-05T09:43:18.000Z

Thanks for the hints, I will retest this. I think we also talked about this in earlephilhower/mklittlefs#31. If it's just -s that's missing, that's a small change to make it work properly.

Answer 13 · 2023-10-05T10:41:31.000Z

@maxgerhardt in my tests -s was not enough. Esp32 littlefs is build in IDF with LFS_NAME_MAX 64and the earlephilower mklittlefs is built for esp8266 with LFS_NAME_MAX 32.
Thats why i built a extra mklittlefs for ESP32x with LFS_NAME_MAX 64 -> https://github.com/Jason2866/mklittlefs_esp32

Answer 14 · 2023-11-18T21:28:44.000Z

Due to the ancient version of "tool-mklittlefs"/mklittlefs in platformio (dating back to Feb 11, 2021, auto FS size retrieval not included yet), I ran into the same issue (littlefs/lfs.c:84: lfs_bd_read: Assertion `block < lfs->cfg->block_count' failed).

For me (currently working on esp8266) it was enough to pimp download_fs.py#L61 with an additional "-s", str(self.length), argument to hand over the size known to the download script to mklittlefs for unpacking images different from the default length correctly.

Answer 15 · 2024-01-21T15:11:35.000Z

For esp8266 it is enough to add "-s" as mentioned from @Henning-Lenz
I have refactored the script for Tasmota (only using LittleFS) and with the changed mklittlefs for esp32 it is working now for esp8266 and esp32 (core2 and core3 alpha).
https://github.com/arendst/Tasmota/blob/development/pio-tools/custom_target.py