maxgerhardt/pio-esp32-esp8266-filesystem-downloader

"downloaded_fs_0x300000_0xfa000.bin": no such file or directory

leohxj opened this issue · 8 comments

I’ve tried it on my own project by copy download_fs.py to my project and adding extra_scripts = download_fs.py as described in https://github.com/maxgerhardt/pio-esp32-esp8266-filesystem-downloader#using-in-a-different-project.

I run custom -> Download Filesystem, but got below error:

Processing nodemcuv2 (platform: espressif8266; board: nodemcuv2; framework: arduino)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 (3.2.0) > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: 
 - framework-arduinoespressif8266 3.30002.0 (3.0.2) 
 - tool-esptool 1.413.0 (4.13) 
 - tool-esptoolpy 1.30000.201119 (3.0.0) 
 - toolchain-xtensa 2.100300.210717 (10.3.0)
Converting main.ino
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 35 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <ESP8266WebServer> 1.0
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266WiFi> 1.0
Building in release mode
Auto-detected: /dev/cu.usbserial-0001
command_download_fs(["downloadfs"], [])
Entrypoint
Retrieving filesystem info for ESP8266.
FS_START: 0x300000
FS_END: 0x3fa000
FS_PAGE: 0x100
FS_BLOCK: 0x2000
Recognized SPIFFS filesystem.
Parsed FS info: FS type FSType.SPIFFS Start 0x300000 Len 1024000 Page size 256 Block size 8192 Tool: /Users/leohxj/.platformio/packages/tool-mkspiffs/mkspiffs
Executing flash download command.
"/Users/leohxj/.platformio/penv/bin/python" "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin"
scons: *** [downloadfs] "/Users/leohxj/.platformio/penv/bin/python" "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin": No such file or directory
==================================================================================== [FAILED] Took 0.81 seconds ====================================================================================
The terminal process "platformio 'run', '--target', 'downloadfs', '--environment', 'nodemcuv2'" terminated with exit code: 1.

Where is downloaded_fs_0x300000_0xfa000.bin file, or how to build or generate it??

The script is supposed to trigger the generation of that file by starting esptool.py, but in the output you can see it failed.

scons: *** [downloadfs] "/Users/leohxj/.platformio/penv/bin/python" "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin": No such file or directory

I think I remember a bug report saying that quoting the executable (python) does not work like that on a Mac (but does on my Windows machine).

Can you change this line in the method of the SPIFFSInfo class

def get_extract_cmd(self, input_file, output_dir):
return f'"{self.tool}" -b {self.block_size} -p {self.page_size} --unpack "{output_dir}" "{input_file}"'

to

    def get_extract_cmd(self, input_file, output_dir):
        return f'{self.tool} -b {self.block_size} -p {self.page_size} --unpack "{output_dir}" "{input_file}"'

and retry?

yes, I am on MacOS.
I change the code below, but it also gets errors like before.

then I print log in get_extract_cmd, but it does not execute.

Can you manually open a CLI and execute the command?

/Users/leohxj/.platformio/penv/bin/python "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin"

what does it output?

@maxgerhardt
It works! and the output is:

 /Users/leohxj/.platformio/penv/bin/python "/Users/leohxj/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.usbserial-0001" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/leohxj/Documents/PlatformIO/Projects/access-point/downloaded_fs_0x300000_0xfa000.bin"
esptool.py v3.1
Serial port /dev/cu.usbserial-0001
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 98:cd:ac:26:2a:2f
Uploading stub...
Running stub...
Stub running...
1024000 (100 %)
1024000 (100 %)
Read 1024000 bytes at 0x300000 in 91.5 seconds (89.5 kbit/s)...
Hard resetting via RTS pin...

Weird that it works when the command is executed in the commandline, if it's the same command it should work.

What is the full output now of the "Download Filesystem" task?

nighi commented

I have the same issue under linux (Manjaro).
I could execute the cmds directly via shell, but I think the problem is already the download cmd, not only the extraction cmd.

Removing the quotes around the python-path doesn't help, neither in the mentioned get_extract_cmd method nor in download_fs.
But with changing the call to subprocess.call(cmd, shell=True) fixed the issue with download and extraction (lines 297 & 274). Executing via shell is considered as a security issue because it could enable shell injection, but I see no problem in this case. It could be a problem for Windows users, don't know. Maybe there is a better option escaping the cmd correctly or use the arguments of subprocess.call ?

With the script working there is another issue: The extracted files are only a part of the file system. But perhaps I should open a new Issue because they don't seem related.

I'll double-check this on Linux today to see where / why it fails, but thanks for the hint on shell = True. Sadly I won't have a Mac to test on, but maybe it equally resolves the issue with that.

Fixed per PR above