API to get the results rather than dumping to file (IDFGH-11934)
shubhamdp opened this issue · 3 comments
If someone wants to use this utility through the script then it can't be used directly (or I may be missing something). For workaround I tried the below option
import import esp_idf_size.core
sys.argv = ['esp_idf_size', '--format', 'json', map]
esp_idf_size.core.main()
results are dumped to stdio (or to the specified file) and it its not convenient to use it directly.
Then, I used '--output', '/tmp/.tmp_profiler_stat.json', and I'm parsing the json file.
It would be good if there's an API which returns the json (or as per the format specified) output which is parsable.
Hello @shubhamdp ,
thank you for your report. We are working on a refactored version, which should allow this more easily. Basically it generates a generic memory map python dictionary and all other outputs are generated from it. You can take a look here. Currently it's implemented as ng submodule in esp_idf_size. The format should be easy to follow and basic description could be found here
IIUC you would like to use it in python script. Maybe the following could help.
#!/usr/bin/env python3
import sys
import json
from esp_idf_size.ng import memorymap
len(sys.argv) != 2 and sys.exit(f'usage: {sys.argv[0]} <map file>')
memmap = memorymap.get(sys.argv[1], True)
print(json.dumps(memmap, indent=4)) Note it's a raw output with all the info and it's not the same as the current json format generated by esp_idf_size. On the other hand it should contain much more information and also easier handling of two map files comparison(diff).
It's for further discussion, but I think we can add a version to the memory map dict and make it available as official interface. Would this work for your use case?
Thank you
Thanks @fhrbata for the detailed explanation.
My use case is very simple, I want to track static size increase between two releases and for that I would be more interested into high level values like used_iram, used_dram, used_flash. Probably, this could be added to the CI which might flag the PRs/MRs having significant increase in the values, then we may be more interested into which module caused this increase.
It's for further discussion, but I think we can add a version to the memory map dict and make it available as official interface. Would this work for your use case?
What version you want to include in the memory map dict?
PS: My question has been answered, please close or let me know if I need to close this issue if you think if this is not relevant futher.
Thanks @fhrbata for the detailed explanation.
Thank you for your feedback! It's very much appreciated.
My use case is very simple, I want to track static size increase between two releases and for that I would be more interested into high level values like
used_iram,used_dram,used_flash. Probably, this could be added to the CI which might flag the PRs/MRs having significant increase in the values, then we may be more interested into which module caused this increase.
Understood, the key aspect is that we've received several requests for various types of detailed information. Using a generic memory map approach hopefully offers a convenient method for obtaining the necessary details. For instance, the high-level overview can be obtained by
#!/usr/bin/env python3
import sys
import json
from esp_idf_size.ng import memorymap
len(sys.argv) != 2 and sys.exit(f'usage: {sys.argv[0]} <map file>')
memmap = memorymap.get(sys.argv[1], True)
for mem_type_name, mem_type_info in memmap['memory_types'].items():
print(f'{mem_type_name}: size: {mem_type_info["size"]}, used: {mem_type_info["used"]}') giving the following output for the hello_world example for esp32
IRAM: size: 16384, used: 16383
DIRAM: size: 345856, used: 53548
DRAM: size: 0, used: 0
Flash Code: size: 8388576, used: 100691
Flash Data: size: 33554400, used: 52944
RTC FAST: size: 8192, used: 40
RTC SLOW: size: 8192, used: 0It's for further discussion, but I think we can add a version to the memory map dict and make it available as official interface. Would this work for your use case?
What version you want to include in the memory map dict?
It's already presented in the esp-idf-size package and it can be used even now. It's available since 1.0.0 version in the ng subpackage. To make this really usable we should probably add some versioning and also add it to the official module API.
I guess we can also make the other json formats available through the API, but these are all generated from the generic memory map.
For your usage in CI the memorymap.diff might be also useful.
PS: My question has been answered, please close or let me know if I need to close this issue if you think if this is not relevant futher.
Let's keep this open. You have made a very valid point/request. I will discuss this within our team.
Thank you