ninja process hanging forever when building esp32 arduino lib builder with component having ${CMAKE_BINARY_DIR} in INCLUDE_DIRS, idf_component_register (IDFGH-14412)

Question

ninja process hanging forever when building esp32 arduino lib builder with component having ${CMAKE_BINARY_DIR} in INCLUDE_DIRS, idf_component_register (IDFGH-14412)

bhcuong2008 opened this issue 10 months ago · 6 comments

bhcuong2008 commented 10 months ago

Answers checklist.

I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.4

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

What is the expected behavior?

Build successfully

What is the actual behavior?

ninja hanging forever at step [2820/2821] Linking CXX executable arduino-lib-builder.elf

Steps to reproduce.

Install requirements following https://github.com/espressif/esp32-arduino-lib-builder for tag idf-release_v5.4, for example, at folder /esp32/lib5.4
Run initial to install all tools, libraries, esp-idf, and wait until the end
/esp/lib5.4/build.sh -A idf-release/v5.4 -I release/v5.4 -i v5.4 -t 'esp32s3' -b idf-libs
Create test component in directory components inside lib-builder folder, in this folder /esp32/lib5.4/components with command
source ../esp-idf/export.sh
idf.py create-component test
Edit created test/CMakeLists.txt as following:

idf_component_register(
    SRCS "test.c"
    INCLUDE_DIRS "${CMAKE_BINARY_DIR}" "include"
)

Save and go to folder /esp32/lib5.4, run command again with option -s to avoid download tools, libs,...again.
build.sh -s -A idf-release/v5.4 -I release/v5.4 -i v5.4 -t 'esp32s3' -b idf-libs
The build process will run and hang at 2820/2821

[2819/2821] Building C object CMakeFiles/arduino-lib-builder.elf.dir/project_elf_src_esp32s3.c.obj
[2820/2821] Linking CXX executable arduino-lib-builder.elf

Now, remove "${CMAKE_BINARY_DIR}" in file CMakeLists.txt, and run again, it will be successful.
build.sh -s -A idf-release/v5.4 -I release/v5.4 -i v5.4 -t 'esp32s3' -b idf-libs

So, why ${CMAKE_BINARY_DIR} or ${BUILD_DIR} causes this hang?

Build or installation Logs.

...
[2799/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ZigbeeHandlers.cpp.obj
[2800/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeDimmableLight.cpp.obj
[2801/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeSwitch.cpp.obj
[2802/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeLight.cpp.obj
[2803/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/MatterEndpoints/MatterHumiditySensor.cpp.obj
[2804/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeTempSensor.cpp.obj
[2805/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeThermostat.cpp.obj
[2806/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeOccupancySensor.cpp.obj
[2807/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeFlowSensor.cpp.obj
[2808/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeeCarbonDioxideSensor.cpp.obj
[2809/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Zigbee/src/ep/ZigbeePressureSensor.cpp.obj
[2810/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/MatterEndpoints/MatterThermostat.cpp.obj
[2811/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/MatterEndpoints/MatterPressureSensor.cpp.obj
[2812/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/MatterEndpoints/MatterOnOffPlugin.cpp.obj
[2813/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/MatterEndpoints/MatterOccupancySensor.cpp.obj
[2814/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/MatterEndpoints/MatterContactSensor.cpp.obj
[2815/2821] Building CXX object esp-idf/arduino/CMakeFiles/__idf_arduino.dir/libraries/Matter/src/Matter.cpp.obj
[2816/2821] Linking C static library esp-idf/arduino/libarduino.a
[2817/2821] Linking C static library esp-idf/main/libmain.a
[2818/2821] Generating ld/sections.ld
warning: default on the choice symbol SEC_CERT_DAC_PROVIDER (defined at /cfm/esp32/lib5.4_mp1.24/managed_components/espressif__esp_matter/connectedhomeip/connectedhomeip/config/esp32/components/chip/Kconfig:929, /cfm/esp32/lib5.4_mp1.24/managed_components/espressif__esp_matter/components/esp_matter/Kconfig:48) will have no effect, as defaults do not affect choice symbols
warning: the choice symbol SEC_CERT_DAC_PROVIDER (defined at /cfm/esp32/lib5.4_mp1.24/managed_components/espressif__esp_matter/connectedhomeip/connectedhomeip/config/esp32/components/chip/Kconfig:929, /cfm/esp32/lib5.4_mp1.24/managed_components/espressif__esp_matter/components/esp_matter/Kconfig:48) is defined with a prompt outside the choice
info: INFO: Symbol SEC_CERT_DAC_PROVIDER defined in multiple locations (see below). Please check if this is a correct behavior or a random name match:
    /cfm/esp32/lib5.4_mp1.24/managed_components/espressif__esp_matter/components/esp_matter/Kconfig:48
    /cfm/esp32/lib5.4_mp1.24/managed_components/espressif__esp_matter/connectedhomeip/connectedhomeip/config/esp32/components/chip/Kconfig:929
[2819/2821] Building C object CMakeFiles/arduino-lib-builder.elf.dir/project_elf_src_esp32s3.c.obj
[2820/2821] Linking CXX executable arduino-lib-builder.elf

More Information.

No response

Answer 1 · 2025-01-14T09:54:52.000Z

It's actually hanging while trying to collect all files for Arduino. Having that directive there is probably making it search through folders that is not prepared to go through and is running in circles. You can check the out folder for what is being put there and maybe add some prints in the build script to try and figure out where is it looping.

Answer 2 · 2025-01-14T10:00:26.000Z

Is there anyway to control this behavior, such as excluding some folders for searching? Because some components could include build dir in INCLUDE_DIRS rather than PRIV_INCLUDE_DIRS, for example, micropython, micropython-ulab use INTERFACE library while building. It refers to runtime created header files inside build dir. So if including build dir (CMAKE_BINARY_DIR) in INCLUDE_DIRS, the whole build process could not finished. I waited for several hours but it is still there.

Answer 3 · 2025-01-14T10:26:10.000Z

The tooling here is tailored specifically for what Arduino needs. It extracts the files it also needs from the build dir (sdkconfig.h, bootloaders, etc), so maybe you need to be more specific than just adding the whole build dir to includes.

Answer 4 · 2025-01-14T10:30:46.000Z

Could you help me which codes, in which files, to search through include dirs to collect files for Arduino? I will modify the script to exclude build dir while collecting files but still existing when compile codes. Currently my compiling/linking is successful, but stuck at collecting files phase that you describe.

Answer 5 · 2025-01-14T10:44:21.000Z

I found it in file tools/copy-libs.sh, I will investigate it.

Answer 6 · 2025-01-14T11:07:53.000Z

I tried but ninja still hanging at phase "Linking CXX executable arduino-lib-builder.elf" before executing copy-libs.sh. I don't know whether it is ninja process or arduino build/collect scripts.