Support of Arduino Matter library
Humancell opened this issue ยท 83 comments
Related area
Arduino Library
Hardware specification
All Boards
Is your feature request related to a problem?
The Matter specification has been released, and there has been IDF support for quite some time. At one level is would be good to see examples of using these APIs from the Arduino environment. At another level it would be great if there was a higher level Arduino library created that leveraged these APIs and provided examples for both implementing and controlling a Matter device.
Describe the solution you'd like
It would be great if there was a higher level Arduino library created that leveraged the IDF Matter APIs and provided examples for both implementing and controlling a Matter device.
Describe alternatives you've considered
There are some other people outside of this project starting to experiment with this, but they are not fully open source nor sponsored by Espressif.
Additional context
For the Espressif platform to dominate in the market for Matter devices it could be engaging the Arduino developers better.
I have checked existing list of Feature requests and the Contribution Guide
- I confirm I have checked existing list of Feature requests and Contribution Guide.
Why does this need to be part of the core? It can be handled in a 3rd party library.
Hello @Humancell,
thanks for opening this feature Request.
I'm adding this to next major milestone 3.0.0 and we will evaluate it.
Hello,
Thank you for considering this!
@lbernstone I can understand your question, but I see this no differently from all of the other libraries that are already a part of core. We already have Bluetooth, HTTP, DNS, Webserver, NetBIOS, RainMaker and many more. Just as many of these are built on the esp-protocols, this - it seems - would be built on https://github.com/espressif/esp-matter .
It is a protocol layer provided by Espressif in the IDF, and besides the technical reasons I believe by exposing it to Arduino developers it could assist the ESP32 in becoming an even more powerful platform for the development of Matter controllers and devices.
You can try library I have created few days ago https://github.com/jakubdybczak/esp32-arduino-matter
Hello,
You can try library I have created few days ago https://github.com/jakubdybczak/esp32-arduino-matter
I did see this library, but from my understanding you have not made this 100% open source, which is a problem for us. We can not use any closed source code in our projects.
If I am incorrect, and this really is 100% open source then we could take a look.
@Humancell
Why do you think that this one is not open-source?
arduino-esp32 also contains precompiled libraries and scripts that do this are in esp32-arduino-lib-builder repo.
Same with esp32-arduino-matter, you can find scripts used to build it there esp32-arduino-matter-builder.
We will consider this request for next releases in 2023... please be patient.
I have done an initial effort to build Matter and Arduino as Component.
The steps are here:
Hi all
Is there any progress for supporting Matter to the Arduino core?
@HamzaHajeir - We are currently working to release OpenThread as an Arduino Library. This shall be done later this month.
Next step is to work on releasing Matter as an Arduino Library.
I believe that it will be done by the end of this year.
About Matter Library implementation for Arduino Core 3.0.x:
- We will initially implement it to be supported within WiFi. In the future also within Thread Protocol.
- WiFi Matter will be supported for ESP32, ESP32-C3, ESP32-S3, ESP32-S2 and ESP32-C6. No support for ESP32-H2 and ESP32-P4, which have no WiFi radio. ESP32-S2 has no BLE, it won't have Matter Device BLE advertising service.
- Thread Network Protocol will be supported for the ESP32-C6 and ESP32-H2. Both have also BLE, therefore, BLE Commissionable discovery is also available.
- Commissionable discovery will be done through BLE (whenever available in the SoC), WiFi SoftAP (future feature), DNS-SD for Thread Border Routers (future feature).
- Commissioning will follow Matter standard, including QR Code generated from a URL printed in the log output.
- ESP32 Matter implementation for BLE Commissionable discovery is based on NimBLE. Therefore, it won't be compatible with ESP32 BLE Arduino Library that uses Bluedroid instead. This means that there may be issues when using both together in the same Arduino Sketch. This will be a limitation. We are planning to move ESP32 Arduino BLE Library to use NimBLE as well in the future, which may solve this incompatibility problem.
- ESP32 Matter Arduino implementation will make available all possible Matter 1.0 device types as Arduino Classes: Lighting, Smart Plugs, Switches and Controls, Sensors, Closures and HVACs. Each Device Type Class will have necessary implementation of the device data model, endpoints, roles, clusters, commands and events.
- The current implementation is only for Matter Device. No plans for Matter Controllers or Matter Commissioner library.
- This implementation will use testing and development Matter Vendor ID (0xFFF1..4), therefore valid for CSA for testing purpose only. In case you need your own VID, please follow CSA processes and add certificates as necessary to your code, on your own.
About Matter Library implementation for Arduino Core 3.0.x:
- We will initially implement it to be supported within WiFi. In the future also within Thread Protocol.
- WiFi Matter will be supported for ESP32, ESP32-C3, ESP32-S3, ESP32-S2 and ESP32-C6. No support for ESP32-H2 and ESP32-P4, which have no WiFi radio. ESP32-S2 has no BLE, it won't have Matter Device BLE advertising service.
- Thread Network Protocol will be supported for the ESP32-C6 and ESP32-H2. Both have also BLE, therefore, BLE Commissionable discovery is also available.
- Commissionable discovery will be done through BLE (whenever available in the SoC), WiFi SoftAP (future feature), DNS-SD for Thread Border Routers (future feature).
- Commissioning will follow Matter standard, including QR Code generated from a URL printed in the log output.
- ESP32 Matter implementation for BLE Commissionable discovery is based on NimBLE. Therefore, it won't be compatible with ESP32 BLE Arduino Library that uses Bluedroid instead. This means that there may be issues when using both together in the same Arduino Sketch. This will be a limitation. We are planning to move ESP32 Arduino BLE Library to use NimBLE as well in the future, which may solve this incompatibility problem.
- ESP32 Matter Arduino implementation will make available all possible Matter 1.0 device types as Arduino Classes: Lighting, Smart Plugs, Switches and Controls, Sensors, Closures and HVACs. Each Device Type Class will have necessary implementation of the device data model, endpoints, roles, clusters, commands and events.
- The current implementation is only for Matter Device. No plans for Matter Controllers or Matter Commissioner library.
- This implementation will use testing and development Matter Vendor ID (0xFFF1..4), therefore valid for CSA for testing purpose only. In case you need your own VID, please follow CSA processes and add certificates as necessary to your code, on your own.
Glad to hear that's coming support of Matter in the Arduino core. I have just some questions and comments:
- ESP-IDF identifies several tools related to setup Matter device (as esp-matter-mfg-tool), are these included also?
- Per Commissioning, please consider On-Network commisioning method, in which some could have his own device on the network previously, by either Ethernet or custom implementation of WiFi-AP.
- Therefore, I think it's okay to have ESP32-S2 supported with respect to such user responsibility (or even with the feature of WiFiAP commissioning).
- Is it available for some to build a custom device using matter data and interaction model?
With thanks,
Hamza Hajeir
@HamzaHajeir - Thanks for the comments.
ESP-IDF identifies several tools related to setup Matter device (as esp-matter-mfg-tool), are these included also?
Not as part of the tools that ESP32 Arduino would provide.
These tools, as the CHIP tools, or any other Python tools, can be used independently of the ESP32 SDK used to build Matter Applications. These tools can be used while using IDF, Rust or Arduino Frameworks for Matter development.
Per Commissioning, please consider On-Network commisioning method, in which some could have his own device on the network previously, by either Ethernet or custom implementation of WiFi-AP.
Commissioning will always be done on-network.
The device must be already on the network and WiFi/Thread credentials shall be already in place.
Ethernet must have the cable connected.
In all cases, the device must have a valid IP address (DHCP or Static IP).
Therefore, On-Network Commissioning method will be included.
Therefore, I think it's okay to have ESP32-S2 supported with respect to such user responsibility (or even with the feature of WiFiAP commissioning).
Yes, correct.
Is it available for some to build a custom device using matter data and interaction model?
We may consider it for future feature. At first, we'll focus on the Matter 1.0 device types as described above.
But given that ESP32 Arduino is based on IDF and ESP-Matter, it is always possible to add code from those two to an Arduino Sketch and We may consider it for future feature. At first, we'll focus on the Matter 1.0 device types as described above.
But given that ESP32 Arduino is based on IDF and ESP-Matter, it is always possible to add code from those two to an Arduino Sketch and create custom devices, as well as custom interaction models.
@SuGlider Nice, that explains all, I was mixing between commissioning discovery and commissioning process, seems two different operations.
However, it seems that all Factory Data Providers would be 'custom' in Arduino, correct?
However, it seems that all Factory Data Providers would be 'custom' in Arduino, correct?
Arduino will set the Matter Device as a Testing unit using a VIDs allocated by the CSA for testing purposes.
https://developers.home.google.com/matter/get-started#vendor_id
No Real Provider information would be used within Arduino.
Commissioning will happen as defined in the description from https://developers.home.google.com/matter/primer/commissioning
Discovery may be done using BLE and/or WiFi AP, as described here: https://developers.home.google.com/matter/primer/commissionable-and-operational-discovery
However, it seems that all Factory Data Providers would be 'custom' in Arduino, correct?
Arduino will set the Matter Device as a Testing unit using a VIDs allocated by the CSA for testing purposes. https://developers.home.google.com/matter/get-started#vendor_id No Real Provider information would be used within Arduino.
But this might contradict with this (regarding the providers), where it should be a custom one if someone would use the Arduino:
- This implementation will use testing and development Matter Vendor ID (0xFFF1..4), therefore valid for CSA for testing purpose only. In case you need your own VID, please follow CSA processes and add certificates as necessary to your code, on your own.
What I could suggest is making it custom with overridable test providers: defaulted to a derived class of such, that points to test information.
Commissioning will happen as defined in the description from https://developers.home.google.com/matter/primer/commissioning
Discovery may be done using BLE and/or WiFi AP, as described here: https://developers.home.google.com/matter/primer/commissionable-and-operational-discovery
I'll read about them soon and check if any suggestion or comment I can assist with.
@SuGlider I try to build with 3.0 as component and latest esp matter but it not working, show error:
build/esp-idf/chip/gen/include/system/SystemBuildConfig.h:31:38:fatal error: platform/1/CHIPPlatformConfig.h: No such file or directory
31 | #define CHIP_PLATFORM_CONFIG_INCLUDE <platform/ESP32/CHIPPlatformConfig.h>
@SuGlider I try to build with 3.0 as component and latest esp matter but it not working, show error:
build/esp-idf/chip/gen/include/system/SystemBuildConfig.h:31:38:fatal error: platform/1/CHIPPlatformConfig.h: No such file or directory 31 | #define CHIP_PLATFORM_CONFIG_INCLUDE <platform/ESP32/CHIPPlatformConfig.h>
I think that goes after substituting ESP32 to its preprocessor definition 1.
Therefore there should be the compile-time definition of ESP32. What steps you've followed to reach to this point?
@SuGlider I try to build with 3.0 as component and latest esp matter but it not working, show error:
build/esp-idf/chip/gen/include/system/SystemBuildConfig.h:31:38:fatal error: platform/1/CHIPPlatformConfig.h: No such file or directory 31 | #define CHIP_PLATFORM_CONFIG_INCLUDE <platform/ESP32/CHIPPlatformConfig.h>I think that goes after substituting ESP32 to its preprocessor definition 1.
Therefore there should be the compile-time definition of ESP32. What steps you've followed to reach to this point?
I just using Matter sample light project and add Arduino as a components
@SuGlider I try to build with 3.0 as component and latest esp matter but it not working, show error:
build/esp-idf/chip/gen/include/system/SystemBuildConfig.h:31:38:fatal error: platform/1/CHIPPlatformConfig.h: No such file or directory 31 | #define CHIP_PLATFORM_CONFIG_INCLUDE <platform/ESP32/CHIPPlatformConfig.h>I think that goes after substituting ESP32 to its preprocessor definition 1.
Therefore there should be the compile-time definition of ESP32. What steps you've followed to reach to this point?I just using Matter sample light project and add Arduino as a components
Matter + Arduino as IDF Component:
My suggestion is using Matter from the ESP Registry (Managed Components)
https://components.espressif.com/components/espressif/esp_matter/versions/1.3.0
Start with this example:
https://github.com/espressif/esp-matter/tree/f4eaa699f0fd8433bc5ca134328934f68f1e1525/examples/managed_component_light
Build it as it is. It will work as a Matter Light with 5 different White tones/temperature plus a Dimmer.
I have commissioned and tested it with the Alexa APP and it works fine (it requires an Alexa device), but it shall also work with Apple Home, or using Google HA - after setting the google applications to accept matter development accessories.
After making it work with your target ESP32 SoC, you can add /components/arduino or use the Arduino from Registry and modify the source code from the \main folder to add your Arduino Code.
OK, using Registry, I also got the same error.
Let me check how to solve it.
OK, using Registry, I also got the same error. Let me check how to solve it.
Yes, hope you will find solution.
I have found the origin of the problem...
Arduino Component has a -DESP32, which means #define ESP32 1
CHIP (ConnectedHomeIP) has a #define CHIP_PLATFORM_CONFIG_INCLUDE <platform/ESP32/CHIPPlatformConfig.h>
which ends up expanded to #define CHIP_PLATFORM_CONFIG_INCLUDE <platform/1/CHIPPlatformConfig.h> because of Arduino Component defines.
This is the source of the issue.
To solve it, you can change Ardunino Component CMakeLists.txt content at line 323 (remove it, maybe?)
target_compile_options(${COMPONENT_TARGET} PUBLIC
-DARDUINO=10812
-DARDUINO_${idf_target_for_macro}_DEV
-DARDUINO_ARCH_ESP32
-DARDUINO_BOARD="${idf_target_caps}_DEV"
-DARDUINO_VARIANT="${CONFIG_ARDUINO_VARIANT}"
-DESP32)
@dzungpv - I have tested it. -DESP32=ESP32 solves the issue. I'll add a PR to change it in Arduino repository.
Wow, nasty one.
Not easy to find the origin.
@dzungpv - I have tested it.
-DESP32=ESP32solves the issue. I'll add a PR to change it in Arduino repository.
Yes, change it make build sample light matter success, but other problem when I am using Arduino WiFi, the error log:
wind_FindEnclosingFunction -Wl,--wrap=_Unwind_Resume -Wl,--wrap=_Unwind_RaiseException -Wl,--wrap=_Unwind_DeleteException -Wl,--wrap=_Unwind_ForcedUnwind -Wl,--wrap=_Unwind_Resume_or_Rethrow -Wl,--wrap=_Unwind_Backtrace -Wl,--wrap=__cxa_call_unexpected -Wl,--wrap=__gxx_personality_v0 -u __cxa_guard_dummy -lstdc++ esp-idf/pthread/libpthread.a -lgcc esp-idf/cxx/libcxx.a -u __cxx_fatal_exception -u include_esp_phy_override -lphy -lrtc -lphy -lrtc esp-idf/esp_phy/libesp_phy.a -u vfs_include_syscalls_impl -lbtdm_app -u ld_include_hli_vectors_bt && :
/Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj):(.literal._Z16wifiLowLevelInitb+0x40): undefined reference to `esp_netif_create_default_wifi_ap'
/Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj): in function `_Z16wifiLowLevelInitb':
/Users/vietdzung/dev/projects/esp32/sdks/5.1.4/arduino-esp32/libraries/WiFi/src/WiFiGeneric.cpp:178: undefined reference to `esp_netif_create_default_wifi_ap'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
@dzungpv - I have tested it.
-DESP32=ESP32solves the issue. I'll add a PR to change it in Arduino repository.Yes, change it make build sample light matter success, but other problem when I am using Arduino WiFi, the error log:
wind_FindEnclosingFunction -Wl,--wrap=_Unwind_Resume -Wl,--wrap=_Unwind_RaiseException -Wl,--wrap=_Unwind_DeleteException -Wl,--wrap=_Unwind_ForcedUnwind -Wl,--wrap=_Unwind_Resume_or_Rethrow -Wl,--wrap=_Unwind_Backtrace -Wl,--wrap=__cxa_call_unexpected -Wl,--wrap=__gxx_personality_v0 -u __cxa_guard_dummy -lstdc++ esp-idf/pthread/libpthread.a -lgcc esp-idf/cxx/libcxx.a -u __cxx_fatal_exception -u include_esp_phy_override -lphy -lrtc -lphy -lrtc esp-idf/esp_phy/libesp_phy.a -u vfs_include_syscalls_impl -lbtdm_app -u ld_include_hli_vectors_bt && : /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj):(.literal._Z16wifiLowLevelInitb+0x40): undefined reference to `esp_netif_create_default_wifi_ap' /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj): in function `_Z16wifiLowLevelInitb': /Users/vietdzung/dev/projects/esp32/sdks/5.1.4/arduino-esp32/libraries/WiFi/src/WiFiGeneric.cpp:178: undefined reference to `esp_netif_create_default_wifi_ap' collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.
@dzungpv - What is your target SoC? Please post a sketch that I can use to test it.
Please have in mind that matter will control WiFi communication as well and it may represent a conflict.
@dzungpv - I have tested it.
-DESP32=ESP32solves the issue. I'll add a PR to change it in Arduino repository.Yes, change it make build sample light matter success, but other problem when I am using Arduino WiFi, the error log:
wind_FindEnclosingFunction -Wl,--wrap=_Unwind_Resume -Wl,--wrap=_Unwind_RaiseException -Wl,--wrap=_Unwind_DeleteException -Wl,--wrap=_Unwind_ForcedUnwind -Wl,--wrap=_Unwind_Resume_or_Rethrow -Wl,--wrap=_Unwind_Backtrace -Wl,--wrap=__cxa_call_unexpected -Wl,--wrap=__gxx_personality_v0 -u __cxa_guard_dummy -lstdc++ esp-idf/pthread/libpthread.a -lgcc esp-idf/cxx/libcxx.a -u __cxx_fatal_exception -u include_esp_phy_override -lphy -lrtc -lphy -lrtc esp-idf/esp_phy/libesp_phy.a -u vfs_include_syscalls_impl -lbtdm_app -u ld_include_hli_vectors_bt && : /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj):(.literal._Z16wifiLowLevelInitb+0x40): undefined reference to `esp_netif_create_default_wifi_ap' /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj): in function `_Z16wifiLowLevelInitb': /Users/vietdzung/dev/projects/esp32/sdks/5.1.4/arduino-esp32/libraries/WiFi/src/WiFiGeneric.cpp:178: undefined reference to `esp_netif_create_default_wifi_ap' collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.@dzungpv - What is your target SoC? Please post a sketch that I can use to test it. Please have in mind that matter will control WiFi communication as well and it may represent a conflict.
I user Arduino sample WifiClient for ESP32
/*
Go to thingspeak.com and create an account if you don't have one already.
After logging in, click on the "New Channel" button to create a new channel for your data. This is where your data will be stored and displayed.
Fill in the Name, Description, and other fields for your channel as desired, then click the "Save Channel" button.
Take note of the "Write API Key" located in the "API keys" tab, this is the key you will use to send data to your channel.
Replace the channelID from tab "Channel Settings" and privateKey with "Read API Keys" from "API Keys" tab.
Replace the host variable with the thingspeak server hostname "api.thingspeak.com"
Upload the sketch to your ESP32 board and make sure that the board is connected to the internet. The ESP32 should now send data to your Thingspeak channel at the intervals specified by the loop function.
Go to the channel view page on thingspeak and check the "Field1" for the new incoming data.
You can use the data visualization and analysis tools provided by Thingspeak to display and process your data in various ways.
Please note, that Thingspeak accepts only integer values.
You can later check the values at https://thingspeak.com/channels/2005329
Please note that this public channel can be accessed by anyone and it is possible that more people will write their values.
*/
#include <WiFi.h>
const char* ssid = "your-ssid"; // Change this to your WiFi SSID
const char* password = "your-password"; // Change this to your WiFi password
const char* host = "api.thingspeak.com"; // This should not be changed
const int httpPort = 80; // This should not be changed
const String channelID = "2005329"; // Change this to your channel ID
const String writeApiKey = "V6YOTILH9I7D51F9"; // Change this to your Write API key
const String readApiKey = "34W6LGLIFXD56MPM"; // Change this to your Read API key
// The default example accepts one data filed named "field1"
// For your own server you can ofcourse create more of them.
int field1 = 0;
int numberOfResults = 3; // Number of results to be read
int fieldNumber = 1; // Field number which will be read out
void setup()
{
Serial.begin(115200);
while(!Serial){delay(100);}
// We start by connecting to a WiFi network
Serial.println();
Serial.println("******************************************************");
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
}
void readResponse(WiFiClient *client){
unsigned long timeout = millis();
while(client->available() == 0){
if(millis() - timeout > 5000){
Serial.println(">>> Client Timeout !");
client->stop();
return;
}
}
// Read all the lines of the reply from server and print them to Serial
while(client->available()) {
String line = client->readStringUntil('\r');
Serial.print(line);
}
Serial.printf("\nClosing connection\n\n");
}
void loop(){
WiFiClient client;
String footer = String(" HTTP/1.1\r\n") + "Host: " + String(host) + "\r\n" + "Connection: close\r\n\r\n";
// WRITE --------------------------------------------------------------------------------------------
if (!client.connect(host, httpPort)) {
return;
}
client.print("GET /update?api_key=" + writeApiKey + "&field1=" + field1 + footer);
readResponse(&client);
// READ --------------------------------------------------------------------------------------------
String readRequest = "GET /channels/" + channelID + "/fields/" + fieldNumber + ".json?results=" + numberOfResults + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n";
if (!client.connect(host, httpPort)) {
return;
}
client.print(readRequest);
readResponse(&client);
// -------------------------------------------------------------------------------------------------
++field1;
delay(10000);
}
I know matter handle wifi but it show error event I not calling any Arduino wifi function in my project, I use WifiClient only for confirm the bug.
I think that Arduino WiFi library won't work when Matter is used along with it.
Matter has many native IDF calls to WiFi, LwIP and other network components.
I'd suggest that the project would need to use native IDF WiFi API in order to make it work.
The same will happen with BLE. Actually it is worst because Arduino BLE Library uses Blueroid and IDF Matter uses NimBLE. Both won't work together.
I will try comment out wifi client lib from Arduino. but it work with Arduino 2.x. At least not show error when build code
And many lib using wifi lib, not for create connection but check IP address or something, so remove wifi lib make Arduino useless. I can write code in native C++ but loose advantage of Arduino. From my log it failed at final build state when merge all the libs together.
Are you mixing WiFi calls and ESP Matter calls in the same project?
Or just trying to build an Arduino WiFi Project using Arduino as IDF component and adding to it ESP Matter library?
ESP_Matter over WiFi will use the settings from NVS to establish a connection the AP.
If none, it will wait for Matter Commissioning to complete in order to establish a connection to the AP.
Arduino WiFi library may break it in case it starts a connection to a different AP or it stops the current connection started by ESP_Matter.
Once WiFi is set and the ESP32 STA connects to the AP, it may be possible to use some Arduino WiFi API calls, but it may depend on previously configuration and setting values for the internal WiFi Class members. Therefore, it may not work as well.
ESP_Matter can't work within WiFi AP mode.
If the Arduino WiFi API method calls IDF LwIP/netif API directly, it may get correct results, but remember that ESP_Matter relies on IPv6.
Adding on top of the above, I think that the WiFi Events would have a conflict when Arduino WiFi Library and, at the same time, the ESP Matter also consume them.
I want to use matter with exist Arduino projects and libs. Esp idf or arduino wifi handle I think not difference. I have merge Rainmaker and Esp Homekit sdk too, RM handle wifi but I can use Arduino to check wifi state, ip and more even Arduino only support Bluedroid.
I try to delete DESP32 tag and see that problem may come with namespace Wifi using by Matter, the compiler confuse with ::Wifi and Wifi from Arduino lib. Sum up, it could not build, so we will not talk about compatible now, now I just want to be success build with Arduino lib.
@dzungpv - Thanks for the explanation about your request.
Let me investigate it and try to come up with a way to get these information when using Arduino + Matter as IDF Component:
- Get the WiFi connection Status
- Get the local IP address
Blueroid is for BLE operation only. NimBLE is another BLE stack.
Both are not related to WiFi.
For Nimbel and Bluedroid, it is only example. With Rainmaker project I use it to get Bssi, Rssi and Wifi scan too, it work fine with native IDF wifi
RainMaker is an application protocol, based on Netowrk Client/Server. It doesn't care about the network transport layer.
Matter is also an application protocol, but it cares about the network transport layer.
Therefore Matter controls the WiFi, Ethernet and Thread stack.
Matter has its own WiFi Event Loop.
Arduino WiFi Library also controls the WiFi Event Loop, which causes a conflict.
In order to get the WiFi Status, your application shall use the Matter Callbacks and Matter mechanism for getting events in the user application side.
It is already in the example code and WiFi events can be received using the void app_event_cb(const ChipDeviceEvent *event, intptr_t arg) function.
ChipDeviceEvent has all necessary information for knowing when it has connected, what is the IP address, etc.
chip::DeviceLayer::DeviceEventType:: namespace is what you need.
@dzungpv - It seems that CHIP/Matter provides a way to control WiFi using RPC.
https://project-chip.github.io/connectedhomeip-doc/guides/esp32/rpc_console.html
@SuGlider Yes, now how I can build project with Arduino core 3.x and Matter? Because it simply failed.
We must find a way to make build success, because many Arduino libs using Wifi lib.
Then later we can use alternative way to get Wifi information.
This is a long term development process.
The main goal of the current effort is to allow users to build Matter Accessories using an ESP32 Arduino Matter Library (to be created). In such case, it will use the native Matter WiFi control layer and the native Matter Thread control layer as the first approach.
Currently I'm working on:
- Making it work using Arduino Core 3.0.x + IDF 5.1 using Arduino + Matter as IDF Components.
- Making it work using Arduino Core 3.1.x + IDF 5.3 also using Arduino + Matter as IDF Components.
- Once it works fine, next step would be building the necessary Arduino pre-compiled libraries using Arduino Lib Builder tools.
I would say that making Arduino WiFi Library and Arduino Matter Library compatible with each other is not in the priority list of the current project. But it can be achieved in the long term.
Yes, now how I can build project with Arduino core 3.x and Matter?
I'll post an example using Arduino + Matter as IDF Component based on Arduino Core 3.0.5 and IDF 5.1.
It shall be placed in this folder:
https://github.com/espressif/arduino-esp32/tree/master/idf_component_examples/
I think I may have it done by the end of the next week.
This example will be based on the this code:
https://github.com/espressif/esp-matter/tree/f4eaa699f0fd8433bc5ca134328934f68f1e1525/examples/managed_component_light
@dzungpv - I have tested it.
-DESP32=ESP32solves the issue. I'll add a PR to change it in Arduino repository.Yes, change it make build sample light matter success, but other problem when I am using Arduino WiFi, the error log:
wind_FindEnclosingFunction -Wl,--wrap=_Unwind_Resume -Wl,--wrap=_Unwind_RaiseException -Wl,--wrap=_Unwind_DeleteException -Wl,--wrap=_Unwind_ForcedUnwind -Wl,--wrap=_Unwind_Resume_or_Rethrow -Wl,--wrap=_Unwind_Backtrace -Wl,--wrap=__cxa_call_unexpected -Wl,--wrap=__gxx_personality_v0 -u __cxa_guard_dummy -lstdc++ esp-idf/pthread/libpthread.a -lgcc esp-idf/cxx/libcxx.a -u __cxx_fatal_exception -u include_esp_phy_override -lphy -lrtc -lphy -lrtc esp-idf/esp_phy/libesp_phy.a -u vfs_include_syscalls_impl -lbtdm_app -u ld_include_hli_vectors_bt && : /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj):(.literal._Z16wifiLowLevelInitb+0x40): undefined reference to `esp_netif_create_default_wifi_ap' /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj): in function `_Z16wifiLowLevelInitb': /Users/vietdzung/dev/projects/esp32/sdks/5.1.4/arduino-esp32/libraries/WiFi/src/WiFiGeneric.cpp:178: undefined reference to `esp_netif_create_default_wifi_ap' collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.
Undefined reference implies that some sources are not included in the build. I remember I had this one just recently, I just enabled something within WiFi component through esp-idf menuconfig, then it compiles.
Per ESP-Matter and WiFi library coexistence, I believe it's possible through telling Matter that WiFi is controlled by Application.
And, here the source shows not reacting to calls (and wifi events) if so: https://github.com/project-chip/connectedhomeip/blob/master/src/platform/ESP32/ConnectivityManagerImpl_WiFi.cpp [Search for kWiFiStationMode_ApplicationControlled].
I believe that the designers of CHIP just considers this range of use cases, where clients implement their own wifi management code. Consider a product which was in market before adopting Matter and yet have its own wifi management, should the manufacturer deliberately use CHIP wifi management? Nope.
@dzungpv - I have tested it.
-DESP32=ESP32solves the issue. I'll add a PR to change it in Arduino repository.Yes, change it make build sample light matter success, but other problem when I am using Arduino WiFi, the error log:
wind_FindEnclosingFunction -Wl,--wrap=_Unwind_Resume -Wl,--wrap=_Unwind_RaiseException -Wl,--wrap=_Unwind_DeleteException -Wl,--wrap=_Unwind_ForcedUnwind -Wl,--wrap=_Unwind_Resume_or_Rethrow -Wl,--wrap=_Unwind_Backtrace -Wl,--wrap=__cxa_call_unexpected -Wl,--wrap=__gxx_personality_v0 -u __cxa_guard_dummy -lstdc++ esp-idf/pthread/libpthread.a -lgcc esp-idf/cxx/libcxx.a -u __cxx_fatal_exception -u include_esp_phy_override -lphy -lrtc -lphy -lrtc esp-idf/esp_phy/libesp_phy.a -u vfs_include_syscalls_impl -lbtdm_app -u ld_include_hli_vectors_bt && : /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj):(.literal._Z16wifiLowLevelInitb+0x40): undefined reference to `esp_netif_create_default_wifi_ap' /Users/vietdzung/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: esp-idf/arduino-esp32/libarduino-esp32.a(WiFiGeneric.cpp.obj): in function `_Z16wifiLowLevelInitb': /Users/vietdzung/dev/projects/esp32/sdks/5.1.4/arduino-esp32/libraries/WiFi/src/WiFiGeneric.cpp:178: undefined reference to `esp_netif_create_default_wifi_ap' collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.Undefined reference implies that some sources are not included in the build. I remember I had this one just recently, I just enabled something within WiFi component through esp-idf menuconfig, then it compiles.
Per ESP-Matter and WiFi library coexistence, I believe it's possible through telling Matter that WiFi is controlled by Application.
And, here the source shows not reacting to calls (and wifi events) if so: https://github.com/project-chip/connectedhomeip/blob/master/src/platform/ESP32/ConnectivityManagerImpl_WiFi.cpp [Search for kWiFiStationMode_ApplicationControlled].
I believe that the designers of CHIP just considers this range of use cases, where clients implement their own wifi management code. Consider a product which was in market before adopting Matter and yet have its own wifi management, should the manufacturer deliberately use CHIP wifi management? Nope.
Follow your hint I was find the config that make Arduino could not build, it is CONFIG_ESP_WIFI_SOFTAP_SUPPORT=n, I enable it then the code can build. Now let play with Wifi part.
@SuGlider It is https://github.com/espressif/esp-matter/blob/f4eaa699f0fd8433bc5ca134328934f68f1e1525/examples/managed_component_light/sdkconfig.defaults#L36 change n to y then it is OK to build
I believe that the designers of CHIP just considers this range of use cases, where clients implement their own wifi management code. Consider a product which was in market before adopting Matter and yet have its own wifi management, should the manufacturer deliberately use CHIP wifi management? Nope.
Correct.
When using ESP32 Matter, WiFi is managed by Matter when CONFIG_ENABLE_WIFI_STATION or CONFIG_ENABLE_WIFI_AP are set in thesdkconfig file.
When both are unset, Matter won't manage WiFi. In such case, WiFi provisioning shall be managed by the application.
I believe that the designers of CHIP just considers this range of use cases, where clients implement their own wifi management code. Consider a product which was in market before adopting Matter and yet have its own wifi management, should the manufacturer deliberately use CHIP wifi management? Nope.
Correct. When using ESP32 Matter, WiFi is managed by Matter when
CONFIG_ENABLE_WIFI_STATIONorCONFIG_ENABLE_WIFI_APare set in thesdkconfigfile. When both are unset, Matter won't manage WiFi. In such case, WiFi provisioning shall be managed by the application.
Some correction by considering that CONFIG_ENABLE_WIFI_STATION needs to be defined, so Matter can react to change its state onConnect and onDisconnect. Rather the developer needs to set ConnectivityMgrImpl()._SetWiFiStationMode(kWiFiStationMode_ApplicationControlled) if he wants to manage WiFi, if he have solid and reliable implementation of.
@SuGlider The PR has a ugly side effect. It causes mass of redefine warnings with pioarduino / Platformio.
Building in release mode
Compiling .pio/build/tasmota32-nspanel/src/tasmota.ino.cpp.o
Building .pio/build/tasmota32-nspanel/bootloader.bin
Generating partitions .pio/build/tasmota32-nspanel/partitions.bin
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
esptool.py v4.7.6
Creating esp32 image...
Merged 2 ELF sections
Successfully created esp32 image.
Compiling .pio/build/tasmota32-nspanel/lib3cf/Wire/Wire.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Compiling .pio/build/tasmota32-nspanel/lib53b/Adafruit_CCS811-1.0.0.14/Adafruit_CCS811.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Compiling .pio/build/tasmota32-nspanel/lib18d/SPI/SPI.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Archiving .pio/build/tasmota32-nspanel/lib3cf/libWire.a
Indexing .pio/build/tasmota32-nspanel/lib3cf/libWire.a
Compiling .pio/build/tasmota32-nspanel/libc70/Adafruit_MAX31865-1.1.0-custom/Adafruit_MAX31865.cpp.o
Archiving .pio/build/tasmota32-nspanel/lib18d/libSPI.a
Archiving .pio/build/tasmota32-nspanel/lib53b/libAdafruit_CCS811-1.0.0.14.a
Indexing .pio/build/tasmota32-nspanel/lib53b/libAdafruit_CCS811-1.0.0.14.a
<command-line>: warning: "ESP32" redefined
Indexing .pio/build/tasmota32-nspanel/lib18d/libSPI.a
<command-line>: note: this is the location of the previous definition
Compiling .pio/build/tasmota32-nspanel/libf9d/Adafruit_MCP9808_Tasmota/Adafruit_MCP9808.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Compiling .pio/build/tasmota32-nspanel/lib2f4/Adafruit_BusIO/Adafruit_BusIO_Register.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Archiving .pio/build/tasmota32-nspanel/libc70/libAdafruit_MAX31865-1.1.0-custom
Indexing .pio/build/tasmota32-nspanel/libc70/libAdafruit_MAX31865-1.1.0-custom
Compiling .pio/build/tasmota32-nspanel/lib2f4/Adafruit_BusIO/Adafruit_I2CDevice.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Archiving .pio/build/tasmota32-nspanel/libf9d/libAdafruit_MCP9808_Tasmota.a
Indexing .pio/build/tasmota32-nspanel/libf9d/libAdafruit_MCP9808_Tasmota.a
Compiling .pio/build/tasmota32-nspanel/lib2f4/Adafruit_BusIO/Adafruit_SPIDevice.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Compiling .pio/build/tasmota32-nspanel/libc3a/Adafruit_PM25AQI-1.0.6/Adafruit_PM25AQI.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Compiling .pio/build/tasmota32-nspanel/lib0ea/Adafruit_SGP30-1.2.0/Adafruit_SGP30.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Archiving .pio/build/tasmota32-nspanel/lib2f4/libAdafruit_BusIO.a
Indexing .pio/build/tasmota32-nspanel/lib2f4/libAdafruit_BusIO.a
Compiling .pio/build/tasmota32-nspanel/libdee/Adafruit_SGP40-1.1.0/Adafruit_SGP40.cpp.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
Archiving .pio/build/tasmota32-nspanel/libc3a/libAdafruit_PM25AQI-1.0.6.a
Indexing .pio/build/tasmota32-nspanel/libc3a/libAdafruit_PM25AQI-1.0.6.a
Compiling .pio/build/tasmota32-nspanel/libdee/Adafruit_SGP40-1.1.0/sensirion_voc_algorithm.c.o
<command-line>: warning: "ESP32" redefined
<command-line>: note: this is the location of the previous definition
...
EDIT: Probably fixed with espressif/esp32-arduino-lib-builder#217
Yes, espressif/esp32-arduino-lib-builder#217 will fix PIO adding -DESP32=ESP32 to its settings.
I believe that the designers of CHIP just considers this range of use cases, where clients implement their own wifi management code. Consider a product which was in market before adopting Matter and yet have its own wifi management, should the manufacturer deliberately use CHIP wifi management? Nope.
Correct. When using ESP32 Matter, WiFi is managed by Matter when
CONFIG_ENABLE_WIFI_STATIONorCONFIG_ENABLE_WIFI_APare set in thesdkconfigfile. When both are unset, Matter won't manage WiFi. In such case, WiFi provisioning shall be managed by the application.Some correction by considering that
CONFIG_ENABLE_WIFI_STATIONneeds to be defined, so Matter can react to change its state onConnect and onDisconnect. Rather the developer needs to setConnectivityMgrImpl()._SetWiFiStationMode(kWiFiStationMode_ApplicationControlled)if he wants to manage WiFi, if he have solid and reliable implementation of.
Thank @HamzaHajeir for the information!
I see that, when both CONFIG_ENABLE_WIFI_STATION and CONFIG_ENABLE_WIFI_AP are disabled in the sdkconfig, the header file ConnectivityManagerImpl.h will use the platform/internal/GenericConnectivityManagerImpl_NoWiFi.h as the Matter Network Manager Implementation.
Therefore, maybe, we can override GenericConnectivityManagerImpl_NoWiFi class to use Arduino WiFi Library calls and achieve its integration with Matter.
Something to dig out.
Thank @HamzaHajeir for the information!
I see that, when both
CONFIG_ENABLE_WIFI_STATIONandCONFIG_ENABLE_WIFI_APare disabled in thesdkconfig, the header fileConnectivityManagerImpl.hwill use theplatform/internal/GenericConnectivityManagerImpl_NoWiFi.has the Matter Network Manager Implementation.Therefore, maybe, we can override
GenericConnectivityManagerImpl_NoWiFiclass to use Arduino WiFi Library calls and achieve its integration with Matter.Something to dig out.
To my understanding, we don't need to override the No_WiFi implementation. we just keep CONFIG_ENABLE_WIFI_STATION defined, and CONFIG_ENABLE_WIFI_AP if needed to activate AP. But telling Matter to not touch WiFi with 'ApplicationControlled' concept.
That way the application should be responsible to manage WiFi (+WiFiAP) machine state by itself. Considering some issues as WiFi AP SSID should comply to Matter's standard, for an example.
Therefore we have something like WiFiManager library. But however I don't know what Arduino WiFi library intersect with that matter, is it something as just managing automatic reconnection? We might consider relying on CHIP's implementation of WiFi management, at least as an available option...
Therefore we have something like WiFiManager library. But however I don't know what Arduino WiFi library intersect with that matter, is it something as just managing automatic reconnection? We might consider relying on CHIP's implementation of WiFi management, at least as an available option...
This integration with Arduino WiFi Library will be left for the long run.
As a first approach, the Arduino Matter Library will use CHIP and ESP_Matter as it is now, meaning that ESP_Matter will manage WiFi/Ethernet/Thread directly through ESP-IDF calls.
I also need to understand the implication of using Arduino WiFi Lib instead of CHIP WiFi Manager regarding Network Provisioning and Matter Fabric Commissioning processes.
#10467 is merged and Matter Library will be released into Arduino Core 3.1.0-RC2 or into initial release of Core 3.1.0.
Next step is the implementation of more Endpoint profiles as in the list:
- Simple On/Off Light
- Dimmable Light
- Temperature Color (CW/WW) Light
- Color RGB Light
- Enhanced Color (RGB + CW/WW) Light
- Generic Switch (single click smart button)
- Fan
- Temperature Sensor
- Humidity Sensor
- Pressure Sensor
- Contact Sensor
- Occupancy Sensor
- Thermostat
- Smart on/off Plug
Hi, I created my own Matter RGBWW LED Spot. It works fine, but I'm not able to find any easy solution to give a device a unique code to setup multiple devices. Did I missed something?
@Fideus the library is using development license, which can provide only one code. To get your own code, you need to have a device and apply for and buy certificatation from the appropriate authorities: https://matter-smarthome.de/en/development/how-the-matter-certification-works/
You can setup the devices one by one using the same code and it will be OK
Hi, I created my own Matter RGBWW LED Spot. It works fine, but I'm not able to find any easy solution to give a device a unique code to setup multiple devices. Did I missed something?
As @me-no-dev said, the setup code for commissioning (34970112332) can be used for all the devices you may need in the same Matter network. Just commission one by one and each will have its own ID in the Matter environment.
Each device will be added to the Matter Controller with an Unique IPv6 address and each Matter RGBWW will be controlled individually by the APP (smartphone) or by an Automation.
Scenes and Groups can also be created for each Matter RGBWW light.
Unfortunately this does not work with my google nest hub. As soon I try to add another device compiled with the same code, It will remove the old one, and the new one also isn't connected.
In general this approach is possible. In project Tasmota we use the development key for all devices which are setup as Matter device. It is no problem to set up more devices. So it is not an issue in general. Cant speak for this Matter implementation since the one in Tasmota is completely written from scratch (based on the espressif IDF Matter implementation) but Tasmota is written with actual Arduino Core.
I can't find a way to connect more than 1 device to my google home nest hub.
Isn't it possible to generate different test devices (with same test vendor)?
Or is there any other idea to solve this?
Unfortunately this does not work with my google nest hub. As soon I try to add another device compiled with the same code, It will remove the old one, and the new one also isn't connected.
It sounds like Google has updated their APP and their device Matter firmware in 2025.
I'd suggest trying to update the firmware of your Google Nest Hub.
The google Nest Hub is always up to date. II created a support ticket for that to google. Maybe they can solve that.
For the Arduino Matter library there isn't any chance to generate multiple different device ID's?
The google Nest Hub is always up to date. II created a support ticket for that to google. Maybe they can solve that.
I have an Alexa 5th gen here. It works fine. I have already added up to 8 endpoints using the same commissioning code, VID and PID.
For the Arduino Matter library there isn't any chance to generate multiple different device ID's?
We will consider that, for sure.
I will add this request to the feature list.
Good day. Please tell me, is it possible to create your own endpoint of a new device in Arduino? As I understand, everything is already compiled and it is possible to use only the endpoints of existing devices? If so, please add the DoorLock endpoint. Matter consumes a lot of RAM, is this a consequence of the Bluetooth library being enabled by default? Is it possible on esp32 not to use Bluetooth with Matter, but only Wi-Fi?
Good day. Please tell me, is it possible to create your own endpoint of a new device in Arduino? As I understand, everything is already compiled and it is possible to use only the endpoints of existing devices? If so, please add the DoorLock endpoint. Matter consumes a lot of RAM, is this a consequence of the Bluetooth library being enabled by default? Is it possible on esp32 not to use Bluetooth with Matter, but only Wi-Fi?
yes, the already compiled part is just ESP-Matter and CHIP code.
The Endpoints are coded in the Arduino Matter Library.
Current implementation has disable CHIPoBLE, therefore, the Bluetooth stack doesn't consume any RAM. It is already WiFi only.
We will add more endpoints along this year. DoorLock is in the list.
Thank you!
During my tests I encountered a problem when esp32 with Matter example Matter.isWiFiConnected() true and Matter.isDeviceConnected() true , the hub returns the disconnect status. How can I accurately determine that the device is working normally and the connection with the hub is established? Sometimes I have to completely reset the device and reconnect it for normal operation.
Thank you!
During my tests I encountered a problem when esp32 with Matter example Matter.isWiFiConnected() true and Matter.isDeviceConnected() true , the hub returns the disconnect status. How can I accurately determine that the device is working normally and the connection with the hub is established? Sometimes I have to completely reset the device and reconnect it for normal operation.
Matter.isWiFiConnected() returns true whenever the device is connected to WiFi network.
Matter.isDeviceCommissioned() will return true whenever the Matter device has a valid Matter Fabric and most possiby it is also connected, but this may also not be correct, once the Matter Controller may be out of reach or down.
Right now, Matter.isDeviceConnected() is the same as Matter.isWiFiConnected(), but we are working to change it and return true when the Matter stack says that the communication over it is stablished and the Controller has reached it.
We plan to release a package of improvements for Matter along this 1st Half of 2025.
This is one of the features we expect to deliver.
A gentle request: I would like to request to be able to change the number of endpoints rather than fixed 15 endpoints on all types of esp32 in the Arduino Matter library. I understand the issue of RAM on some devices like esp32C3, but we have PSRAM on esp32S3 that can support even 100 endpoints. In esp-matter it is easy to change it CONFIG_ESP_MATTER_MAX_DYNAMIC_ENDPOINT_COUNT=42.
Please consider this kind of option in the Arduino Matter Library.
Good day! If it's not a secret, tell me how to use PSRAM with Matter?
Good day! If it's not a secret, tell me how to use PSRAM with Matter?
For ESP32 Arduino, it is transparent.
When dynamic memory allocation, using malloc() or new, over 4KB is done, it is automatically allocated into PSRAM.
If the Sketch or the Matter Library "mallocs" more than 4KB, it will use PSRAM as well.
This feature comes from IDF configuration used in Arduino (sdkconfig):
CONFIG_SPIRAM_USE_MALLOC=y
CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096
Hello! Thanks for the answer!
esp32, Arduino 1.8.12, core 3.2.0.
I specially soldered the psRam PSRAM64(H) memory, it starts, is used by json buffers without errors, ESP.getFreeHeap() returns a decrease in memory of less than 70 kb; I tried to forcibly allocate memory for Matter endpoints, the memory is allocated, but when the point starts, it reboots. Kernel 3.2.0 and 3.1.2 behave the same. I do not get any benefit from psram with Matter, the console says that ESP.getPsramSize()-ESP.getFreePsram()=2180 used bytes at rest. When using the web and websocket, the PSRAM memory is used, but when the heap is reduced by ESP.getFreeHeap() less than 60 kb, the web may not load images and the page. Debug mode does not display errors on PSRAM memory. So I think that I'm doing something wrong. I'm not a professional, more of a newbie. Thanks for your help and your work!!!
psramReport.txt
Hello! Thanks for the answer! esp32, Arduino 1.8.12, core 3.2.0. I specially soldered the psRam PSRAM64(H) memory, it starts, is used by json buffers without errors, ESP.getFreeHeap() returns a decrease in memory of less than 70 kb; I tried to forcibly allocate memory for Matter endpoints, the memory is allocated, but when the point starts, it reboots. Kernel 3.2.0 and 3.1.2 behave the same. I do not get any benefit from psram with Matter, the console says that ESP.getPsramSize()-ESP.getFreePsram()=2180 used bytes at rest. When using the web and websocket, the PSRAM memory is used, but when the heap is reduced by ESP.getFreeHeap() less than 60 kb, the web may not load images and the page. Debug mode does not display errors on PSRAM memory. So I think that I'm doing something wrong. I'm not a professional, more of a newbie. Thanks for your help and your work!!! psramReport.txt
A bit hard to see what could be the issue here based on the report.
It may not be related to Matter.
It sounds like the Arduino Libraries used in this project may be allocating memory in a different way. It would require you to read its code and understand how it works. This may not be easy for a beginner.
Note that some IDF components may be using HEAP directly. It is possible for IDF to manage which memory sapce to use when allocating memory. I think that it will be better to change your project to use Arduino as IDF Component and then you will be able to tune IDF Components to use the memory the best way for your application. IDF has menuconfig oprtion for changing its settings for each component. BUT... this is not a beginner procedure.
Good luck!
Thank you!!!
I think there are two ways, wait or start the transition. The growth rate of RAM costs is very high, kernel 1.0.2 on my code 180kb of free memory, on 3.1.2 already 102kb free, and on 3.2.0 there are 84kb left and this is without launching Matter))). At this rate, PSRam will soon become simply necessary))).
From the tests I understood that everything that is more than 4096 is allocated as expected, but only to those components that began to be created after the initialization of PSRam. All dynamic variables created earlier receive memory from the heap without using PSRam.
In Arduino, unfortunately, there is no menuconfig support.
class MatterColorLight does not contain onChangeBrightness call, when using the device, the brightness control scale is there but does not work.
Thank you for your efforts!!!
Thank you!!! I think there are two ways, wait or start the transition. The growth rate of RAM costs is very high, kernel 1.0.2 on my code 180kb of free memory, on 3.1.2 already 102kb free, and on 3.2.0 there are 84kb left and this is without launching Matter))). At this rate, PSRam will soon become simply necessary))). From the tests I understood that everything that is more than 4096 is allocated as expected, but only to those components that began to be created after the initialization of PSRam. All dynamic variables created earlier receive memory from the heap without using PSRam. In Arduino, unfortunately, there is no menuconfig support.
class MatterColorLight does not contain onChangeBrightness call, when using the device, the brightness control scale is there but does not work.
Thank you for your efforts!!!
Yes, mostly, your observations are correct.
- core 1.0.2 is quite old and mostly directly poking the MCU registry. More memory left for the application. But fewer feature and a very old IDF version with some potential bugs of the past.
- core 3.1 is recent, it has more memory allocation because its strucure is way different from version 1.x. It uses newer IDF 5.
- PSRAM is not allocated within DATA and BSS (RAM Global Variables), such space has grown from Core 1.x to 3.x, therefore, there is less space left in RAM for user application. But if the application uses C
malloc()or C/CPPnewand it is over 4096, it will allocate it from PSRAM, always when it is available. - Matter Library may not use plain C
malloc(), but instead it may use CPP STD functions which, I'm not sure if it may use PSRAM.
My suggestion would be moving the project to Arduino as IDF Component in order to change sdkconfig settings for the best possible performance and memory allocation necessary. This settings can be changed using idf.py menuconfig command.
It may be possible to change Matter configuration to use PSRAM (I've not investigated it, but it may be possible).
It is also possible to change the 4096 limit to lower or higher using menuconfig.
When the application reaches the MCU peripherals limits, it may require better tools, tunning the configuration, planning the resources and even changing the development framework.
Arduino is designed for simple applications, targeting beginners that want to learn how to create MCU application using a very simple API. It has a cost in memory and processor time (lower performance) in order to make it easy to use and user mistake proof.
With the community Platformio fork pioarduino you can easily use Arduino as an component of IDF. There is a Matter IDF Arduino example here https://github.com/pioarduino/platform-espressif32/tree/main/examples/espidf-arduino-matter-light
In this setup IDF menuconfig can be used to tune sdkconfig settings for your use case.
For sure there is a learning curve ;-)
When your OS is Windows use WSL(2). Since it is a big project it is impossible to build with Windows and pioarduino! And even when it would work, compiling under Windows is horrible slow...
Once again, thank you very much!!!
I think this information will be very useful for those who write on Arduino and decided to use PsRam in their projects. Obviously, a lot of time will be saved. The right direction will be clear.
I have been writing my projects for a long time, in my free winter time, this is more of a hobby, that's why Arduino. I have already started preparing the ground for the transition, but before the transition I got stuck on optimizing the memory in the project. I want to bring everything to perfection, while I remember everything well. If I started with IDF it would be better, because I consider it a habit. In any case, my transition will not happen before next winter, when there will be enough time.
In Arduino, you also can't allocate memory in psram for a task, allocating memory leads to a reboot). So Arduino and psram are quite incompatible concepts, it is configured very strangely menuconfig.
Thank you!!!
In Arduino, you also can't allocate memory in psram for a task, allocating memory leads to a reboot). So Arduino and psram are quite incompatible concepts, it is configured very strangely menuconfig.
@ValdayMl - Please describe it. I'd like to better understand this problem.
Good day! Yes, of course.
`
StaticTask_t xTaskBuffer_loop_15s;
StackType_t *xStack_loop_15s;
xStack_loop_15s=(uint8_t*)malloc(4124);
loop_15s_in_0_core=xTaskCreateStaticPinnedToCore(loop_15s_in_0_core_f,"15 sec loop",4124,NULL,1,xStack_loop_15s,&xTaskBuffer_loop_15s,1);
SPIRAM Memory Info:
07:20:10.242 -> ------------------------------------------
07:20:10.242 -> Total Size : 4194304 B (4096.0 KB)
07:20:10.242 -> Free Bytes : 4192124 B (4093.9 KB)
07:20:10.242 -> Allocated Bytes : 0 B ( 0.0 KB)
07:20:10.242 -> Minimum Free Bytes: 4192124 B (4093.9 KB)
07:20:10.295 -> Largest Free Block: 4128756 B (4032.0 KB)
07:20:10.295 -> Bus Mode : QSPI
07:20:10.295 -> ------------------------------------------
Compile Date/Time : Apr 14 2025 07:13:20
07:20:01.279 -> Compile Host OS : windows
07:20:01.279 -> ESP-IDF Version : v5.3.2-584-g489d7a2b3a-dirty
07:20:01.279 -> Arduino Version : 3.1.3
07:20:01.333 -> Board Info:
07:20:01.333 -> ------------------------------------------
07:20:01.333 -> Arduino Board : ESP32_DEV
07:20:01.333 -> Arduino Variant : esp32
07:20:01.333 -> Arduino FQBN : esp32:esp32:esp32:JTAGAdapter=default,PSRAM=enabled,PartitionScheme=default_8MB,CPUFreq=240,FlashMode=qio,FlashFreq=80,FlashSize=16M,UploadSpeed=921600,LoopCore=1,EventsCore=1,DebugLevel=debug,EraseFlash=none,ZigbeeMode=default
07:20:01.333 -> ============ Before Setup End ============
07:20:01.534 -> 4194304 bytes
07:20:08.593 -> 93860psram used bytes
07:20:08.593 -> create my static task
07:20:08.593 ->
07:20:08.593 -> assert failed: xTaskCreateStaticPinnedToCore freertos_tasks_c_additions.h:298 (xPortcheckValidStackMem(puxStackBuffer))
07:20:08.593 ->
07:20:08.593 ->
07:20:08.593 -> Backtrace: 0x40082b9d:0x3ffb1f90 0x400905c1:0x3ffb1fb0 0x40096de6:0x3ffb1fd0 0x40093f3e:0x3ffb2100 0x400ff8ba:0x3ffb2150 0x4012b063:0x3ffb2270 0x4009133e:0x3ffb2290
07:20:08.593 ->
Decoding stack results
0x40082b9d: panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c line 463
0x400905c1: esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c line 92
0x40096de6: __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c line 80
0x40093f3e: xTaskCreateStaticPinnedToCore at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/esp_additions/freertos_tasks_c_additions.h line 299
0x400ff8ba: setup() at C:\DATA\Arduino_3\testRenew\piro_motion_vkl_rgb_minimum_rgb_esp32_ir/piro_motion_vkl_rgb_minimum_rgb_esp32_ir.ino line 415
0x4012b063: loopTask(void*) at C:\Users\VML\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.1.3\cores\esp32\main.cpp line 59
0x4009133e: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c line 139
`
`
xStack_loop_15s=(uint8_t*)heap_caps_calloc(1, 51024, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT | MALLOC_CAP_32BIT);
loop_15s_in_0_core=xTaskCreateStaticPinnedToCore(loop_15s_in_0_core_f,"15 sec loop",51024,NULL,1,xStack_loop_15s,&xTaskBuffer_loop_15s,1);
7:33:43.785 -> 93860 psram used bytes
07:33:43.785 -> create my static task
07:33:43.832 ->
07:33:43.832 -> assert failed: xTaskCreateStaticPinnedToCore freertos_tasks_c_additions.h:298 (xPortcheckValidStackMem(puxStackBuffer))
07:33:43.832 ->
07:33:43.832 ->
07:33:43.832 -> Backtrace: 0x40082b9d:0x3ffb1f90 0x400905c1:0x3ffb1fb0 0x40096de6:0x3ffb1fd0 0x40093f3e:0x3ffb2100 0x400ff8c2:0x3ffb2150 0x4012b06b:0x3ffb2270 0x4009133e:0x3ffb2290
07:33:43.832 ->
Decoding stack results
0x40082b9d: panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c line 463
0x400905c1: esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c line 92
0x40096de6: __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c line 80
0x40093f3e: xTaskCreateStaticPinnedToCore at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/esp_additions/freertos_tasks_c_additions.h line 299
0x400ff8c2: setup() at C:\DATA\Arduino_3\testRenew\piro_motion_vkl_rgb_minimum_rgb_esp32_ir/piro_motion_vkl_rgb_minimum_rgb_esp32_ir.ino line 415
0x4012b06b: loopTask(void*) at C:\Users\VML\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.1.3\cores\esp32\main.cpp line 59
0x4009133e: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c line 139
`
Core 3.2.0
Software Info:
08:20:38.850 -> ------------------------------------------
08:20:38.850 -> Compile Date/Time : Apr 14 2025 08:15:42
08:20:38.897 -> Compile Host OS : windows
08:20:38.897 -> ESP-IDF Version : v5.4.1-1-g2f7dcd862a-dirty
08:20:38.897 -> Arduino Version : 3.2.0
08:20:38.897 -> ------------------------------------------
08:20:38.897 -> Board Info:
08:20:38.897 -> ------------------------------------------
08:20:38.897 -> Arduino Board : ESP32_DEV
08:20:38.897 -> Arduino Variant : esp32
08:20:38.897 -> Arduino FQBN : esp32:esp32:esp32:JTAGAdapter=default,PSRAM=enabled,PartitionScheme=default_8MB,CPUFreq=240,FlashMode=qio,FlashFreq=80,FlashSize=16M,UploadSpeed=921600,LoopCore=1,EventsCore=1,DebugLevel=debug,EraseFlash=none,ZigbeeMode=default
08:20:38.944 -> ============ Before Setup End ============
08:20:46.170 -> 95388 psram used bytes
08:20:46.170 -> create my static task
08:20:46.170 ->
08:20:46.170 -> assert failed: xTaskCreateStaticPinnedToCore freertos_tasks_c_additions.h:299 (xPortcheckValidStackMem(puxStackBuffer))
08:20:46.170 ->
08:20:46.170 ->
08:20:46.170 -> Backtrace: 0x40082c5c:0x3ffb1f80 0x40090345:0x3ffb1fa0 0x40096b61:0x3ffb1fc0 0x40093d1b:0x3ffb2100 0x400fe65a:0x3ffb2150 0x4012a27f:0x3ffb2270 0x4009106a:0x3ffb2290
08:20:46.170 ->
Decoding stack results
0x40082c5c: panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c line 454
0x40090345: esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c line 87
0x40096b61: __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c line 80
0x40093d1b: xTaskCreateStaticPinnedToCore at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/esp_additions/freertos_tasks_c_additions.h line 300
0x400fe65a: setup() at C:\DATA\Arduino_3\testRenew\piro_motion_vkl_rgb_minimum_rgb_esp32_ir/piro_motion_vkl_rgb_minimum_rgb_esp32_ir.ino line 415
0x4012a27f: loopTask(void*) at C:\Users\VML\AppData\Local\Arduino15\packages\esp32\hardware\esp32\3.2.0\cores\esp32\main.cpp line 59
0x4009106a: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c line 139
Psram sometimes uses ESPAsyncWebServer.
Nobody uses it automatically anymore.
String.reserve(4128) also allocates memory without problems.
There is a problem with char buffer, memory has to be allocated before PSRamEsp32TrueLoad=psramInit(); otherwise I got a crash on 3.1.3.
I haven't tried it on other versions.
If you allocate memory before starting psramInit()
`xStack_loop_15s=(uint8_t*)heap_caps_calloc(1, 5*1024, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT | MALLOC_CAP_32BIT);
PSRaMEsp32TrueLoad=psramInit();`
Memory is allocated, but the same error occurs when used.
I suspect the problem lies in the menuconfig settings of the arduino.
@SuGlider what's the milestone (if already set) for Bluetooth commissionable discovery ? Thanks!
@SuGlider what's the milestone (if already set) for Bluetooth commissionable discovery ? Thanks!
This is ready for the Arduino Core 3.3.x release.
It can already be tested using release/v3.3.x branch.
https://github.com/espressif/arduino-esp32/tree/release/v3.3.x
The examples are using BLE for WiFi or Thread commissioning.
WiFi: ESP32-S3, ESP32-C3, ESP2-C6
Thread: ESP32-H2, ESP32-C5
WiFi, but no BLE Comissioning:
ESP32-S2 : this SoC has no BLE radio.
ESP32: this SoC has Bluedroid BLE for Classic Bluetooth and no NimBLE, which is necessary for Matter commissioning.
ESP32 can use NimBLE, and therefore, use Matter with BLE Commssioning, but the project shall be built using Arduino as IDF Component with the correct sdkconfig to enable NimBLE and CHIPoBLE.