esphome/feature-requests

USB hub function on ESP32-S3 running ESPHOME

Opened this issue · 50 comments

ruteclrp commented

Describe the problem you have/What new integration you would like

I need an ESPHOME device to connect to a modbus slave via USB. This requires the board to have USB hub functionality. The Raspberry Pico W can do this, but requires the code to set dr_mode=host.

this is not yet implemented by in esphome for RP2040 boards.

Please describe your use case for this integration and alternatives you've tried:

ability to set dp_mode to host for the USB connection on the board

Additional context

clydebarrow commented

It needs a lot more than that. The USB device will need a (software) driver on the Pico, and that will depend on what the device is. Why a Pico? There is code for an ESP32-S3 that will operate in USB host mode driving serial ports, but again it depends on what the device is.

ruteclrp commented
randybb commented

Why are you making it hard? Just take two gpios and connect them ta rs485 transceiver.

ruteclrp commented
clydebarrow commented

RS485 wasn't mentioned in the original request so I assume it's not involved.

Can you confirm that activating the host function on an ESP32-S3 board
is available in ESPHOME?

There is currently no USB host support merged into ESPHome for any board, and it's not as simple as "activating the host function". As I said before the device needs a a driver to talk to it over USB. It seems that there is a Linux driver since it works on a Pi, but a Pico is not a Pi and doesn't run Linux.

I do have code I wrote that works as an ESPHome external component and enables an ESP32-S3 to drive a USB serial adapter but without knowing what device you are using I can't tell you if it will work. I am using it to talk modbus via a 4 port USB-RS485 VCP vendor specific adapter. It also works with USB CDC (ACM) devices.

randybb commented

modbus is a protocol working on top of the rs485 physical layer, but it is still uart. So if uart over usb host works, and you have such adapter, then we got a solution :)

ruteclrp commented
clydebarrow commented

modbus is a protocol working on top of the rs485 physical layer, but it is still uart.

Modbus is a request/reply protocol that is implemented on a wide variety of transports - RS485 is common, but it is also used over RS232, UDP, USB and other transports that have nothing to do with RS485.

ruteclrp commented

Hi Clyde. Let's not spend more time on the rs485 discussion as this is entirely related to getting the usb host function working to enable the VCP connection to the slave.
you mentioned that you have developed a "custom component" which can be used with esphome? is that something you could share?
also, presume it need to be called via some lamda scripting in the .yaml file to be included in the firmware upload to the device?

Thanks again
Robert

clydebarrow commented

What's the VID and PID of the USB device?

ruteclrp commented
nagyrobi commented

Windows: Assuming your device is connected to the computer, go to "Device manager", find your device, right click on it, select "Properties", go to "Details" tab, select "Hardware IDs" from the drop-down, and you will find an entry of a form:

HID\VID_046D&PID_C05A

which are correspondingly vendor and product IDs.

On Linux: Run the following command from your terminal:

lsusb

You will see pairs VID:PID near the names of all of the USB devices connected...

clydebarrow commented

It does work out of the box with a Rasbperry Pi 4 attached

So on Linux the lsusb command will list attached devices with their VID/PID.

E.g.

➜  ~ lsusb
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 002: ID 1a40:0101
Bus 001 Device 003: ID 067b:2303

The first two of those are hubs, device 003 is a PL2303 USB-serial adapter.

Oh I see Robi beat me to it :-)

ruteclrp commented
nagyrobi commented

@ruteclrp no image can be seen, you should probably use the Github web UI to communicate within the issue, not your e-mail client, which also apparently spams your posts with random html code.

ruteclrp commented

Sorry guys, my webmail client.

Here it is with slave disconnected:
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 002: ID 2109:3431
Bus 002 Device 002: ID 174c:1156
Bus 002 Device 001: ID 1d6b:0003

Here it is with slave connected:
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 004: ID 0483:5740
Bus 001 Device 002: ID 2109:3431
Bus 002 Device 002: ID 174c:1156
Bus 002 Device 001: ID 1d6b:0003

So the slave is:

ID 0483:5740

/Robert

nagyrobi commented

See https://the-sz.com/products/usbid/index.php?v=0x0483&p=0x5740&n=

It's just a Virtual COM Port by STMicroelectronics

ruteclrp commented

Yes, I know, it is a VCP on the modbus master display controller of a heat pump. It acts as a slave via the VCP to allow a modbus master to read its registers. That works with a direct connection to the RPI4 running my HA. However, as the modbus implementation on the controller is unconventional, I am trying to have an ESPHOME device to do the modbus master function - using the custom function procedure to access the unconventional holding registers.

/Robert

clydebarrow commented

Well, you might just be in luck - the STM32 VCP appears to be a vanilla CDC-ACM port, which is supported. Try this:

external_components:
  - source: github://clydebarrow/esphome@power-meter
    components: [modbus, usb_uart, usb_host]
    refresh: 1h

usb_uart:
  - type: STM32_VCP
    channels:
      id: usb_uart_id
      baud_rate: 9600
      buffer_size: 1024
      debug: true
      dummy_receiver: true

You can use usb_uart_id as an id wherever you need a uart component, e.g. in the modbus config. With this you should see log events when you plug the device in to the S3 USB port.

You will ideally need an S3 board that has two USB connectors, one for UART0 and one for the USB interface - use UART0 in the logger config and flash via the UART port leaving the USB port free for your device. Alternatively flash and log via WiFi.

ruteclrp commented

Thanks Clyde, much appreciated.
I am now trying to set up my .yaml file for the device, and wonder if I need to specify the GPIOs for the usb_uart?
I gather it would default to the s3 board's hardware uart for Arduino framework (USB_CDC on pin 19/20)?
I would also need to set the logger to uart0?

sorry for being a bit slow here, by I am really a newbie to ESPs and lower level fiddling with micro processors.

ruteclrp commented

Also, tried to compile and got this error:
INFO ESPHome 2024.11.3
INFO Reading configuration /config/esphome/dvi-lv12.yaml...
INFO Generating C++ source...
Traceback (most recent call last):
File "/usr/local/bin/esphome", line 8, in
sys.exit(main())
^^^^^^
File "/esphome/esphome/main.py", line 1036, in main
return run_esphome(sys.argv)
^^^^^^^^^^^^^^^^^^^^^
File "/esphome/esphome/main.py", line 1023, in run_esphome
rc = POST_CONFIG_ACTIONS[args.command](args, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/esphome/esphome/main.py", line 452, in command_compile
exit_code = write_cpp(config)
^^^^^^^^^^^^^^^^^
File "/esphome/esphome/main.py", line 212, in write_cpp
generate_cpp_contents(config)
File "/esphome/esphome/main.py", line 224, in generate_cpp_contents
CORE.flush_tasks()
File "/esphome/esphome/core/init.py", line 684, in flush_tasks
self.event_loop.flush_tasks()
File "/esphome/esphome/coroutine.py", line 246, in flush_tasks
next(task.iterator)
File "/esphome/esphome/main.py", line 204, in wrapped
await coro(conf)
File "/data/external_components/a4d2ac70/esphome/components/usb_host/init.py", line 50, in to_code
add_idf_sdkconfig_option("CONFIG_USB_HOST_CONTROL_TRANSFER_MAX_SIZE", 1024)
File "/esphome/esphome/components/esp32/init.py", line 149, in add_idf_sdkconfig_option
raise ValueError("Not an esp-idf project")
ValueError: Not an esp-idf project

ruteclrp commented

Thanks. Learning all the time.

now, it fails when compiling the usb_uart component with:

src/esphome/components/usb_uart/USBUartComponent.cpp: In member function 'void esphome::usb_uart::USBUartComponent::start_output(esphome::usb_uart::USBUartChannel*)':
src/esphome/components/usb_uart/USBUartComponent.cpp:229:66: error: missing template arguments before '(' token
uart::UARTDebug::log_hex(uart::UART_DIRECTION_TX, std::vector(data, data + len), ','); // NOLINT()
^
*** [.pioenvs/dvi-lv12/src/esphome/components/usb_uart/USBUartComponent.o] Error 1

Suspect this is related to my earlier question on setting gpio for this uart?

This is my .yaml file:

esphome:
  name: dvi-lv12
  friendly_name: DVI LV12

esp32:
  board: esp32-s3-devkitc-1
  framework:
    type: esp-idf

- # Enable logging
logger:
  hardware_uart: uart0
  level: DEBUG

- # Enable Home Assistant API
api:
  encryption:
    key: "HlgHOxf4ZUAARa3kbbGM6aEh96SWGwU0gV4xGzhn7Fk="

ota:
  - platform: esphome
    password: "ff6e09a8bf4003b3fe6e9b170b97c2e3"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

 -  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Dvi-Lv12 Fallback Hotspot"
    password: "SNt1JKssAYZb"

captive_portal:

web_server:
  port: 80
  auth:
    username: !secret web_server_username
    password: !secret web_server_password

external_components:
  - source: github://clydebarrow/esphome@power-meter
    components: [modbus, usb_uart, usb_host]
    refresh: 1h

modbus:
  uart_id: usb_uart_id
  id: dvi_modbus

usb_uart:
  - type: STM32_VCP
    channels:
      id: usb_uart_id
      baud_rate: 9600
      buffer_size: 1024
      debug: true
      dummy_receiver: true
      
- #uart:
- #  id: modbus_serial
- #  tx_pin: GPIO20
- #  rx_pin: GPIO19
- #  baud_rate: 9600
- #  stop_bits: 1
- #  parity: NONE

modbus_controller:
  modbus_id: dvi_modbus
  command_throttle: 500ms
  id: dvi
  update_interval: 30s
  address: 0x10
  
sensor:
  - id: ext_temp
    platform: modbus_controller
    modbus_controller_id: dvi
    name: "LV12 Outside temp"
    address: 0x07
    register_count: 1
    unit_of_measurement: "°C"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    device_class: temperature
    filters:
      - multiply: 0.1
clydebarrow commented

wonder if I need to specify the GPIOs for the usb_uart?

You can't specify them, they are fixed by the hardware. And don't use Arduino. Like I said above you will preferably use a board with two USB connectors, and select UART0 for logging.

ruteclrp commented

Thanks Clyde,
much better with esp-idf. Though it fails on compiling the special usb_uart component. Seems something missing in my config.

clydebarrow commented

No, that was my fault, I made a change but obviously didn't properly test it. In the external component source line, replace @power-meter with @58d8721775c7db9fc1a789d2fb125827f335cae5 - that rolls back to a previous commit.

clydebarrow commented

I just updated the repository, so the original should now work, if the external component is refreshed.

ruteclrp commented

Hi Clyde. Now got my hands on some ESP32-S3 boards. Just rather confused. I can build a simple installation with the Arduino framework which nicely connects to my wifi. When I change the framework setting to esp-idf in the .yaml file and install it nothing happens. It will not connect. Also, I cannot read the log via USB when on the idf framework. What is it that I do not understand here???

clydebarrow commented

Best place for general support for ESPHome is on Discord: https://discord.com/channels/429907082951524364/429907082955718657

ruteclrp commented

Thumbs up. Found the missing pieces to get the board running under the idf framework - now I am in business :-)
But, powering the board via the first USB port (the one also used to flash) and connecting the modbus slave to the other still gives me the no response message when firing the modbus request. Is this because I need to specify something about which uart shall be routed to which USB port?
The log says:
[10:48:37][C][logger:185]: Logger:
[10:48:37][C][logger:186]: Level: DEBUG
[10:48:37][C][logger:188]: Log Baud Rate: 115200
[10:48:37][C][logger:189]: Hardware UART: UART0
[10:48:37][C][modbus:165]: Modbus:
[10:48:37][C][modbus:167]: Send Wait Time: 250 ms
[10:48:37][C][modbus:168]: CRC Disabled: NO
[10:48:37][C][usb_host:267]: USBClient
[10:48:37][C][usb_host:268]: Vendor id 0483
[10:48:37][C][usb_host:269]: Product id 5740
[10:48:37][C][usb_uart:169]: UART Channel 0
[10:48:37][C][usb_uart:170]: Baud Rate: 9600 baud
[10:48:37][C][usb_uart:171]: Data Bits: 8
[10:48:37][C][usb_uart:172]: Parity: NONE
[10:48:37][C][usb_uart:173]: Stop bits: 0
[10:48:37][C][usb_uart:175]: Debug: true
[10:48:37][C][usb_uart:177]: Dummy receiver: true

both seems to use uart0

I would expect that the USB hub would use USBCDC

If not, I guess setting the logger to baud_rate: 0 would disable the logger and allow the coexistence on uart0?

Just not sure what you had espected to happen with the hub component I included in the build

/Robert

clydebarrow commented

Do you see anything in the log when you plug in the USB device? Is the board providing 5V to the USB port?

ruteclrp commented

The log says:

[13:56:41][C][logger:185]: Logger:
[13:56:41][C][logger:186]: Level: VERBOSE
[13:56:41][C][logger:188]: Log Baud Rate: 115200
[13:56:41][C][logger:189]: Hardware UART: USB_SERIAL_JTAG

[13:56:41][C][modbus:165]: Modbus:
[13:56:41][C][modbus:167]: Send Wait Time: 250 ms
[13:56:41][C][modbus:168]: CRC Disabled: NO
[13:56:41][C][usb_host:267]: USBClient
[13:56:41][C][usb_host:268]: Vendor id 0483
[13:56:41][C][usb_host:269]: Product id 5740
[13:56:41][C][usb_uart:169]: UART Channel 0
[13:56:41][C][usb_uart:170]: Baud Rate: 9600 baud
[13:56:41][C][usb_uart:171]: Data Bits: 8
[13:56:41][C][usb_uart:172]: Parity: NONE
[13:56:41][C][usb_uart:173]: Stop bits: 0
[13:56:41][C][usb_uart:175]: Debug: true
[13:56:41][C][usb_uart:177]: Dummy receiver: true
[13:56:41][C][modbus_controller:349]: ModbusController:
[13:56:41][C][modbus_controller:350]: Address: 0x10
[13:56:41][C][modbus_controller:351]: Max Command Retries: 4
[13:56:41][C][modbus_controller:352]: Offline Skip Updates: 0
[13:56:41][C][modbus_controller:354]: sensormap
[13:56:41][C][modbus_controller:356]: Sensor type=4 start=0x7 offset=0x0 count=1 size=2
[13:56:41][C][modbus_controller:360]: ranges
[13:56:41][C][modbus_controller:362]: Range type=4 start=0x7 count=1 skip_updates=0
[13:56:41][C][modbus_controller:365]: server registers
[13:56:41][C][modbus_controller.sensor:010]: modbus_controller.sensorModbus Controller Sensor 'LV12 Outside temp'
[13:56:41][C][modbus_controller.sensor:010]: modbus_controller.sensor Device Class: 'temperature'
[13:56:41][C][modbus_controller.sensor:010]: modbus_controller.sensor State Class: ''
[13:56:41][C][modbus_controller.sensor:010]: modbus_controller.sensor Unit of Measurement: '°C'
[13:56:41][C][modbus_controller.sensor:010]: modbus_controller.sensor Accuracy Decimals: 1

[13:56:56][V][modbus_controller:232]: Updating modbus component
[13:56:56][V][modbus_controller:198]: Range : 7 Size: 1 (4) skip: 0
[13:56:56][V][modbus_controller:043]: Sending next modbus command to device 16 register 0x07 count 1
[13:56:56][V][usb_uart:126]: Channel not initialised - write ignored
[13:56:56][V][modbus:223]: Modbus write: 10.04.00.07.00.01.83.4A (8)
[13:56:56][V][modbus_controller:568]: Command sent 4 0x7 1 send_count: 1
[13:56:56][V][modbus:042]: Stop waiting for response from 16
[13:56:57][V][modbus_controller:043]: Sending next modbus command to device 16 register 0x07 count 1
[13:56:57][V][usb_uart:126]: Channel not initialised - write ignored
[13:56:57][V][modbus:223]: Modbus write: 10.04.00.07.00.01.83.4A (8)
[13:56:57][V][modbus_controller:568]: Command sent 4 0x7 1 send_count: 2
[13:56:57][V][modbus:042]: Stop waiting for response from 16

[13:57:29][V][esp-idf:000]: E (53276) HUB: Root port reset failed

[13:57:29][W][component:237]: Component usb_host took a long time for an operation (313 ms).
[13:57:29][W][component:238]: Components should block for at most 30 ms.

Since the last attempt I removed the uart assignment for the logger to uart0. Now the logger defaults to USB_SERIAL_JTAG

Still, the USB host component seems not to work as intended.

You asked if the board provided 5V to the uab port. How would I check that easiest?

ruteclrp commented

Found an anomaly.

The stop bits were not set, and should default to 1. However the validation check of the configuration file showed that stop bits were 0.

The spec for the VCP is:
Com port settings:
9600 baud
8 bit word
1 stopbit
No parity
No hardware flow control

I have now set the stop bits to 1 in the conf, and recompiles.

The log now says:
[14:25:19][C][usb_host:267]: USBClient
[14:25:19][C][usb_host:268]: Vendor id 0483
[14:25:19][C][usb_host:269]: Product id 5740
[14:25:19][C][usb_uart:169]: UART Channel 0
[14:25:19][C][usb_uart:170]: Baud Rate: 9600 baud
[14:25:19][C][usb_uart:171]: Data Bits: 8
[14:25:19][C][usb_uart:172]: Parity: NONE
[14:25:19][C][usb_uart:173]: Stop bits: 0
[14:25:19][C][usb_uart:175]: Debug: true
[14:25:19][C][usb_uart:177]: Dummy receiver: true

stop bits seems wrong here!

But, the log continues:

[14:25:36][D][usb_host:027]: Event flags 2X
[14:25:37][W][component:237]: Component usb_host took a long time for an operation (310 ms).
[14:25:37][W][component:238]: Components should block for at most 30 ms.
[14:25:37][W][component:237]: Component usb_host took a long time for an operation (59 ms).
[14:25:37][W][component:238]: Components should block for at most 30 ms.
[14:25:37][D][usb_host:032]: New device 1
[14:25:37][D][usb_host:068]: Open device 1
[14:25:37][D][usb_host:074]: Get descriptor device 1
[14:25:37][D][usb_host:080]: Device descriptor: vid 483 pid 5740
[14:25:37][D][usb_host:089]: Device connected: Manuf: STMicroelectronics; Prod: STM32 Virtual COM Port ; Serial: 48D874673036
[14:25:44][V][modbus_controller:232]: Updating modbus component
[14:25:44][V][modbus_controller:198]: Range : 7 Size: 1 (4) skip: 0
[14:25:44][V][modbus_controller:043]: Sending next modbus command to device 16 register 0x07 count 1
[14:25:44][V][usb_uart:126]: Channel not initialised - write ignored
[14:25:44][V][modbus:223]: Modbus write: 10.04.00.07.00.01.83.4A (8)
[14:25:44][V][modbus_controller:568]: Command sent 4 0x7 1 send_count: 1
[14:25:44][V][modbus:042]: Stop waiting for response from 16
[14:25:45][V][modbus_controller:043]: Sending next modbus command to device 16 register 0x07 count 1
[14:25:45][V][usb_uart:126]: Channel not initialised - write ignored
[14:25:45][V][modbus:223]: Modbus write: 10.04.00.07.00.01.83.4A (8)

So, now the slave is recognised.

But, why is the channel not initialised?

ruteclrp commented

Just found your other support thread with #2944 (comment)

My board is the same - also have that soldering point on the back. Suspect that the issue is the missing 5 V to the USB port.

Still, the stop bits issue persist and suspect it is set in your component rather than with parameter in config? can see that when I state the "stop_bits: 1" in config it is ignored and the validation returns "stop_bits: '1'"

Anyway, will solder the usb-otg connection on the board tomorrow and see how it goes.

ruteclrp commented

Some observations.

  1. soldering the usb-ota point on the back side of the board did not generate any noticeable change. It may have caused 5 V to the USB port, but my slave is externally powered anyway, thus this change was not needed to establish the connection.
  2. I observed the same pattern that building the firmware with the esp-idf framework could not be flashed via cable as the board did not properly boot. Starting out with a minimal Arduino based firmware first and then replace with the esp-idf firmware wia ota was also for me the way to go. Did not however make the conclusion that it was the flashing method that caused it.
  3. The issue I have is not that the slave unit is not seen. When I unplug/plug the slave USB it is clearly recognised in the log.
  4. If is set the logger to hardware_uart: uart0, then it does not work. Leaving it out of the configuration defaults the port to: hardware_uart: USB_SERIAL_JTAG
  5. The issue that remain is that the modbus channel need initialising.
  6. Whether this is related to the stop_bits issue is not clear to me
    It seems that my issue and Felipecrs experience are almost identical.

Thanks for your great assistance.

Robert

ruteclrp commented

OK, had a look at you code and looks like your default stop_bits are set to 1, even when the validation says 0?

Maybe it is all because there is not a mbc_master_create_serial function being invoked?

clydebarrow commented

Maybe it is all because there is not a mbc_master_create_serial function being invoked?

No, that's part of the ESP-IDF modbus implementation and is not used by ESPHome. Stop bits aren't even used by a virtual CDC_ACM device so that's just cosmetic.

The "channel not initialised" error is odd, since the device is clearly being connected, but there should be more log entries related to it. Attach your YAML, and turn on verbose logging for usb_uart - that will dump the device descriptor.

ruteclrp commented

dvi-lv12.yaml.txt

Here is the YAML file for the board.

When you say "turn on verbose logging for usb_uart" do you mean, verbose level under logger, and debug under usb_uart?
or, is there some extra setting under usb_uart I need to set?

ruteclrp commented

logs_dvi-lv12_logs(1).txt

OK, here is the log with the descriptor after I plugged the slave in.

ruteclrp commented

So, the log states that the usb_uart sits on uart0. Should it not have been on the USB_CDC pin pair for the S3 board?

image

clydebarrow commented

the log states that the usb_uart sits on uart0.

No it doesn't, and it can't anyway - UART0 is a completely different peripheral, built into the ESP, "UART Channel 0" is what your USB device is meant to implement.

do you mean, verbose level under logger

yes. Though you already have that, but you have not set the logger uart to UART0. Uncomment the hardware_uart line and remove the baud_rate line. You will not see all the logs over WiFi, you need to capture via UART0. Once it's working that's not so important, but you're not there yet.

And here is the problem:

[21:01:09][E][component:164]: Component usb_host set Error flag: No CDC-ACM device found

The STM is evidently not implementing the CDC-ACM spec. The verbose logging will dump the descriptors which will shed some light on it once you get the logging working properly.

ruteclrp commented

Clive, you have to help me here with something which may be obvious, just not to me.
I have been trying to get the log output to my connected laptop via the usb (com). This is the one which defaults to the JTAG if I do not specify uart0 in the logger section.
However, how do I get the uart0 routed to my laptop?

clydebarrow commented

You either need a board with two USB connectors, one for the built-in USB and one for UART0 via an on-board interface chip, or you need a separate USB-serial adapter to wire to your board's UART pins.

clydebarrow commented

I just updated the external component so that the USB config dump will work over WiFi, so that might make things easier.

  • 👍1
ruteclrp commented

You either need a board with two USB connectors, one for the built-in USB and one for UART0 via an on-board interface chip, or you need a separate USB-serial adapter to wire to your board's UART pins.

My board is a 2 port board. However, connecting to either of those ports when the board is running does not allow access the to log. Somehow uart0 is not routed the the usb-uart bridge. Not specifying the hardware_uart routes the logger to JTAG which is available.
Was thinking that I needed a manual usb-serial adapter to connect to the uart0 pins - which seems is required for this board.

Anyway, will try to dump the log via wifi now you have made the update to the component.

Reverting.

ruteclrp commented

logs_dvi-lv12_logs(2).txt

Here is the extended log of the usb connection. I repeated plugging in the device a few times. I hope this makes sense to you.

clydebarrow commented

I just pushed a commit that should make it more flexible about identifying devices that resemble CDC-ACM but don't actually match the spec, which is what your device looks like.

ruteclrp commented

Hi Clyde

Happy to report that we are in business. Had to remove the dummy receiver but expect that is as intended.
Thanks for your great support and patience with my ignorance.

clydebarrow commented

Awesome!