dbus-opendtu

Attention: ⚠️Please Update your Installation to latest stable version log file handling⚠️

dbus-opendtu

Introduction

This project integrates (supported) Hoymiles Inverter into Victron Energy's (Venus OS) ecosystem. It works upon openDTU respectively AhoyDTU and also with other generic REST Devices (template configuration needed to include them).

Installation

With the scripts in this repo, it should be easy possible to install, uninstall, restart a service that connects the OpenDTU or Ahoy to the VenusOS and GX devices from Victron.

Get the code

Just grap a copy of the main branch and copy the content to /data/ e.g. /data/dbus-opendtu. After that call the `install.sh script.

The following commands should do everything for you:

wget -O main.zip https://github.com/henne49/dbus-opendtu/archive/refs/heads/main.zip
unzip main.zip "dbus-opendtu-main/*" -d /data
mv /data/dbus-opendtu-main /data/dbus-opendtu
chmod a+x /data/dbus-opendtu/*.sh

⚠️Edit the following configuration file according to your needs before proceeding⚠️ see Configuration for details.

cp /data/dbus-opendtu/config.example /data/dbus-opendtu/config.ini
nano /data/dbus-opendtu/config.ini

Tha last step is to install the service and remove the downloaded files:

/data/dbus-opendtu/install.sh
/data/dbus-opendtu/restart.sh
rm main.zip

Check configuration after that - because the service is already installed and running. In case of wrong connection data (host, username, pwd) you will spam the log-file! Also, check to set a proper (minimal) log level

Update the code

Just grap a copy of the main branch and copy the content to /data/ e.g. /data/dbus-opendtu. The process will preserve your existing config ini. After that call the `install.sh script.

cp /data/dbus-opendtu/config.ini /data/dbus-opendtu/config.backup
wget -O main.zip https://github.com/henne49/dbus-opendtu/archive/refs/heads/main.zip
unzip main.zip "dbus-opendtu-main/*" -d /data
cp -R /data/dbus-opendtu-main/* /data/dbus-opendtu
rm -rf /data/dbus-opendtu-main/
chmod a+x /data/dbus-opendtu/*.sh
/data/dbus-opendtu/uninstall.sh
/data/dbus-opendtu/install.sh
/data/dbus-opendtu/restart.sh
rm main.zip

The last 4 step is to install the service and remove the downloaded files:

If the script does not work or start, please check the config.example file and update your config.ini. Or reconfigure the config.example with your configuration and save as config.ini. The process also creates a copy of your old config.ini called config.backup.

Configuration

Within the project there is a file /data/dbus-opendtu/config.ini. Most important is the DTU variant, Host and Username and Password, if you use authentication.

Default options

Config value	Explanation
SignOfLifeLog	Time in minutes how often a status is added to the log-file `current` with log-level INFO
NumberOfTemplates	Number ob Template Inverter to query
DTU	Which DTU to be used ahoy, opendtu or template REST devices Valid options: opendtu, ahoy, template. template is template only, ahoy and opendtu can use a dtu and templates together.
NumberOfInvertersToQuery	Number of Inverters to query. Set a value larger than "0" when not all inverters should be considered. *1
useYieldDay	send YieldDay instead of YieldTotal. Set this to 1 to prevent VRM from adding the total value to the history on one day. E.g. if you don't start using the inverter at 0.
ESP8266PollingIntervall	For ESP8266 reduce polling intervall to reduce load, default 10000ms
Logging	Valid options for log level: CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET, to keep logfile small use ERROR or CRITICAL
MaxAgeTsLastSuccess	Maximum accepted age of ts_last_success in Ahoy status message. If ts_last_success is older than this number of seconds, values are not used. Set this to < 0 to disable this check.
DryRun	Set this to a value different to "0" to prevent values from being sent. Use this for debugging or experiments.
Host	IP or hostname of ahoy or OpenDTU API/web-interface
HTTPTimeout	Timeout when doing the HTTP request to the DTU or template. Default: 2.5 sec
Username	use if authentication required, leave empty if no authentication needed
Password	use if authentication required, leave empty if no authentication needed

*1: Please assure that the order is correct in the DTU, we can only extract the first one in a row.

Inverter options

This applies to each INVERTER[X] section. X is the number of Inverter starting with 0. So the first inverter is INVERTER0, the second INVERTER1 and so on.

Config value	Explanation
Phase	which Phase L1, L2, L3 to show; use 3P for three-phase-inverters *1
DeviceInstance	Unique ID identifying the OpenDTU in Venus OS
AcPosition	Position shown in Remote Console (0=AC input 1; 1=AC output; 2=AC input 2) *2
Servicename	e.g. com.victronenergy.pvinverter see Service names

*1: Use 3P to split power equally over three phases (use this for Hoymiles three-phase micro-inverters as they report total power only, not seperated by phase). *2 Important for proper visualization in Venus OS / VRM

Template options

This applies to each TEMPLATE[X] section. X is the number of Template starting with 0. So the first template is TEMPLATE0, the second TEMPLATE1 and so on.

Config value	Explanation
Host	IP or hostname of Template API/web-interface
Username	use if authentication required, leave empty if no authentication needed
Password	use if authentication required, leave empty if no authentication needed
DigestAuth	TRUE if authentication is required using Digest Auth, as for Shelly Plus Devices, False if you Basic Auth to be used
CUST_SN	Serialnumber to register device in VenusOS
CUST_API_PATH	Location of REST API Path for JSON to be used
CUST_POLLING	Polling interval in ms for Device
CUST_Total	Path in JSON *4 where to find total Energy
CUST_Total_Mult	Multiplier to convert W per minute for example in kWh
CUST_Total_Default	[optional] Default value if no value is found in JSON
CUST_Power	Path in JSON *4 where to find actual Power
CUST_Power_Mult	Multiplier to convert W in negative or positive
CUST_Power_Default	[optional] Default value if no value is found in JSON
CUST_Voltage	Path in JSON *4 where to find actual Voltage
CUST_Voltage_Default	[optional] Default value if no value is found in JSON
CUST_Current	Path in JSON *4 where to find actual Current
CUST_Current_Default	[optional] Default value if no value is found in JSON
CUST_DCVoltage	Path in JSON 4 where to find actual DC Voltage (e.g. Batterie voltage) 2
CUST_DCVoltage_Default	[optional] Default value if no value is found in JSON
Phase	which Phase L1, L2, L3 to show; use 3P for three-phase-inverters *3
DeviceInstance	Unique ID identifying the OpenDTU in Venus OS
AcPosition	Position shown in Remote Console (0=AC input 1; 1=AC output; 2=AC input 2 please do not use)
Name	Name to be shown in VenusOS, use a descriptive name
Servicename	e.g. com.victronenergy.pvinverter see Service names

*2: is only used if Servicename is com.victronenergy.inverter

*3: Use 3P to split power equally over three phases (use this for Hoymiles three-phase micro-inverters as they report total power only, not seperated by phase).

*4: Path in JSON: use keywords and array index numbers separated by /. Example (compare tasmota_shelly_2pm.json): StatusSNS/ENERGY/Current/0 fetches dictionary (map) entry StatusSNS containting an entry ENERGY containing an entry Current containing an array where the first element (index 0) is taken.

Service names

The following servicenames are supported:

com.victronenergy.pvinverter
com.victronenergy.inverter (non-PV - see below)
others might work but are not tested or undocumented yet

Note: Non-PV inverters are BETA! The functionality will be limited (due to limited understanding of Victrons/Venus's behavior).

The difference between the two is that the first one (com.victronenergy.pvinverter) is used as a PV inverter connected to PV and the grid (like a Fronius or SMA inverter). The second one (com.victronenergy.inverter) is used for a battery inverter like a Victron AC inverter and is - from Victron's view - not connected to the grid. For more Information about non-PV inverters, see this Issue #42. Also, please note the use case about non-PV inverters below.

It is possible that other servicenames are supported, but they have not been tested by us. If you have a device with a different servicename, please open an issue. Any help or research is welcome and appreciated.

Videos how to install

Here are some videos on how to install and use the script. They are in German, but you can use subtitles and auto-translate to your language. (Don't be confused that the config they used is not the up-to-date.)

https://youtu.be/PpjCz33pGkk Meine Energiewende
https://youtu.be/UNuIOa72eP4 Schatten PV

Use Cases

In this section we describe some use cases and how to configure the script for them.

Use case 1: Use a PV inverter

In order to use a PV inverter, you need to know the IP address of the DTU (in my case Ahoy) and the servicename of the PV-Inverter. The servicename is com.victronenergy.pvinverter.

A Basic configuration could look like this:

[DEFAULT]
# Which DTU to be used ahoy, opendtu, template
DTU=ahoy

#Possible Options for Log Level: CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET
#To keep current log small use ERROR or CRITICAL
Logging=ERROR

#IP of Device to query <-- THIS IS THE IP OF THE DTU
Host=192.168.1.74

### Ahoy Inverter
# AcPosition 0=AC input 1; 1=AC output; 2=AC output 2
# 1st inverter
[INVERTER0]
Phase=L1
DeviceInstance=34
AcPosition=0

The result will be that the first inverter is shown in the Remote Console of Venus OS.

Use case 2: Use a battery inverter

NOTE: BETA - Victron never intended to use a Non-PV inverter (besides Multiplus, Quattro, etc.) to be connected to the existing grid directly (Grid synchronization).

In order to use a battery inverter, you need to know the IP address of the DTU (in my case Ahoy) and the servicename of the battery inverter. The servicename is com.victronenergy.inverter.

The term battery inverter is used for a device that is connected to the grid and can discharge a battery. This is different from a PV inverter, which is only connected to PV-Modules and feeds in energy.

You might want to use a battery inverter to use a battery to store energy from an MPPT charger / AC charger etc. and use it later.

A Basic configuration could look like this:

[DEFAULT]
# Which DTU to be used ahoy, opendtu, template
DTU=ahoy

#Possible Options for Log Level: CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET
#To keep current log small use ERROR or CRITICAL
Logging=ERROR

#IP of Device to query <-- THIS IS THE IP OF THE DTU
Host=192.168.1.74

### Ahoy Inverter
# AcPosition 0=AC input 1; 1=AC output; 2=AC output 2
# 1st inverter
[INVERTER0]
Phase=L1
DeviceInstance=34
AcPosition=0
Servicename=com.victronenergy.inverter

The Result looks like this:

Usage

These are some useful commands which help to use the script or to debug.

Check if the script is running

svstat /service/dbus-opendtu show if the service (our script) is running. If the number of seconds shown is low, it is probably restarting and you should look into /var/log/dbus-opendtu/current.

How to debug

dbus-spy show all DBus values interactively.

This is useful to check if the script is running and sending values to Venus OS.

How to install

/data/dbus-opendtu/install.sh installs the service persistently (see above).

This also activates the service, so you don't need to run svcadm enable /service/dbus-opendtu manually.

How to restart

/data/dbus-opendtu/restart.sh restarts the service - e.g. after a config.ini change.

How to uninstall

/data/dbus-opendtu/uninstall.sh stops the service and prevents it from being restarted (e.g. after a reboot).

If you want to remove the service completely, you can do so by running rm -rf /data/dbus-opendtu.

How does it work

The script is inspired by @fabian-lauer dbus-shelly-3em-smartmeter implementation. So what is the script doing:

Running as a service
Connecting to DBus of the Venus OS com.victronenergy.pvinverter.http_{DeviceInstanceID_from_config}
After successful DBus connection, OpenDTU (resp. Ahoy) is accessed via REST-API - simply the /status (resp. api/live) is called which returns a JSON with all details.
- A sample JSON file from OpenDTU can be found here.
- A sample JSON file from Ahoy can be found here
Serial/devicename is taken from the response as device serial
Paths are added to the DBus with default value 0 - including some settings like name etc.
After that, a "loop" is started which pulls OpenDTU/AhoyDTU data every 5s (configurable) from the REST-API and updates the values in the DBus, for ESP 8266 based ahoy systems we even pull data only every 10seconds.

Thats it 😄

Pictures

Tested Devices

The code allows you to query either an Ahoy or OpenDTU Device, plus multiple template based (PV) Inverter in a single script.

Following combinations are possible:

Use one or more devices configured via template configuration
Use a OpenDTU device
Use a AhoyDTU device
Use either a OpenDTU or a AhoyDTU device and one or more template devices.

Tested examples for template devices:

Tasmota unauthenticated
Shelly 1 PM authenticated/unauthenticated
Shelly Plus 1 PM unathenticated

All configuration is done via config.ini. Examples are commented in config.ini

Frequently asked questions

Frequently asked questions:

Can I use multiple instances? - YES, but limit the usage to a minimal level and check the stability of your system for free space on your device and load. Also remeber, that templates can be queried together with a single dtu, no need for another instance.
Do I need multiple instances to query a DTU and templates? - NO, templates can be queried together with a single dtu, no need for another instance. But limit the usage of templates and check the stability of your system.

Troubleshooting

Please open a new issue on github, only here we can work on your problem in a structured way: https://github.com/henne49/dbus-opendtu/issues/new/choose

⚠️ Change the Logging Parameter under DEFAULT in /data/dbus-opendtu/config.ini to Logging = DEBUG, please revert back to ERROR or CRITICAL, once debugging and troubleshooting is complete.

Rerun the script and share the log file current.log use the following command to generate human readable timestamps. We require this to start any troubleshooting.

cat /var/log/dbus-opendtu/current | tai64nlocal > current.log

Please provide the config.ini and JSON file and upload to the github issues, you can download the JSON file using your browser or using a commandline like tool like curl

Type of DTU	URL
OpenDTU	http://REPLACE_WITH_YOUR_IP_OR_HOSTNAME/api/livedata/status
Ahoy	http://REPLACE_WITH_YOUR_IP_OR_HOSTNAME/api/live
Template Tasmota	http://REPLACE_WITH_YOUR_IP_OR_HOSTNAME/cm?cmnd=STATUS+8
Template Shelly 1	http://REPLACE_WITH_YOUR_IP_OR_HOSTNAME/status
Template Shelly Plus	http://REPLACE_WITH_YOUR_IP_OR_HOSTNAME/rpc/Switch.GetStatus?id=0
Template Your Own	You will know best

OpenDTU Curl example which uses jq to make the output pretty:

curl http://REPLACE_WITH_YOUR_IP_OR_HOSTNAME/api/livedata/status | jq > export.json

also describe the problem as best as you can.

Please also show, what you can see in Venus OS and VRM Portal, as the source of truth is Venus OS and not VRM.

Security settings in OpenDTU

For openDTU, you can use authentication for the web Interface, but allow access to the status page unauthenticated. For this please use the settings like below.

Inspiration

Idea is inspired on @fabian-lauer & @vikt0rm project linked below. This project is my first on GitHub and with the Victron Venus OS, so I took some ideas and approaches from the following projects - many thanks for sharing the knowledge:

henne49/dbus-opendtu