jbuehl/solaredge

Table driven data interpretation for new devices

Closed this issue · 5 comments

jbuehl commented

Originally, this project was only concerned with 2 device types within the 0500 performance data message - the 0000 optimizer and the 0010 inverter. As more and more people have begun to use it, new device types have emerged and some individuals have contributed code to the project to handle their devices. More and more device types will continue to emerge. Additionally, only a small subset of messages within the SE protocol are interpreted. It isn't practical for someone who doesn't have a particular device to do the necessary detective work of figuring out the format of the data fields within the message and what they represent and not everyone using the project is comfortable with coding in python.

This issue proposes a feature in semonitor.py that allows for table driven parsing of messages and device data within the 0500 message. This would be a complete rewrite of seData.py. Data tables would contain an entry for each message type or device type, each with a list of fields containing the length, data type, and name of the field.

Exact syntax is TBD, but an example might look something like this:

{messages:
   {0x0012, PROT_CMD_PARAMS_GET_SINGLE:
     {2, H, param},
   },
   {0x0011, PROT_RESP_PARAMS_SINGLE:
     {4, L, value},
     {2, H, param},
   },
   {0x0090, PROT_CMD_PARAMS_SET_SINGLE:
     {2, H, param},
     {4, L, value},
   },
     ...
   }
 0500data:
   {0x0000, optimizer:
     {4, L, Inverter}, 
     {4, L, Uptime},
     {4, f, Vmod},
     {4, f, Vopt}, 
     {4, f, Imod}, 
     {4, f, Eday}, 
     {4, f, Temp},
   },
   {0x0010, inverter:
     {4, L, Uptime}, 
     {4, L, Interval},
     {4, f, Temp},
     {4, f, Eday}, 
     {4, f, Eac}, 
     {4, f, Vac}, 
     {4, f, Iac},
     {4, f, freq},
     {4, x, unknown},
     {4, x, unknown},
     {4, f, Vdc},
       ...
   },
}

Note - the 0080 optimizer would not be able to be parsed by this method because of the bit shifting required. The data table format would need to accomodate such a special case.

Geoff99 commented

Hi @jbuehl,

First may I say an enormous thank you to you (and other contributors) who developed this project. I have been using it since late last year, and am enormously impressed by what it can do!

I was interested in decoding battery messages, and discovering which message / messages transmitted the import / export and consumption data that can be seen on the solaredge website. I believe I have tracked them down (batteries in seType 0x0030, and consumption etc data in seType 0x0022).

I have coded a new module, seDataDevices.py, and adapted other parts of the semonitoring suite to use it.

I began by following the general approach you have suggested above, and ended up making a number of modifications, viz :

  • the field definitions are stored in a list rather than a dictionary - because field order matters :-)
  • I ended up storing them in a class which I have called ParseDevice
    • an instance of ParseDevice is (essentially) a dictionary of fieldName : fieldValue mappings
    • a new device is defined by creating a subclass of ParseDevice with it's own list of field definitions.
    • the machinery for reading the list of definitions and parsing a message data block can easily be inherited by new devices
  • I found complexities like bit shifting were too hard to generalise in a table format, so instead I added a hook function to the class, which subclasses can override to specify their own codeDerivations.
  • I have used a couple of tricks (explained in the README.ParseDevice.md) to insulate the remainder of code for the semonitor project from the details of any new subclasses that may be added into seDataDevices.py - new subclasses are simply recognised dynamically as they appear by ParseDevice itself.

I have used this approach to develop parser subclasses (ParseDevice_0x0022 and ParseDevice_0x0030) for the two new message types mentioned above.

I have also written a (rather long) explanation / tutorial about the approach I have taken in README.ParseDevice.md

Would you be interested and have the time to review a pull request containing these proposed enhancements? I warn in advance that my "style" (or lack thereof) tends towards verbose, and isn't as nicely concise and elegant as the code you have already written. I won't be at all offended if you decide it is not suitable for inclusion.

Regards

Geoff

  • 👍1
jbuehl commented

This sounds good. I don't see anything different in your fork of this project. Where can I see it?

  • 🎉1
Geoff99 commented

Thanks @justinhuntgc for answering @jbuehl 's question. I should have thought to mention that in my original comment. I didn't want to presume to submit a pull request (which I have just done #36 seDataDevices) until I heard back. I'm a bit of a novice wrt the mechanics and protocols of GitHub (I've used it a few times now, but not frequently enough to be confident that I'm doing everything the right way).

It's the (very early) hours of the morning here, so I'll sign off again for a few hours. Let me know if you have any problems finding my code.

jbuehl commented

I merged @Geoff99's changes into the project. I think it addresses this issue very well, but I'll leave it open for now so people can comment.