How to send messages in `rt` mode?
rec opened this issue · 32 comments
The reverse engineered documentation of the Twinkly API talks about an operating mode "rt - receive effect in real time".
This is intriguing but the API doesn't seem to have any way to actually send the data.
(If you were to point me in approximately the right direction, I could probably send you a pull request to accomplish this...)
Yes, xled library nor CLI doesn't support sending of packages in rt mode. I'd say the right direction is a documentation. There is #51 which points out that Gen II/some lights use slightly different packet structure.
All right, I'll see what I can do over the next few weeks!
I wrote most of this library though the copyright holders of the name have now suspended development on the main trunk, and it would be nearly trivial to write a BiblioPixel driver for xled.
This would have the advantage of giving you the whole reasonably large list of animations that BP has and also to do things like have a web interface or control it from MIDI - except that does tie you to the BP library which is not in active development.
At some point in future I intend to go back to the whole idea and write a new LED animation library from scratch, but that point is far away right now.
Having it work with BiblioPixel would be interesting to me as I have the BibliPixel (AllPixel) hardware as well.
Their (maniacal labs) hardware production is now at end of life (in October they announced closing down sale) but getting BiblioPIxel to drive more things may well help to get it more widely adopted.
I ended up firing simple curl commands at it from Home-Assistant to turn on/off lights but it was clear that it could do much more.
Fascinating news!
I'd have to work in a fork of BP. It would have to be sub rosa as ManiacalLabs told me that further development on the code would have to be done under a different name "for legal reasons".
I don't think they can actually prevent my fork from existing, though. 😀
We'll see how it goes. Right now, I'm actually diverted by another project (doing digital audio in numpy) but knowing even one person might use it is somewhat intriguing.
So I opened my notes to start this up, but now my theory is that I won't succeed.
Unfortunately, last time I wasn't able to get the authentication part of xled working (MacOS), so I just bypassed it. This lets me do things like select patterns, but it seems to be that sending rt messages actually requires the authentication key as part of the message.
(It is so stupid that the lights do not require authentication to e.g. turn them on and off any time I like, but do require it to set each light individually!)
In some point in the next few weeks I might go back and try to get the authentication working again, but the urgency isn't there for me...
@rec Integrating with BiblioPixel sounds great. I have not heard about it before, but I started writing my own library to control my Twinkly in a similar way, and BiblioPixel sounds like a much better starting point. Please go ahead and fork a "LibrePixel" if the name was a problem. ;-) I'd love to help with putting in xled support in.
As for the authentication problem: I suspect you might have run into #30. The simplest solution for that is to comment out line 402 and 403 in auth.py. If you bypassed more of it before, you might have skipped the parts that were actually required.
It appears to want the authentication token given during the set mode RT request, you're just continuing the session with more data.
I'm now at the point of crafting some packets to test with, but the lights freeze and reporting back being it rt mode, and then swap back after no activity like the docs suggest. So RT mode on my light would appear to be working ok, I just haven't actually sent any packets.
I've made some changes on my end to enable rt mode and a realtime option, that then returns the contact address and auth token from the current auth session. Might also want the led count and type there as well. Then you could build the UDP packet header and supply whatever client with a header and the string led type so it can build the right length of byte string.
I'll have to play a bit more with it and see where I've messed up.
Trying to send a loop of frames, light each led up, just to test things.
Was sending way too many bytes, not enough led bytes and the led descriptor count I guess was wrong too.
Testing code below , hopefully helps you move forward.
cli.py
@main.command(name="realtime", help="Turns realtime on.")
@click.pass_context
def realtime(ctx):
control_interface = common_preamble(ctx.obj.get("name"), ctx.obj.get("hostname"))
log.debug("Get device info...")
response = control_interface.get_device_info()
number_of_led = response["number_of_led"]
led_profile = response["led_profile"]
bytes_per_led = len(led_profile);
max_frame = (bytes_per_led*number_of_led)
click.echo("LED profile: {}".format(led_profile))
click.echo("Format: {}".format(number_of_led))
click.echo("Bytes per LED: {}".format(bytes_per_led))
click.echo("Max Frame Size: {}".format(max_frame))
click.echo("Max Packet Size: {}".format(max_frame+10))
log.debug("Turning realtime on...")
control_interface.realtime()
click.echo("Realtime turned on. Send packets to {}:7777".format(control_interface.host))
click.echo("Authentication Token: {} / {}".format(
control_interface._session.access_token,
control_interface._session.decoded_access_token.hex()))
click.echo("Open UDP Socket.")
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
for l in range(1, number_of_led+1):
#click.echo("LED {}".format(l))
packet = bytearray(b'\x01')
packet.extend(control_interface._session.decoded_access_token)
packet.append(bytes_per_led)
for f in range(1, number_of_led+1):
for x in range(0,bytes_per_led):
if f==l :
packet.append(255)
else:
packet.append(0)
sock.sendto( packet, (control_interface.host, 7777))
#click.echo("Length: {}".format(len(packet)))
#click.echo("{}".format(packet.hex()))
time.sleep(0.1)
control.py
def set_mode ....
....
assert mode in ("movie", "demo", "off", "rt") <-- add rt
def realtime(self):
"""
Sets realtime mode
"""
return self.set_mode("rt")
Auth.py
@property
def decoded_access_token(self):
"""Current decoded authentication token if exists. None if it wasn't fetched yet."""
return base64.b64decode(getattr(self.client, "authentication_token", None))
Isn't pretty, but it sequentially lights up led 1->250 at a 0.1s interval.
My strip is two strings of 125, so one strip lights up, and then the second strip lights up.
I wonder if something is missing in the picture here. @rec Your Wireshark recording seems to be missing the actual mode=rt request. I'm wondering if it had some additional json data attached to it?
The 0x03 at the beginning of the multipacket format seems to be a format marker, not an indication of the number of packets needed (see the example by Walderbash, which only needs two packets).
But nowhere in this format is the total number of LEDs specified. It does not seem to match the rest of the API. I believe there is a way in which the client tells the device the coming layout of the UDP frames.
I -- admittedly naively -- tried to use the multipacket format on my 210 LED device, by sending 50 + 50 + 110 LED packets. This did not work. Either my device has no support for multipacket formats (not impossible, but it would also seem strange for Twinkly to program in a prohibition for such packets), or there is some piece of the puzzle missing. I have some hacky code in the branch gif-viewer on my personal fork, if someone with a multi-packet capable Twinkly wants to try.
Or has it something to do with the result of /xled/v1/led/config?
I was curious where you all were at on this situation? More on how you possibly think rt data should be dealt with.
My igloo has melted now, so I've got the lights easier at hand to use for the rest of the year.
Just a personal project here so I've just continued hacking away but I've made some progress continuing on from above.
I took a bit of plywood from a pallet and drilled a 15x15 grid, and inserted the two light strings in the holes.
I added the ability to map my lights to a grid via a csv. So each cell visually represents my grid and the value in each cell is the led number. This creates a map.
Then I added the ability to import an image or gif and step through the frames. it will resize the image to the map size and then create packet data based on the RGBA values of the pixels.
Obviously this is a bunch of junk you don't need in the library, at the same time, you do need the tokens in the packets.
I'm running for now, but interested to hear what your ideas are for RT implementation.
I have it working on Windows.... it should work everywhere else.
The question for you and me an well mostly, Scrool, is, how do you plan to send your data/who makes the socket/packet and sends it.
The authentication is simple, the problem is mostly the packet structure.
We need to know how many leds you have to verify the packet size.
Mine (RGB) works with 0x01 as a sync byte, token , 0x03/0x04? number of bytes per led, data
01 1791e18e1db3264a 03 ... ... ... ... ... ...
I guess this is Version 1.
https://xled-docs.readthedocs.io/en/latest/protocol_details.html#movie-format
Does someone on this thread have code that works to send full
rtmode
messages I can start from?
Have you seen #67 ?
Why does this not tell me what firmware version I have??
Because it is part of different API call https://xled-docs.readthedocs.io/en/latest/rest_api.html#get-firmware-version
Sorry for all the traffic, but good news.
There's a spelling mistake in that branch as you know.
I corrected it here and astonishingly, it bworked right the first time!
import xled, random
from xled import realtime
import time
def frame():
return bytes(random.randint(0, 255) for i in range(750))
dd = xled.discover.discover()
control = xled.ControlInterface(dd.ip_address, dd.hw_address)
rtc = realtime.RealtimeChannel(control, 250, 3)
rtc.start_realtime()
BLACK = bytes(750 * [0])
FRAMES = [frame(), frame(), frame(), frame(), frame(), BLACK, BLACK]
for i in range(0, 255):
rtc.send_frame(FRAMES[i % len(FRAMES)])
time.sleep(0.05)
rtc.send_frame(BLACK)
I can easily send out a pull request for the tweaked branch and the original author will get the credit. :-)
We have now support for UDP realtime mode in the library. I guess main question from the report has been answered so I'm going to close this one.
We have now support for UDP realtime mode in the library. I guess main question from the report has been answered so I'm going to close this one.
I'm stuck in a strange scenario. I would like to use udp_client.py to send a frame to my light strand however I'm not really sure how to construct a "frame" before sending it. I've read through the documentation but I wasn't able to see an example of what a frame that should be sent to the udp_client.py class should look like? I've scoured the internet for longer than I would like to admit for an example although my searching was unfruitful :-/ . I have a large frame (600 lights) to construct, however, if I could see an example of what a 3 or 4 led strand would look like I feel I would be able to extrapolate what else I would need to do from that.
For my project I'll be receiving many data points that I will use to build the frame and I will need to push a full frame one after the next. I really appreciate your project and hope I'm not being a burden.
Sincerely,
Doc
There's a poor example above of V1 that turns each led on and off which helped me at least find string orientation.
I then converted this to read pixel values from gifs and display those.
https://xled-docs.readthedocs.io/en/latest/protocol_details.html#movie-format
Depending on version and firmware you need to send packets wgere first byte is the protocol version
1 Byte for version
0x01- 0x03
Next 8 bytes are the auth token.
1-2Bytes of spacer/syncbyte
0x00 -- 0x0000
Then you send data for each led
Frame 1 [0x01, 0x12345678, 0x00, LED1, LED2, ..... , LED250]
Frame 2 [0x01, 0x12345678, 0x00, LED1, LED2, ....., LED250]
Frame 3 [0x01, 0x12345678, 0x00, LED1, LED2, ....., LED250]
Maybe this will help, not perty but eh.
#This reads a map csv, basically a coordinate map, I have a 15x15 grid, each cell contains the led number that should be in the cells position.
with open("./xled.map") as csvfile:
reader = csv.reader(csvfile)
for row in reader:
map.append(row)
map_width = len(map[0]);
map_height = len(map);
image = Image.open(gif)
log.debug("Turning realtime on...")
control_interface.realtime()
click.echo("Realtime turned on. Send packets to {}:7777".format(control_interface.host))
click.echo("Authentication Token: {} / {}".format(
control_interface._session.access_token,
control_interface._session.decoded_access_token.hex()))
click.echo("Open UDP Socket.")
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
try:
frames = image.n_frames
except:
frames = 1
click.echo("Frames: {}".format(frames))
click.echo("File: {}".format(gif))
while frames > 0:
#for each frame in the gif
for frame in range(0,frames):
#click.echo("Seek to Frame: {}".format(frame))
#seek to the frame
image.seek(frame)
#resize to the map
#click.echo("Resize Frame: {}".format(frame))
resized_im = image.resize((map_width, map_height)).convert('RGBA')
#loop over the map
showframe = []
for y in range(0, map_height):
row = []
for x in range(0, map_width):
row.append(resized_im.getpixel((x,y)))
showframe.append(row)
#print(showframe)
#click.echo("Create packet: {}".format(frame))
#create a new packet
packet = bytearray(b'\x01')
#Add the access token
packet.extend(control_interface._session.decoded_access_token)
#Add the total leds
packet.append(bytes_per_led)
for led in range(1, number_of_led+1):
found = 0
#look for the led number in the map.
for row in range(0,map_height):
#print(row)
for col in range(0,map_width):
#click.echo("Search Row {} and Col {} - {} == {} ".format(row,col,map[row][col],led))
if( int(map[row][col]) == led ):
#print("pixel found in map")
#print(showframe[row][col])
#for v in showframe[row][col]:
if(showframe[row][col][3]>0):
packet.append(showframe[row][col][0])
packet.append(showframe[row][col][1])
packet.append(showframe[row][col][2])
else:
packet.append(0)
packet.append(0)
packet.append(0)
found = 1
if( found == 0 ):
packet.append(0)
packet.append(0)
packet.append(0)
#click.echo("Send Packet: {}".format(frame))
sock.sendto( packet, (control_interface.host, 7777))
Thanks @Emerica this is useful.
What is required at it's most basic form? I'd like to statically make a frame for example and send it to the strand using real time mode(Strand =600leds, Gen2, RGB, Layout just horizontally stretched out so not 2D). I'm learning this library but I think I'm hitting a roadblock when the frame is made programmatically as many people use it to make 2D designs or videos and I just want to assign every LED a color using RT mode and not movie mode.
Thanks again.
Try something the example above
#62 (comment)
You might have to tweak somethings, but that should turn each led on and off in sequence making a frame, turning on only a single led.
I have a 250 set that's two strings, and it just starts at 0 on one string and when it gets to the end of that string it starts on the next.