Tello realt time flight data and 2.0 SDK integration

[annesteenbeek]

I'm currently working on a project requiring me to read a bit more of the tello sensor data.
As i see from the code, currently the IMU data is interpolated from the log data among other things.
However, i was wondering what the time delay is between a movement and reading it in the log? Can this data be assumed to be near real time? Or is there a possibility to also get the timestamp of recording from this log?
(On a side note, in protocol the IMU data after quaternion is interpreted as vg_x, vg_y and vg_z, whereas in tellolib these values are interpreted as relative velocities N, E and D. Is there some more documentation that I could read about the log data values and their units/meaning? )

Additionally in the 1.3 SDK manual it is explained that you can receive the state over the 8890 port. This is quite interesting data. And could possibly have a shorter delay. However, I can't find any reference that it is being used in this code? Is this something that was skipped on purpose or something that I could still maybe implement and create a pull request for (this could also include integration for the 2.0 sdk and landing pad)? I can see that there is some overlapping information in the FlightData object, but I can't find a lot of documentation about the FLIGHT_MSG data structure, and what the units are for these parameters. How did you guys discover these parameters, is there some more extensive online documentation I could read?

[hanyazou]

You can find some related information at the thread on the TelloPilos web site and DatCon site.
That describes about the fields on the log stream. But I could not find something like timestamp which we can use to synchronize video and sensor data. I'm curious about delay and time stamp issue too.

Has anyone decoded the log headers/messages from the Tello?
https://tellopilots.com/threads/has-anyone-decoded-the-log-headers-messages-from-the-tello.511/page-3#post-4888

DatCon
http://www.datfile.net/DatCon/intro.html

V3 .CSV column descriptions
https://datfile.net/DatCon/fieldsV3.html

[annesteenbeek]

Thanks for the info. I'll integrate the tello state messages and compare the timings. See if I can tell you some smart things about the timings.

[annesteenbeek]

I ran into a problem using the SDK.
If you want to use the SDK state, you need to send a command packet to the drone, once you receive a ok message back, the drone starts sending out the state to port 8890 on your local pc.
However, once the command is sent, the drone no longer responds to the same type of commands it used to. It stops receiving video, and log data.
It is as if it internally switched from dev mode to sdk mode or something. I need to reboot the tello before tellopy works as expected again.

I'll do some tests, comparing the timings between logs and state messages, see if I can compare some plots between comparable flights.
The "dev" sdk is a bit harder to understand then the documented SDK, but contains quite a lot more functionallity.

Here is the test code I used.

import threading
import socket
import tellopy
from time import sleep

def handler(event, sender, data, **args):
    drone = sender
    if event is drone.EVENT_CONNECTED:
        print('connected')
        drone.start_video()
        drone.set_exposure(0)
        drone.set_video_encoder_rate(4)
    if event is drone.EVENT_LOG_DATA:
        print("received log")

def send_command(drone):
    print("Sending 'command' to tello")
    drone.sock.sendto('command'.encode('utf-8'), drone.tello_addr)

def __state_thread():
    print('started state thread')
    # Create UDP socket to receive tello state
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    port = 8890 # Tello state receive port
    sock.bind(('', port))
    sock.settimeout(1.0)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 8 * 1024)
    print('State receive buffer size = %d' %
                sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF))

    while True:
        try:
            response, ip = sock.recvfrom(1024)
            out = response.replace(';', ';\n')
            out = 'Tello State:\n' + out
            print("received state")
            # print(out)
        except socket.timeout as ex:
            print('state recv: timeout')

def test():
    drone = tellopy.Tello()
    
    th = threading.Thread(target=__state_thread)
    th.setDaemon(True)

    # drone.set_loglevel(drone.LOG_ALL)
    drone.subscribe(drone.EVENT_CONNECTED, handler)
    drone.subscribe(drone.EVENT_LOG_DATA, handler)
    
    drone.connect() # connect to drone
    sleep(5) # should receive video data cleanly
    send_command(drone) # send 'command' to tello
    # should have received 'ok' packet before state can be received
    sleep(0.1)
    th.start() # once ok is received, we can listen to state messages on 8890
    sleep(2) # for some reason, after 'command' was sent, it no longer receives video
    drone.quit()

if __name__ == '__main__':
    test()

[samlaf]

@annesteenbeek Did you find a solution for this?
I just raised #95 without realizing you had raised this issue already.

[annesteenbeek]

Ok, this was a while back, and i've since moved away from this project. But what i remember was indeed that when you start using the official commands, the low level (I think this is the undocumented, reverse engineerd, comm that is used for the official app) stops working.
So in order to get the mission pads to work with the current state of tellopy, you would have to do the same kind of reverse engineering on the edu version, to find the bytes to decode the message stream.
I think there was quite a lot of information on this on the tellopilots.com forum. (But i just noticed you already posted there, so no need to link :P)

I don't think it's that difficult, you can clearly see there is additional information sent over the comm port when using the edu version, these bytes are just not translated into any data used by TelloPy.

For me this turned out to be too time consuming, as the tello was just a small part of the entire project.
Best of luck.

[samlaf]

Thanks man :)
Reverse engineering the mission pads sounds quite hard and time-consuming for me as well since I've never done this.
I might try to use apriltags instead...

[juanmill4]

thanks mannnn, yours replys have helped me