Error: Start location not available for DJI video

I have a video recorded with DJI phantom and it has a metadata attached to it. But when I load the video this error pops up. The status shows "Not MISB"

Yes, if your video does not follow the stanag show this message in qgis

Can you please elaborate on the what you meant by "stanag". And also how to solve this issue?

MISB standards require specific fields from your DJI Flight Log Data.

For Android Phones: Go into your files folder: Ex: Pixel 2 XL\ dji.go.v4 \ FlightRecord...

Then you will need to exctract a ,csv file using CsvView

Once you've done this you can pull the following columns.

Custom.updateTime - Column A
CUSTOM.isVideo - Column C
OSD.latitude - Column - M
OSD.longitude - Column - N
OSD.altitude - Column P
OSD.yaw - Column V
GIMBAL.pitch - Column BR

These will then meed to be inserted into a table containing these headers.

Column A = Unix Time Stamp (See below)
Column B = Platform Heading Angle (OSD.yaw)
Column C = Platform Pitch Angle (0 if camera is Gimbal’d)
Column D = Platform Roll Angle (0 if camera is Gimbal’d)
Column E = Sensor Latitude (OSD.latitude)
Column F = Sensor Longitude (OSD.longitude)
Column G = Sensor True Altitude (OSD.altitude)
Column H = Sensor Horizontal Field of View ( Per your camera - Phantom 3 Pro = 81)
Column I = Sensor Vertical Field of View ( Per your camera - Phantom 3 Pro = 66)
Column J = Sensor Relative Azimuth Angle (0)
Column K = Sensor Relative Elevation Angle (GIMBAL. pitch)
Column L = Sensor Relative Roll Angle (0)

From here you can merge (multiplex) your video and flight information. The only tool I've found so far to Multiplex is ESRI's.

Once this is done you can export a MISB Compliant video.

There's actually a lot more to it than even what I described, but if your path did not look something like this you do not have a MISB compliant video.

@Eandelin Thanks for the detailed answer. I was able to make a MISB compliant video by using your instructions but I am still getting the same error. I believe there is something wrong with my data, so can you please have a look at the data and tell me if you find something wrong in it.

Also if you have a csv file that worked for you can you share it with me for reference.

meta_3.xlsx

Thanks in Advance.

@Kuppharish QGisFmv uses ffmpeg to extract the video metadata.

The plugin launches the following command:

#extract 1 second of klv metadata from the data-re stream, and return the content in the console output stream (data -)

ffmpeg -i videoPath -ss 00:00:00 -to 00:00:01 -map data-re -preset ultrafast -f data -

It assumes the metadata stream name is "data-re". Can you check that in your video? What comes out from this command for your video if you run that in a command shell?

You can alternatively post a small video chunk if possible and I'll get a look.

@ltbam This is the output I'm getting

And you can get the video file in this link:

https://drive.google.com/file/d/1e5IZQ9_gVUObpTmbwHzslMFSZknBk8Te/view?usp=sharing

When I run the same command I get an empty output:
ffmpeg.exe -i "out.mp4" -ss 00:00:00 -to 00:00:01 -map data-re -preset ultrafast -f data -
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

I don't think the klvdata are encoded correctly in this mp4 file.

If you look to this example:
https://community.esri.com/docs/DOC-8527-cheyennevahospitalmpeg4
and run the same command, you should see binary klv metadata in the console:
ffmpeg.exe -i "CheyenneVAhospital.mpeg4" -ss 00:00:00 -to 00:00:01 -map data-re -preset ultrafast -f data -
C#��ESRI_Metadata_Collect��N97826��°n��Ü�Ý�
�:ä�ñ��ÁÇ¶í��1Ã��ï�� Ì��═�DE��±&ff�� 7╗�� :àbç��Á|F¼��#ü�
.......
size= 1kB time=00:00:00.83 bitrate= 12.4kbits/s speed=79.4x
video:0kB audio:0kB subtitle:0kB other streams:1kB global headers:0kB muxing overhead: 0.000000%

so the klvdata are not properly encoded in your out.mp4 file, unfortunately I never created such data from DJI so I can't help much more here.

@Eandelin Thanks for the detailed answer. I was able to make a MISB compliant video by using your instructions but I am still getting the same error. I believe there is something wrong with my data, so can you please have a look at the data and tell me if you find something wrong in it.

Also if you have a csv file that worked for you can you share it with me for reference.

meta_3.xlsx

Thanks in Advance.

Looking at your .xlsx file, Check your header. Even a misplaced blank space will cause ESRI's multiplexer to fail.

You should have created a CSV for import. Make sure there is no scientific notation in the UNIX TimeStamp Column.

Fimally, your video sample does not suggest this is a good video capture for MISB. Start with a straight down (NADIR) video capture lasting approximately 1 minute and maintain and constant speed. Your sample is of level flight in a small area with the camera facing forward. ESRI's Multiplexing tool will not deal with video that has the horizon in it. The footprint it creates would theoretically go to infinity.

Heres a presentation I recently did on MISB Video capture and processing. Take a look.

https://docs.google.com/presentation/d/e/2PACX-1vTKcb4AV71yapX2hrOCIUCCvdP0FIOUqO1OvfEG4cHKvo0wvVM9pmIA0vMuzLXVANmhySRlFOgTAHGf/pub?start=true&loop=false&delayms=10000

@Eandelin
I noticed that your work is is done using ArcGis. But when we have to work on Qgis it is asking for more information like frame center latitude, frame center longitude, target width, slant range. Problem is this information is not being provided by DJI. So is there any solution for this?

Also I noticed that Unix Time Stamp should be given in milli seconds which was wrongly mentioned as micro in official MISB STD 0601.

The Multiplexing (merging of Video and Flight Record data) is done in ESRI's Arc Catalog using their FMV toolkit. I don't know of any other way to Multiplex. Please share if you do. As part of ESRI's Multiplexing, it Calculates for you the corners position(s) of your image frame. Once your video has been multiplexed, you can then run it in ERSI or QGIS or any software
that can display MISB compliant video.

@Eandelin I am able to make a MISB compliant video using ESRI's multiplexer. But the problem is while ARCGIS requires just the data you have mentioned in above comment , this plugin in Qgis requires more data. You can see in the code below that to calculate the corner data i.e footprint, it is asking a lot of data not just the data you mentioned above.

QGISFMV/code/utils/QgsFmvUtils.py

Lines 770 to 824 in 3eb1fcd

    
           def CornerEstimationWithOffsets(packet): 
        
               ''' Corner estimation using Offsets ''' 
        
               try: 
        
                   OffsetLat1 = packet.GetOffsetCornerLatitudePoint1() 
        
                   OffsetLon1 = packet.GetOffsetCornerLongitudePoint1() 
        
                   OffsetLat2 = packet.GetOffsetCornerLatitudePoint2() 
        
                   OffsetLon2 = packet.GetOffsetCornerLongitudePoint2() 
        
                   OffsetLat3 = packet.GetOffsetCornerLatitudePoint3() 
        
                   OffsetLon3 = packet.GetOffsetCornerLongitudePoint3() 
        
                   OffsetLat4 = packet.GetOffsetCornerLatitudePoint4() 
        
                   OffsetLon4 = packet.GetOffsetCornerLongitudePoint4() 
        
                   frameCenterLat = packet.GetFrameCenterLatitude() 
        
                   frameCenterLon = packet.GetFrameCenterLongitude() 
        
                   # Lat,Lon 
        
                   cornerPointUL = (OffsetLat1 + frameCenterLat, 
        
                                    OffsetLon1 + frameCenterLon) 
        
                   cornerPointUR = (OffsetLat2 + frameCenterLat, 
        
                                    OffsetLon2 + frameCenterLon) 
        
                   cornerPointLR = (OffsetLat3 + frameCenterLat, 
        
                                    OffsetLon3 + frameCenterLon) 
        
                   cornerPointLL = (OffsetLat4 + frameCenterLat, 
        
                                    OffsetLon4 + frameCenterLon) 
        
                   if hasElevationModel(): 
        
                       pCornerPointUL = GetLine3DIntersectionWithDEM( 
        
                           GetSensor(), cornerPointUL) 
        
                       pCornerPointUR = GetLine3DIntersectionWithDEM( 
        
                           GetSensor(), cornerPointUR) 
        
                       pCornerPointLR = GetLine3DIntersectionWithDEM( 
        
                           GetSensor(), cornerPointLR) 
        
                       pCornerPointLL = GetLine3DIntersectionWithDEM( 
        
                           GetSensor(), cornerPointLL) 
        
                       UpdateFootPrintData(packet, 
        
                                           pCornerPointUL, pCornerPointUR, pCornerPointLR, pCornerPointLL) 
        
                       UpdateBeamsData(packet, pCornerPointUL, pCornerPointUR, 
        
                                       pCornerPointLR, pCornerPointLL) 
        
                   else: 
        
                       UpdateFootPrintData(packet, 
        
                                           cornerPointUL, cornerPointUR, cornerPointLR, cornerPointLL) 
        
                       UpdateBeamsData(packet, cornerPointUL, cornerPointUR, 
        
                                       cornerPointLR, cornerPointLL) 
        
                   SetGCPsToGeoTransform(cornerPointUL, cornerPointUR, 
        
                                         cornerPointLR, cornerPointLL, frameCenterLon, frameCenterLat) 
        
               except Exception: 
        
                   return False 
        
               return True

Here packet is the meta data that we provide and when ever a packet.Getxxxx() is used that particular column is being called.
My point here is We need ArcGis pro for multiplexing and even after multiplexing we cant use the MISB video in current plugin with DJI video because DJI doesn't give us all the data required. So as of now we can't use this plugin.

Kuppharish, It does work. The install of the plugin is a bit difficult and if you do not edit the FFMPEG path correctly in the correct folder, it will not create the footprint. Try this one attached. I know it works on my machine in both QGIS and ESRI ArcGIS. If the image footprint does not show up on yours in QGIS, you need to review the installation.

https://drive.google.com/open?id=1xbJ7bazHFfJrs6gy6o7Pe7ZX-uL5-xDy

	def CornerEstimationWithOffsets(packet):
	''' Corner estimation using Offsets '''
	try:

	OffsetLat1 = packet.GetOffsetCornerLatitudePoint1()
	OffsetLon1 = packet.GetOffsetCornerLongitudePoint1()
	OffsetLat2 = packet.GetOffsetCornerLatitudePoint2()
	OffsetLon2 = packet.GetOffsetCornerLongitudePoint2()
	OffsetLat3 = packet.GetOffsetCornerLatitudePoint3()
	OffsetLon3 = packet.GetOffsetCornerLongitudePoint3()
	OffsetLat4 = packet.GetOffsetCornerLatitudePoint4()
	OffsetLon4 = packet.GetOffsetCornerLongitudePoint4()
	frameCenterLat = packet.GetFrameCenterLatitude()
	frameCenterLon = packet.GetFrameCenterLongitude()

	# Lat,Lon
	cornerPointUL = (OffsetLat1 + frameCenterLat,
	OffsetLon1 + frameCenterLon)
	cornerPointUR = (OffsetLat2 + frameCenterLat,
	OffsetLon2 + frameCenterLon)
	cornerPointLR = (OffsetLat3 + frameCenterLat,
	OffsetLon3 + frameCenterLon)
	cornerPointLL = (OffsetLat4 + frameCenterLat,
	OffsetLon4 + frameCenterLon)

	if hasElevationModel():
	pCornerPointUL = GetLine3DIntersectionWithDEM(
	GetSensor(), cornerPointUL)
	pCornerPointUR = GetLine3DIntersectionWithDEM(
	GetSensor(), cornerPointUR)
	pCornerPointLR = GetLine3DIntersectionWithDEM(
	GetSensor(), cornerPointLR)
	pCornerPointLL = GetLine3DIntersectionWithDEM(
	GetSensor(), cornerPointLL)

	UpdateFootPrintData(packet,
	pCornerPointUL, pCornerPointUR, pCornerPointLR, pCornerPointLL)

	UpdateBeamsData(packet, pCornerPointUL, pCornerPointUR,
	pCornerPointLR, pCornerPointLL)
	else:

	UpdateFootPrintData(packet,
	cornerPointUL, cornerPointUR, cornerPointLR, cornerPointLL)

	UpdateBeamsData(packet, cornerPointUL, cornerPointUR,
	cornerPointLR, cornerPointLL)

	SetGCPsToGeoTransform(cornerPointUL, cornerPointUR,
	cornerPointLR, cornerPointLL, frameCenterLon, frameCenterLat)

	except Exception:
	return False

	return True