Incorrect video frame footprint for FMV video

[MervynLotter]

Hi there

Through the use of the Cirruas app, I have been able to collect the flight log file as a CSV for use in the Multiplexer GP tool. Given that many drones do not record altitude very accurately, from the app there is an option to correct the take-off altitude before exporting the log file. I do this so my elevation values should be correct. I suspect it determines the tack-off altitude from an image service.

In order to create the video frame footprint (frame outline), one needs to include elevation values for the ground surface (or height of drone above ground). In ArcMap one only has the option of specifying the flight altitude while for Pro one can also use an imagery service. I have tried both methods and I still have the same problem. I hope someone can advise me how to fix it?

The problem I have is that my video frame footprints are not being accurately located in GIS. In fact they almost never overlap with the actual footprint. The calculated video footprint is always displaying a much larger area to the front of the drone. 

I tried looking at this in a scene view and the multiplexed video correctly shows a 'leader-line' or video image center line that is centered in the actual video footprint (not the calculated one).  See below image.

However for this same view, the calculated footprint is way off. The calculated video frame is far from the actual video center line - and it covers a much wider area.

Any advice as to what the cause for this error may be?  I suspect it may either be the elevation data or the way the metadata was captured. In Pro I use the Esri Terrain service as my input into the multiplexer tool.

The Cirruas app records the below metadata which should be adequate in calculating video image footprints. I include the minimum and maximum values recorded for the above flight in case anything stands out. For example, should Platform Heading Angle and Sensor Relative Azimuth Angle not be in the range of 0-360?

	Minimum	Maximum
UNIX Time Stamp
Platform Heading Angle	-178.4	176.2
Platform Pitch Angle	-19.8	22.6
Platform Roll Angle	-151.1	159.1
Sensor Latitude
Sensor Longitude
Sensor True Altitude	1487.6	1551
Sensor Horizontal Field Of View	76	76
Sensor Vertical Field Of View	48	48
Sensor Relative Azimuth Angle	-183.5	351.4
Sensor Relative Elevation Angle	-168.2	168.2
Sensor Relative Roll Angle	-43	47.1

[NickWallingford]

Mervyn,

Angles in the range from -180 to 360 are fine.

It's very strange that the image centerline is correct, but the image center ground point (the + symbol) is wrong. My initial guess would be that something is off with the elevation. MISB (and therefore we) expect elevation to be given in meters; perhaps is your elevation data recorded in feet?

Are you able to attach your data file?

-Nick

[MervynLotter]

Hi Nick

Thank you for responding. My elevation values are all in meters. Just to add that I have tried to multiplex about 4 videos for different areas, all with the same footprint error. 

I am happy to provide the csv file or even video and csv data if needed. 

The csv file can be downloaded here (was not able to figure out how else to attach CSV file):

Dropbox link

[NickWallingford]

Mervyn,

Sorry for the terribly late reply, I hadn't forgotten about you.

The good news is that this no bug longer seems to present itself in the latest build. It was fixed early in the development of 2.3 as part of a lot of rework that was done about how we calculate coordinates from the platform/sensor's parameters. The bad news is that you'll have to wait for 2.3 to be released. I'm the wrong person to talk about when 2.3 will be released, but my understanding is that it will happen sometime in January.

Regards,

-Nick

[MervynLotter]

Hi Nick

Have you by any chance had a look at the metadata file that I tried to provide to you via Dropbox? Please let me know if you have any trouble accessing it.

Regards

Mervyn

[CodyBenkelman]

Mervyn, I'm curious if you could tell us more, or share your video.  I've used ArcMap to multiplex your CSV file using a random video on this end, and the footprint looks fine to me, but you're indicating it is not in the right location.  That should be improved in Pro 2.3 (available soon) to consider the terrain, but it sounds like you have some relatively consistent errors (footprint "always too large to the front of the drone") - have you tried adjusting the FOV values and/or sensor elevation (pitch) angle?  

Cody B.  

[MervynLotter]

Hi Cody (and Nick)

Thank you for the feedback. 

I can confirm that I am now using ArcGIS pro 2.3.0-beta 1 and I can now get the video frame to more-or-less match that of the video frame :-). But not always, it depends on device used to capture log file.

I have the Cirruas app loaded on two devices; a tablet and a phone. If I (1) record a log file using the phone, then, where it used to get the display error reported above, this is now fixed in Pro 2.3 (awesome). However if I (2) record the log file with my tablet, then I always get the below error in Pro only if I insert an elevation surface to calculate the frame extent. If I remove any elevation surface in the tool, then the tool will multiplex the video although it will have no frame. 

If I try and run the video multiplexer in ArcMap, and set the tool to Calculate Corner Coordinates using an average ground height, then it is able to do so using teh log file where Pro failed. I have also upgraded my version of ArcMap and FMV extension so no more strange video frame errors being created from what I can tell.

The csv log files recorded using my phone and tablet look identical to me and I cant figure out why the one works but not the other in Pro 2.3 (and earlier versions).

As I am based in South Africa, the Cirruas app development team are having some trouble getting the regional settings sorted as the app inserts a comma instead of a full-stop as a separator between the elevation decimal values (thereby splitting my elevation value into two columns). I then always have to merge the two columns and then convert the new field to a number before trying to run the Video Multiplexer GP tool. I have to do this on log files produced on both the phone and tablet.  I am not sure whether this has any bearing on the problem.

Cody, I will email you the log files to take a look at if you dont mind.