Drone2Map Full Motion Video Integration

[Anonymous User]

Does Drone2Map support any automated integration with Full Motion Video tools?

MISB compliant video seems like a natural fit for photogrammetry workflows.

[CodyBenkelman · Accepted solution]

Sorry for the delay - I had not seen your question.  Short answer is no, Drone2Map does not support video. 

But if you are proposing to extract frames from video and then process those video frames in Drone2Map, there are workflows to support this via ArcGIS Pro. 

I've tested it, but my opinion/advice is that the two different modes (single images vs. video) have different advantages and disadvantages, and they can be very complementary for some projects, but video is generally a poor input for Drone2Map for the following reasons:

1)  Single frames extracted from video have lower quality than if you were to shoot single frames --> therefore you CAN build orthos or 3D products (with limitations noted below) but the quality suffers.

2)  For processing in Drone2Map you need overlap both along flightlines and also between flightlines - but we typically shoot video to capture long linear targets (pipelines, powerlines, rivers, coastlines) with a single pass.  Single lines don't work well in Drone2Map, and if you're going to fly parallel lines to capture a site, I'd recommend shooting single images, not video.

3) It's also common to shoot at an oblique angle when capturing video - and unless doing multiple orbits around a building and avoiding aiming above the horizon, the resulting images do not process well in Drone2Map (or any other photogrammetry software).  

4) Last, when you say "Full Motion Video" (FMV) that's a capability in ArcGIS Pro that requires specific metadata to work properly.  The free iPad app "Site Scan Limited Edition" will capture this metadata, but if you use any other flight app, you won't have the complete metadata in the proper format to support FMV (see blog).  

So - my advice is to use video when appropriate, but if you want to process through Drone2Map, shoot high resolution single images.

Cody B. 

p.s. If you disagree and feel this would have great value, add your suggestions under Drone2Map IDEAS https://community.esri.com/t5/arcgis-drone2map-ideas/idb-p/arcgis-drone2map-ideas 

[Anonymous User]

Thank you for this thorough comment - really appreciate your insight on this.

I don't think it's worthy of a new Idea, but I'll add some additional perspective for posterity.

I think ESRI FMV will more or less always be a niche product. It's built on military standards and few commercial/consumer small UAS products support it natively. For the platforms that do expose sufficient telemetry for the multiplexor, it is difficult to build the ETL pipeline (especially to automate and scale). ESRI will naturally focus Site Scan compatibility on popular drones, and it's unrealistic to think there will ever be coverage for every drone. It's also unrealistic to think that mature drone programs will readily swap from Mission Planner, UGCS, etc to Site Scan just for this capability.

The groups who need this capability are probably buying purpose built hardware and software solutions and not rolling their own.

That being said, there are some strong use cases:

1. Historical drone video. It's not hard to think of organizations with a large quantity of archived drone videos. Once you build your own ETL pipeline for ESRI FMV, you can exploit your archive to create new 3D models, ortho imagery, etc. There are a lot of possibilities for change over time, new visualization products, etc. This may also reduce the need to collect new imagery if a task can be done with ortho created from old data.

2. Enable non-GIS or users to tag features in the video. I can sit any domain expert in front of a video and ask them to tag features of interest. The point geometry from FMV is then a useful overlay for the ortho imagery derived from the same video. Oblique or semi-oblique views are far more valuable than high quality nadir for this task.

3. Logistically, small drones with high quality video cameras are much, much smaller and easier to deploy and operate than platforms with larger payloads. Sub centimeter GSD is great. I love being able to resolve blades of grass and leaves in treetops. But if I have to ship tons of gear and multiple people to accomplish that it becomes less exciting. Why not have 1 operator and a drone that fits in overhead carry-on?

There are plenty of drawbacks (and I agree with them). However I think there's a lot of potential also for stronger D2M/FMV integration. As the biggest FMV game in town ESRI is uniquely positioned to make this happen. 

[CodyBenkelman]

I re-read my notes above and realize I failed to mention our Oriented Imagery capability which also supports video.  This is not photogrammetric processing - so it won't create orthos, point clouds, meshes etc. - but re: your comments above about "...a large quantity of archived drone videos..." and "... tag features in the video..." (especially oblique orientation), our growing Oriented Imagery capability provides an excellent foundation for these sorts of use cases.  

See https://doc.arcgis.com/en/imagery/workflows/resources/managing-and-visualizing-oriented-imagery.htm for an overview (with resource links) and an example in a web map here https://oi1.img.arcgis.com/app/index.html?oic=f339778a895948ca923d780b7f9017e6 

[Anonymous User]

This is next on the docket for test/eval. Excited for the possibilities here! Less excited that FMV/OIC have very different deployment options - but that's an uneducated take.