Get Started with ArcGIS Full Motion Video (FMV/Drones/UAS/UAV)

[ReneeBrandt]

The FMV team is hosting a free Esri Live Training Seminar (LTS) on Thursday, October 22nd.  If you work with videos captured by UAVs, drones, UASs, orbital sensors, vehicle-mounted or handheld cameras, or other means, attend this session to learn how you can use FMV to exploit this video inside ArcGIS.

Sign up for the session here:  Esri Training | Live Training Seminar: Get Started with ArcGIS Full Motion Video

[CodyBenkelman]

Rogelio

You did not mention GPS, but I am assuming your system is recording GPS as your UAV flies - is this correct?

The multiplexing tool will merge your video file with GPS data into a MISB-compatible structure.  If you have an orientation sensor, or you can estimate the camera orientation, the multiplexer will populate enough of the MISB metadata to place the video footprint onto your map, but note that the accuracy of that footprint will be limited by the accuracy of the GPS and orientation data.

If you are not able to estimate camera orientation, the MISB metadata wiil be incomplete, and you will not see the video footprint projected into the map - however the flight track will be available based on GPS, and this will still allow you to jump to the proper time within your video based on aircraft location.  For our users who are collecting a large number of UAV videos, this provides a very good data management capability.

Cody B.

[rogelioflores2]

Cody,

Thank you for your response. I did not mention GPS because we are not recording GPS coordinates as the UVA flies. We are using a DJI phantom 2 with a GoPro camera and none of these devices allow us to record GPS coordinates making all the data I have so far useless for FMV. I am planning to attach a GPS logger to the drone to record flight paths and export them into a csv file. I am aware that the GPS coordinates alone will make my MISB metadata incomplete but as you mentioned I can estimate the orientation of the camera. Also, I was trying to look for a better camera that would record better metadata for FMV. Do you have any suggestions about cameras?

Rogelio

[CodyBenkelman]

Rogelio

I will ask a colleague to comment on this - he has MUCH more experience with specific UAVs and cameras - but I wanted to note that for video, I believe (?) it will be uncommon to find a camera that records GPS or other metadata (unless a very expensive camera that may have full MISB support).  The Multiplexer coming with FMV 1.3 is designed to work with two separate data streams (GPS and video), so the GoPro should work fine.

The question of camera metadata is more relevant when you are recording single image frames.

If you have time to comment, I'd like to hear about your application for UAV video.  What are you observing, and how do you use the video after it has been collected?

Thanks

Cody B.

[NeilHavermale]

We have just such a tool that will allow the KLV-ization of essentially any camera with essential MISB compliant data.  We term this as CSV2MISB or XML2MISB or GEM2MISB.  It is essentially a transcoder that will downsize and trim AVI, MPG, MTS, M2TS, MOV, MP4 and WMS to compliant transport stream with KLV MISB compliant video.  GoPros, SONY, NIKON, Panasonic, FLIR, Ogpal, and essentially any Andorid image or video.  This solution relies on Par Government GV2F module.  The FMV that is creates is compatible to C2ISR and C4ISR system consuming MISB compliant FMV. Red Hen Systems 800-237-4182

[DouglasRidenour]

Rogelio,

     At the inexpensive end of the market there are not a lot of choices in cameras that produce metadata that can be used for creating a "MISB-like" metadata stream through the multiplexing option. I think well see this capability start to be available in the next couple of years but so far nothing in the hobbyist range.

   You hit the nail on the head by suggesting a GPS logger to augment your GoPro collection. I'd go one further and say look at an inexpensive GPS/Compass/IMU unit.  For instance, there are open sourced Ardupilot based flight controllers in a 35mm form factor that with a GPS can collect GPS, Roll, Pitch, Yaw, and barometric elevation in flight. The logging function is built in on these flight controllers and you can dump the flight parameters out as a GPX and text file. These are so light you could mount it to the top of a Phantom and collect the data while not actually using the flight controller in place of your integrated DJI controller and would not lose a lot of flight time.

    With a little ingenuity you could use a Ardupilot or Pixhawk flight controller that would collect data independent data to use on platforms like the DJI phantom or TBS Discovery that don't have as robust logging capability built in out of the box. In this case, I would rigidly couple the camera to the aircraft frame so parameters could be collected given the offset to the logger. With a two axis gimbal mount for your GoPro ideally you wouldn't have to account for pitch and roll but it never quite seems to work out that way in practice...at least for me.

   I'm putting together something similar based on ardupilot that will have a tightly coupled  GoPro or Mobius mount on it that will allow me to collect data and video for use on fixed wing models that don't otherwise use a logging flight controller. The goal is to have something 3D printable that can be shared to the community. When that design is complete I'll come back and post a link to it.

    DJI does have an add on data collection capability but I'm not familiar with the details and have not used it with the new multiplexer function of the FMV tool.

I hope this helps,

Doug

[rogelioflores2]

Douglas,

You gave me lots of useful information

[rogelioflores2]

Douglas,

I am currently using the DJI ground station App on a IPad and it is similar to what the ArduPilot would do in terms of creating a flight plan except for collection GPS, Roll, Pitch or Yaw. I have limited knowledge on building drones but what I get from the research I made is that Pixhawk and ArduPilot work together and I can replace the DJI ground station with the “Home Planner”. Is this correct?. Also, Since the logging function is built in I can download the text file and Use that file with FMV on ArcMAP. I see that these devices are mainly use on custom made UAV’s so my concern would be how to make it work with a DJI Phantom. I will keep looking into you suggestion and I will be back with more questions.

Thank you

Rogelio

[DouglasRidenour]

Rogelio,

     Since you already have a platform that you are comfortable with I would not recommend replacing your DJI flight controller with the ArduPilot or PIXHAWK but using it as a data logger only. A full on replacemet would change the flight characteristics and be a pretty significant project although I'm sure it can be done. Also,  I should have been clearer in that I meant the legacy APM hardware when I said ArduPilot. The older APM hardware is a bit dated and not supported by the new firmware but they would work wonderfully as a data logger using the Mission Planning software to download the log files. The PIXHAWK is a great flight controller as well but it might be a little heavy for mounting onto a Phantom as additional hardware but I think the 35mm sized APM would be work great. I have some older APM hardware that I'm going to re-purpose for fixed wing platforms but they have the load capacity to carry the APM 2.5 in the case with no problem. The only technical issue I can think of up front is that if you have roll or pitch drift in your stabilized camera  that would not be captured by the logger since it is tied to the frame and not the GoPro. On the fixed wing I am addressing that by not mounting on gimbals additionally the logger and GPS will be mounted close to the camera to minimize offset. I hope to have some test flights by next week and will run them through the new FMV 1.3 multiplex tool to see how it works. Then this advice will be more practical than theoretical.

Doug

[rogelioflores2]

Douglas,

I was doing some research and you are correct. Some people have done it but for now I will keep the DJI platform as is and add the GPS logger. As for the pitch issue I think I can figure out a fix position that would facilitate data entry into the metadata. I would like to try this approach first before moving into a drone configuration. As for the multiplex tool I do not know if it has been released yet but as soon as it gets release I will take a few test flights as well and see if I get the results I want. Please keep me posted on your project. I would like to know how much beneficial It would be to get data imagery from a fixed win UAV than from a quadcopter.

[DouglasRidenour]

Rogelio,

    I weighed a 3DR APM 2.6 with the external GPS/Compass unit. It weighs 60 grams in the case without the power module. Since it will be a logger only and not powering any servos it can be powered by an alternate method http://copter.ardupilot.com/wiki/common-powering-the-apm2/ safely. If I build a custom mount for it I'll probably un-case it and save the weight. I think a smaller 35mm micro APM would come in well under 20 grams with GPS.