Problem creating an OIC with Insta360 x3 videos

[cdanielvarons]

Hi Imagery and Remote Sensing community, I would like to create an Oriented Imagery Catalog (OIC) in ArcGIS Pro using videos recorded by an insta 360 x3 camera. I have a problem, when I connect the camera with a GPS to obtain the coordinates, these coordinates are stored in a .gpx file independent of the .mp4 file of the video. I would like to know if anyone knows how I can associate each coordinate with each frame of the video and how I can create the OIC with this video.

[GIS-Cambria]

It will take some work. I've not specifically worked with the X3 and GPS files but I've done work with phone videos and gpx and I've done work with drone videos and gpx. Buried in esri tools is a fantastic opportunity to use Full Motion Video (FMV). Unfortunately this toolset, when using ArcGIS Pro, will require you to have the Image Analyst extension to enable full functionality. In ArcMap the toolset was a separate extension that wasn't dependent on Image Analyst. Yes this is a big gripe I have with how the licensing now restricts the full FMV integration. Anyhow, what you will want to do is find a method that will match your GPX timestamps with your video duration. FMV was great because it has scripts that allow you to sync the files (video multiplexer) and make the video fully integrated with the camera's FOV and location. The output file from FMV scripts is a specifically formatted video file that is enhanced with location, camera parameters, and timestamps thus making a fully integrated geospatial video. You might be able to accomplish your work with OIC, I cannot confirm success with that. However, you might also be able to use other resources outside of esri and then bring the data into your GIS environment. I have an X2 and the GPS device so I'm willing to give the OIC a try. TBH though I would love to get all of the FMV tools out of the clutches of the Image Analyst extension licensing. (ahem, esri)

[CodyBenkelman]

I can help you with this.  FMV in ArcGIS Pro is not suitable for 360 videos - the FMV tools assume the video field of view is on the ground, and with a 360 camera the footprint extends to the horizon in all directions. 

So cdanielvarons is on the right track to use Oriented Imagery but I agree with GIS-Cambria that "...it will take some work...".

The starting point is to use this GP tool https://pro.arcgis.com/en/pro-app/latest/tool-reference/conversion/features-to-gpx.htm to generate a point feature class, then you'll have to extract the required fields from the attribute table and edit to create a *.csv with the required metadata fields.  You'll be missing some of the required fields (e.g. horizontalFOV = 360, verticalFOV = TBD, UnixTimeStamp in MICROseconds, more) but you can insert nominal values.  If you need a description and example of the video metadata, send me email at cbenkelman@esri.com.  Note the video and metadata are asynchronous so you don't need metadata for every second or every video frame - as long as your metadata and video start at the same time, our tools can work with the video.  

[CodyBenkelman]

Apologies to anyone that was confused by the above post - I offered the wrong GP tool (as GIS-Cambria observed).  I meant to post the GPX To Features tool.  See my additional comments below

[GIS-Cambria]

If you already have the GPX file then you don't need a conversion from feature to create a new GPX file. You will need the GPX To Features tool. From there you can export the table to a csv format to match points to timestamps and prepare your table with the required fields as @CodyBenkelman states. Another approach could be to use a folder of images. An option to use OIC would be to find a tool that will let you extract images from your video (ffmpeg is one) and set the output interval based on frame rate or time; thinking 1 second is default to match GPX log/timestamps. IIRC the Insta360 Desktop editor will let you extract images to a folder. I don't know if extracting 360 images from the video will inherit the native 360 headers to be recognized as 360 photo. Anyhow, these two steps would then allow you to use OIC tools with the .csv file, and the extracted images folder to build your OIC. According to this link video is supported. Hopefully the video process or the extracted images workflow help you in esri and OIC. Mapillary cou... and then you can use a companion widget with a Mapillary account.

[CodyBenkelman]

Thanks for catching one of my errors.  I discussed this with a colleague, and realized that my post above from earlier today offered poor advice.  Assuming I'd referenced the correct tool (GPX To Features tool), you COULD follow what I offered but it's more complicated than necessary.  Assuming you have Microsoft Excel, I had forgotten that Excel can automatically read and reformat the GPX file, so that is a much simpler first step to get the timestamp + lat/long/elevation (etc.) metadata into a table format.  You will still need to add additional fields - email me if you need documentation on the content of the video metadata table. 

However it's not necessary to extract frames from the video as GIS-Cambria suggested.  That is another alternative, but you'd then be limited to viewing the extracted frames, not the video.   The "Oriented Imagery Classic" tools at https://www.arcgis.com/home/item.html?id=36ee0bbedca64a5a8b68d7c69ab51728 directly support video today.

At risk of further confusion, note that Oriented Imagery is now being integrated as a native capability in ArcGIS Pro - read this blog to learn more -  but today (Pro v3.2) video is not yet supported in the integrated version.  Video support should be added in 2024 (but watch our blogs for confirmation of when this will be available).