topic Re: Unable to use own digital elevation model raster in Video Multiplexer tool in ArcGIS Image Analyst Questions

Unable to use own digital elevation model raster in Video Multiplexer tool

MervynLotter — Thu, 02 Sep 2021 21:53:41 GMT

Hi there

I am struggling with the Video Multiplexer tool. I am unable to use my own Digital Elevation Model as input. I have tried different formats (.tif & .crf) and projections but I always get the " ERROR 002650: Error extracting data from the provided elevation layer." error. Any suggestions as to what may be the problem?

I am able to run the VM tool using the Living Atlas Terrain layer (https://elevation.arcgis.com/arcgis/services/WorldElevation/Terrain/ImageServer). I have a question around this too. This terrain service is made up of different data products. Some very coarse. How can I ensure that the Airbus World Dem 24m is being used when running this tool with the terrain image service? Should I be using the Make Image Server Layer to query for a specific product?

Thank you.

Regards

Mervyn

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

GordonSumerling — Thu, 02 Sep 2021 23:01:03 GMT

Mervyn,

Definately an odd issue seeing how the Esri Terrain layer works. I can see two things here

1) The Esri terrain is an Image Web Service. Do you have the capacity to publish your elevation source as an Esri Elvation image source? If so try that.

2) Log this one with support. It technically should work if your elevation source is a 32bit raster. Needs to be investigated further by Esri.

Cheers

Gordon

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

MervynLotter — Sun, 05 Sep 2021 19:26:13 GMT

Hi Gorden

I dont have a an image server but I can try ArcGIS Image for ArcGIS Online. Do you think it will work? Tile or dynamic imagery?

I have just tried running the Video Multiplexer tool again using a DTM (32bit floating point) created using Drone2Map but that also failed. Pretty frustrating.

Do you perhaps have any comment on the second part of my question around ensuring that the right elevation product (Airbus in my case) gets used when using the terrain service?

Best wishes

Mervyn

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

GordonSumerling — Sun, 05 Sep 2021 22:57:11 GMT

Mervyn,

If the DTM from D2M is also failing I wouls suggest a support call.

In regards to the elevation in ArcGIS Online I would try a Hosted Elevation layer first and if that does not work then I would try an image service in ArcGIS Online. Either formats should work.

Gordon

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

NickWallingford — Tue, 07 Sep 2021 20:47:49 GMT

The video multiplexer is meant to be able to use custom DEMs as input. Can you supply your DEM? And ideally, the metadata .csv file? One thing worth checking is that the extent of the DEM is at least as large as the extent of the metadata.

The multiplexer expects to be able to get elevation data of the full extent of all the data in the metadata file. This includes the platform location, even if the target area is smaller. For instance, if the target area is a football field and a small amount of area around it, say a 200m x 200m area, but the platform is flying in a 10km radius loop around the football field, the multiplexer will expect to be able to read the data of the full 20km x 20km area that the platform is flying in. If the DEM only covers a 500m x 500m area centered on the football field, the video multiplexer tool will fail.

If the sensor ever points above the horizon, it will project a point out some distance (50km I believe) and will include that in the extent it requires.

If this extent mismatch is your problem, there are workflows to construct a DEM composed of a larger raster downloaded from Terrain3D with your DEM overlaid on top of it, but I'm afraid this is outside the area of expertise.

For the other issue, the Video Multiplexer does not allow the user to specify which data product from the Terrain 3D service is used. You would need to use another tool to extract just that data product and save it as a DEM. But unfortunately, again, this is outside my area of expertise.

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

MervynLotter — Wed, 08 Sep 2021 11:38:46 GMT

Hi Nick

Thank you for the feedback. As my study area is 25 km away from the edge of my DEM, and I was flying 20 m above the ground, I really did not think that this could be the issue .... but I think it is! I did try using a coarser 90m SRTM dem that covers a much larger area and then the tool successfully finished.

Could the tool perhaps include better messaging going forward to warn others that the extent of the dem is limited?

I never angled the camera up at the horizon and it would be nice to be able to set thresholds when running the tool to limit footprint extent, like not to calculate footprint beyond 2 km.

I found this interesting Earth Curvature Calculator (https://www.omnicalculator.com/physics/earth-curvature) where you can calculate distance to horizon based on height above ground. At 20 m it is 15km so I should really not be getting an error like this. Not sure if this kind of information can be incorporated into the tool? But perhaps setting a maximum distance may be easier.

Thank you for the feedback. I am happy to share my files with you if needed. Just please provide me with a site to upload them to.

Best wishes

Mervyn

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

NickWallingford — Thu, 09 Sep 2021 16:56:01 GMT

We managed to squeak a better error message for this situation into 2.9, just under the wire. So keep an eye out for that.

Determining what the "right" way to determine how much terrain to index is something I've struggled with a lot. There are always edge cases that seem to crop up. It's true that if you're 20m above ground level, the curvature of the Earth would indicate the horizon is 15km away. But it's also true that at Esri's main campus in Redlands, if you're standing at ground level, you can see Mt San Jacinto 55km east of here, and you can see Mt San Antonio 50km to the northwest. We need to expect to be able to put a telescopic lens on our camera and draw a footprint on those mountains. Similarly, we should expect to be able to fly a drone 20m above ground level on the tops of those mountains, point the camera down a few degrees from the horizon, and draw a footprint in Redlands. So we have this situation where the algorithm needs to know what terrain the camera is looking at before it can calculate what terrain the camera is looking at.

The algorithm we're using now is fairly robust in terms of if there's terrain where the camera is looking, the terrain will be indexed and the correct footprints will be calculated. Unfortunately this comes at the expense of handling smallish DEMs less than optimally, as you've seen.

I think for 3.0 we can enable the processing extent environment variable for the video multiplexer. And then load and index terrain from the DEM for that extent only if it's set, but by default use the current logic. But unfortunately that will have to wait for 3.0; we're already in feature freeze for 2.9. Stay tuned.

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

MervynLotter — Thu, 09 Sep 2021 20:14:01 GMT

Hi Nick

I can certainly see where you are coming from, and thank you for taking the time to reply. The error message fix for 2.9, and extent improvements for 3, will be very useful.

I look forward to testing them one day soon.

Best wishes

Mervyn

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

Xabierr — Mon, 29 Nov 2021 00:24:55 GMT

Hi Nick,

I managed to use a larger DEM and seems to be working ok. However, Im still having a few issues:

- When I export a frame (default ntf format), the image has no coordinate system defined but when I export frames to a folder as a jpg, the images are correctly located.

- Video is located and oriented correctly but scaling (footprint) seems a bit off even though the ground is flat and I used images with fixed positions based on a Phantom 4 RTK. Assuming scaling errors are always expected, right? Would the errors be systematic? Any way to improve/adjust the scaling?

And just double checking... is Sensor True Altitude the height above ground or the height above MSL/geoid? Or it doesn't matter as long as the DEM is referenced to the same vertical datum?

cheers,

Javier

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

CodyBenkelman — Mon, 29 Nov 2021 16:56:10 GMT

Javier

I just tested with Pro 2.8.1, and the *.ntf format images have a coordinate system defined. Are you querying properties of the NTF layer, or the image inside the NTF container?

re: scaling, when you say the scaling seems "a bit" off, can you be more specific/quantitative? It's hard to advise without more information. If you have RTK, I presume your altitude is accurate but other parameters (orientation angles and field of view) impact the accuracy of the extracted image size and location.

Sensor True Altitude refers to height above MSL/geoid. http://esriurl.com/FMVFAQ

Cody B.

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

NickWallingford — Mon, 29 Nov 2021 17:27:10 GMT

`Sensor True Altitude` is height above the MSL/geoid. `Sensor Ellipsoid Height` is height above the WGS84 ellipsoid. We do not have a mechanism to input height above ground.

If the scaling is off, that could indicate a few things. The first things to check are that the altitude is correct and that the field of view is correct.

The Phantom 3 and 4 we use for testing tend to have systematic altitude errors; for any given flight, the altitude reported by the drone is off by a fixed amount for the whole flight. It's so bad we're not necessarily able to tell if the altitude it's reporting is HAE or MSL -- we take the ground elevation of the takeoff point and the reported altitude when takeoff occurred, and apply the difference between the known ground elevation and the reported takeoff altitude, and apply that difference to the reported altitude for the flight. This seems to work well enough.

Unfortunately I don't have a good workflow for determining what the field of view is. I do know that the FoV that works best for our Phantom 4 is not the FoV reported by the Phantom 4 spec sheet. Which is not ideal.

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

Xabierr — Tue, 30 Nov 2021 01:15:37 GMT

Thanks @CodyBenkelman The marker in the image attached is 42x30 cm but measures ~126x80 cm on the map... so about 3x larger. The NTF image comes up with an Unknown Coordinate System.

Thanks @NickWallingford. Altitude should be spot on as Im using the Phantom 4 RTK (~3 cm vertical). It is above ellipsoid but Ive converted to above MSL.

Ive zipped a photo at ground elevation, the video, metadata (from Site Scan LE app), field mapping and DEM: https://www.dropbox.com/s/b6q8omwmqw60mk7/Test_FMV.zip?dl=0

Im using 2.9 but had the same issue with 2.8.

cheers,

Javier

Re: Unable to use own digital elevation model raster in Video Multiplexer tool

Anonymous User — Tue, 25 Jan 2022 00:40:09 GMT

I'm late to this thread, but I've been wrestling with this issue a lot as well. The ESRI Terrain service is great for convenience and quality - but that convenience goes away when the results are unreliable.

Perhaps the error messaging could be a little more explicit? Sometimes the Error 002650 makes it hard to diagnose.

I think it'd be great to have the option to clamp FOV depth to the extent of the available highest quality DEM (in the case of the Terrain service), or the user's DEM. This would make the FOV depth cartographically nice, but no longer accurate. For us that'd be useful when horizon is inevitable in a drone video capture. You'd get the benefit that close up relevant features still fall within the FOV, but avoid the 50km depth that the horizon perspective induces. An arbitrary user-entered depth limitation could be useful as well.

Glad this is under active development! Can't wait to see what 3.0 brings.