How do I check, with python, if a Terrain Model has pyramids or data sources?

5742
15
05-26-2015 04:54 AM
JohnLay
Occasional Contributor

Hello all,

This question has gotten wildly off topic, so I edited the heading to reflect more clearly what I'm trying to ask.

I'm hoping someone can point me in the right direction.

I've written a script (still in development) that creates terrains from LiDAR. They are HUGE terrains, so the process from going from LAS to Multipoint can take up to 30 hours. I've hit a hick-up that I'm trying to pinpoint that occurs sometime after the multipoint creation, but before the building of the terrain.

The code I've written works fine process by process, so I'm not sure what the issue is--it may have been a network thing. I'm trying to troubleshoot it. The problem is the time it takes to run from the beginning until it breaks.

I've added if not arcpy.Exists statements for the elements that I can, but I'm stuck on how to find the properties for the terrain so that I can skip those steps if they've alread been done. For instance, if the terrain already has pyramids defined how do I check for that? Or if / what layers have already been added to the terrain?

I've spent the last 1.5 hours doing a Google search and I've got nothing except for arcpy.mapping: "There are a few specialized layers and datasets that don't fall into one of these three categories: annotation subclasses, dimension features, network datasets, terrain datasets, topology datasets, and so on." but no way to handle these specialized layers.

Suggestions?

Thanks for any help you may provide,

John

Message was edited by: John Lay

15 Replies
LukeWebb
Occasional Contributor III

Are you able to try storing it inside an SQL Server, according to this help I gather that if so, for any terrain dataset which has pyramids, a table should be made with the name:

DTM_<ID>_MRFC

The ID is specific to each terrain dataset.

Can you see if you have these tables, then you just need to find the table that has the terrain dataset names and IDs in to associate the two. Try the query:

select * from GDB_EXTENSIONDATASETS

ArcGIS Server for the Microsoft .NET Framework 9.3 Help

These tables wont be visible in ArcCatalog, you will have to connect using SQL Server client. You may need to be an admin as they may be owned by SDE not your user.

Seperately, this link may also be of use: ESRI File Geodatabase (FileGDB)

If you have OGR setup as I do, this command worked to retrieve the features using SQL from my table 'a':

ogrinfo -dialect FileGDB -sql "select * from a" "pathto\Geodatabase.gdb"

JohnLay
Occasional Contributor

No, unfortunately they are being stored in file geodatabases so that they can be distributed.

The ESRI File Geodatabase (FileGDB) driver looks interesting, however. I'm off to figure out how it works. Thanks!

0 Kudos
ToddBlanchette
Occasional Contributor II

This made for a pretty interesting project this morning.  I think I finally may be on to something lol.

  • The following python code invokes the GDAL/OGR methods (click here if you don't have these installed).
  • Researching File GDB schema breakdown and with Luke Webb​'s confirmation above, we know we're looking for the system table called "TerrainDataset_1_MRFC" - or realistically "*MRFC", which tells us that the terrain dataset has pyramids.
  • Using this search parameter, I loop through all the objects in the File GDB and find this table, I then extract its name and append it to a list.
  • You can now use the names in this list to compare to your known dataset names to see if their pyramids have been created or not (ie: if the name shows up in the list, your pyramids "SHOULD" be good, or at the very least created at one time.

** I'm not sure if this technique will return pyramid tables from other rasters.  I would set this up so that it searches specifically for the datasets you are looking for and not a blanket search on all datasets like I did. ***

# standard imports
import sys

# import OGR
from osgeo import ogr

# use OGR specific exceptions
ogr.UseExceptions()

# get the driver
driver = ogr.GetDriverByName("OpenFileGDB")

# opening the FileGDB
gdb_path = r"D:\terrain\terrain.gdb"
try:
    gdb = driver.Open(gdb_path, 0)
except Exception, e:
    print e
    sys.exit()

# list to store layers'names
featsClassList = []

# parsing layers by index
for featsClass_idx in range(gdb.GetLayerCount()):
    featsClass = gdb.GetLayerByIndex(featsClass_idx)
    terrainFCs = featsClass.GetName()
    suffix = "MRFC"
    if terrainFCs.endswith(suffix):
        featsClassList.append(terrainFCs.split('_', 1)[0])

# sorting
featsClassList.sort()

# printing
print "The following Datasets have pyramids. Catch these results and compare to your list:"
for featsClass in featsClassList:
    print featsClass

# clean close
del gdb
XanderBakker
Esri Esteemed Contributor

When trying to get properties of an object I would normally first try with arcpy.Describe(yourTerrain). Reading up on the Describe function, I cannot find anything that is specific for terrain datasets.

import arcpy
terrain = r"D:\Xander\GeoNet\Terrain\gdb\test.gdb\TerrainData\testTerrain"
desc = arcpy.Describe(terrain)

# dataset properties
print desc.datasetType
print desc.extent
print desc.spatialReference.name
print desc.ZExtent

# object
print desc.baseName
print desc.catalogPath
print desc.children
print desc.dataElementType
print desc.extension
print desc.file
print desc.name
print desc.path

I also tried to use the arcpy.GetCount_management, but this cannot be used for terrain datasets.

In case the Terrain has just been created it will have None as extent and also None spatial reference, but having a spatial reference and an extent does not mean that the terrain dataset has data.

What you could do is creating the pyramids in a Try Except statement. If a level exists, you will not be able to add it.

Same with building the terrain, if there is not data in it it will generate the error (which you can catch):

ERROR 999999: Error executing function.

The Terrain definition is incomplete or invalid.

Failed to execute (BuildTerrain).

It is something that is lacking in arcpy. There should be some option to handle this when using the Describe function on a terrain dataset.

JeffreySwain
Esri Regular Contributor

Right now John, the functionality to do that is not available, but should be available in a future release.

MarcSwartz
New Contributor III

Mr Swain,

It is now November of 2019, has the functionality been made available yet?

0 Kudos