Select to view content in your preferred language

Error calculating raster percentiles by polygon boundaries

1206
7
Jump to solution
05-07-2019 04:23 PM
FrancesBiles1
Occasional Contributor

Xander Bakker‌#

Hello,

I've been trying the last 2 days to successfully run the python code kindly developed by Xander Bakker here.  I used the script further down in that thread posted by 'Extractor' dated Jan 25th, 2019.  I am using an elevation raster that has been converted to an integer type. The polygon layer is a feature class in a file geodatabase. Both the raster and feature class are in the same projected coordinate system.

The code runs great for awhile, then I get the error posted in the following screenshot:

Any ideas what is causing the error?

The only thought I had is that the polygon layer is watershed boundaries. Some of the polygons are nested within larger polygons. Could this be the issue? The code successfully processed 2 polygons where one was a subset of the other (polygons 3 and 4). Polygon 12 finished successfully - 12 is a small watershed nested inside polygon 13. The code failed on polygon 13. When viewing "ras13" it looks correct; however, no attribute table was created. I verified that "ras13" is an integer raster. Perhaps the behavior of crashing on nested polygons is unpredictable, or perhaps it is not the cause at all? 

The only other thing that makes this polygon area different from the others is it is the largest of the set.

For "ras13":

Any help is appreciated,

Frances

0 Kudos
1 Solution

Accepted Solutions
XanderBakker
Esri Esteemed Contributor

Hi Frances Biles ,

I downloaded the data and could replicate the error. First thing I did was loading the TIFF into the FileGeodatabase and using a separate File Geodatabase for the intermediate products (this normally give better results and working with the IN MEMORY workspace has some limitations with size). Still no luck. 

Looking at the data, I notice two large polygons. To avoid limitations by size I separated the two larger polygons from the other smaller polygons and ran the process again using the smaller polygons. It still generated problems in the process. Looking at the temporal products, my ras13 was the last correctly processes raster, and the process halted at number 14. When I loaded the raster I noticed that the raster had no attribute table (hence the error you have seen).

What I also noticed is the large range of values for the DEM comparing ras13 and ras14. So, I used the raster calculator and divided the DEM by 10 explicitely converting the result to integer and tried again and this time it worked for both the small and the large rasters:

Please give it a try and let me know if this worked for you.

View solution in original post

7 Replies
FrancesBiles1
Occasional Contributor

UPDATE:  I removed all nested polygons to test whether the nested polys were causing the error, then re-ran the script. Got the same error on the same watershed (because some polygons were deleted, the error occurred on "ras10."  So I deleted that particular watershed from the feature class and re-ran. Got the same error. This time on "ras12."

0 Kudos
XanderBakker
Esri Esteemed Contributor

Hi fbiles_usfs ,

Sorry to hear that you are having problems with the script. Is there any way you can share the data? It will be very hard to discover what is going on without access to the data.

0 Kudos
FrancesBiles1
Occasional Contributor

Hi Xander,

The files are just over the Geonet allowed limit, but here is a link<https://new.cloudvault.usda.gov/index.php/s/BNXE2eaZ7kcdJdd> that will be active for 3 days to a zipfile of the data.

Please let me know if you have any issues with the download and I will try again.

Contained in the file is the integer DEM, a file gdb containing the polygon feature class, and the python script I was using.

Obviously you will need to change the file path references.

Also, I tried both running in memory and specifying a scratch gdb, but got the same error.

Thanks very much for taking a look!

-Frances

0 Kudos
XanderBakker
Esri Esteemed Contributor

Hi fbiles_usfs ,

Thanks for sharing. I am downloading the files at this moment and I hope to have a bit of time to look at it tomorrow. 

0 Kudos
XanderBakker
Esri Esteemed Contributor

Hi Frances Biles ,

I downloaded the data and could replicate the error. First thing I did was loading the TIFF into the FileGeodatabase and using a separate File Geodatabase for the intermediate products (this normally give better results and working with the IN MEMORY workspace has some limitations with size). Still no luck. 

Looking at the data, I notice two large polygons. To avoid limitations by size I separated the two larger polygons from the other smaller polygons and ran the process again using the smaller polygons. It still generated problems in the process. Looking at the temporal products, my ras13 was the last correctly processes raster, and the process halted at number 14. When I loaded the raster I noticed that the raster had no attribute table (hence the error you have seen).

What I also noticed is the large range of values for the DEM comparing ras13 and ras14. So, I used the raster calculator and divided the DEM by 10 explicitely converting the result to integer and tried again and this time it worked for both the small and the large rasters:

Please give it a try and let me know if this worked for you.

FrancesBiles1
Occasional Contributor

Xander – you’re brilliant!

Dividing the raster by 10 to reduce the range of the numbers did the trick.

Today I ran the script again using the “real” input dataset, which was much larger than the test dataset I was working on.

It failed when I tried to run it in memory, but I pointed to a scratch gdb on the second try and the script completed successfully.

Thanks for taking the time to help and the quick response. I know I will use this useful tool again.

Best regards,

Frances

XanderBakker
Esri Esteemed Contributor

You're welcome.  Glad it is working for you!

0 Kudos