topic Re: diversity index in Spatial Statistics Questions

diversity index

StephanKaegi — Mon, 27 Apr 2020 13:59:40 GMT

Dear all
I would like to use the shanon diversity index to calculate the diversity (number of years) of cleared areas as an indicator for shifting cultivation. I'm working with ArcGIS Pro.
I've downloaded this Diversity Tool and tried to run it. Unfortunately, it didn't work.
Can someone explain to me why it failed to work?

Would there be other alternatives to calculate the shanon diversity index in ArcGIS Pro?

Below you see the raster information of the input data.

Thanks for your support!

Stephen

Re: diversity index

MervynLotter — Mon, 27 Apr 2020 19:43:22 GMT

I have not used the Diversity Tool but I did note a problem with the output raster path file name ("K6_06.04.202\Default"). In the output path file name, you should in general not include any spaces or non-text characters, such as % or @. A hyphen may be interpreted as a minus. You have used a full stop in the path file name which could be misinterpreted by any script such as Diversity Tool.

Best to try and use a short file path with text only characters.

Re: diversity index

StephanKaegi — Wed, 29 Apr 2020 09:03:40 GMT

Thanks for your answer! I changed the output raster file name but the Diversity Tool did not work. Is there an alternative?

Re: diversity index

DanPatterson_Retired — Wed, 29 Apr 2020 09:12:42 GMT

contact the author on the link you posted would be the best bet. Perhaps there are unlisted requirements or version limitations not posted on the page.

Re: diversity index

MervynLotter — Wed, 29 Apr 2020 10:31:27 GMT

Hi Stephen

I did download and install the Diversity Tool and I did get it to successfully run in Pro 2.5. It is a nice tool.

Some advice to running this script.

Make sure that the map projection is the same as the raster projection. The first time I ran this tool it failed because of this.
Zoom in to a smaller area at first to just test the script. Set your extent in the Environments setting. If it works, then expand to your area of interest.

Good luck.

Mervyn

Re: diversity index

StephanKaegi — Fri, 01 May 2020 10:13:32 GMT

Dear Mervyn
Thank you very much for your answer.
I have taken into account your two advises for the calculation.
Now the following has happened: with a small processing extent, the tool has worked. But with the whole grid I got the message > memory error (see picture below). Do you know what this could be? (I still have over 100GB on my hard disk, but maybe it's the cash memory or the RAM?)

Thanks for your support

Stephen

Re: diversity index

StephanKaegi — Fri, 01 May 2020 10:14:57 GMT

Hi Dan

Thanks for the advice. I did write a comment but got no answer from the author. Is there any other possibility to contact the author?

Re: diversity index

DanPatterson_Retired — Fri, 01 May 2020 10:40:31 GMT

memory error is referring to ram not hard disk space

Re: diversity index

DanPatterson_Retired — Fri, 01 May 2020 10:41:55 GMT

maybe he isn't checking his mail during these times. The message you left is routed to their email

Re: diversity index

BobGerlt — Mon, 16 Aug 2021 16:50:59 GMT

Stephan

For some reason I missed your question. I suspect the problem has to do with the size of your input raster and the size of the neighborhood. The combination makes for a really large footprint in memory due to the way the numpy ndimage.generic_filter works and the fact that I am applying an in memory loop that, for practical purposes, is about Image rows X Image columns X neighborhood width X neighborhood height X size in bytes of your pixel values. In your case this is 9238 x 9271 x 167 x 167 x 2 = 4.4G... I think. This is a fair amount of RAM and depending on OS version and available memory could be the problem. Try running the tool with a neighborhood of 5x5 as a test. If this works you can start scaling up the neighborhood until it breaks. The other solution would be to reduce the resolution of the source image. These ideas only apply if the issue is caused by running out of memory. I hope this helps.