ArcGIS 10.5.1 Kernel Density and Bighorn Sheep Homerange

3364
18
08-21-2018 07:29 AM
DanGreene
New Contributor III

Hello,

A kernel density routine developed for the "R" language has been popular for use by government agencies to estimate core home range for Bighorn Sheep using telemetry points as input data.  Mostly out of curiosity, I developed a routine using the most recent ArcGIS kernel density routine in Spatial Analyst.  It produces very similar results using the HREF bandwidth.  I asked the developer of the R home range routine what he thought and this was his answer:

This is his reasoning for using only the R routine and kernel density function and not using the ArcGIS python routine that I developed:

"The main issue is that the Home Range Arc tool uses a quartic kernel density function which is only an approximation to the Gaussian density function employed by the R home range estimator. ArcGIS uses it because it's fast and good enough for visualizations for every day users, but it's not a density function that you actually see in the home range literature. It's also not the density function used when we were estimating the foray frequencies and distances, which is in itself a reason not to go with it."

So, my question to you is "do you agree that ArcGIS is not capable of generating a home range polygon from telemetry points?".  My reasoning is that it is because coming up with a representative polygon is an art and not an exact science.  Sometimes the home range polygon is even hand delineated using expert knowledge.  Also, the ArcGIS kernel density function was improved with version 10.3.

When using the HREF bandwidth what I have observed is that the R-script performance is faster on small datasets of telemetry points but the ArcGIS python routine is faster on larger datasets.  The resulting polygon is very close.

Your thoughts regarding this will be appreciated.  I have the routine working with both desktop and PRO.

Thanks,

Dan

0 Kudos
18 Replies
DanGreene
New Contributor III

Hey Dan P.,

Thanks, but no need to send the sample data.  The clue you gave me was sufficient for me to get past that step and move on to the next one.

Here is the rough routine of getting Gaussian kernel density from a point layer (in this case a shape file).  Note that the searchRadius variable mentioned here was passed from another routine that calculated it using the reference bandwidth.

      #Gaussian Kernel Density from point layer  
      import arcpy  
      from scipy import stats
      import numpy as np
        
      KDEtest = "Yes"
      if KDEtest == "Yes":
         #Create paired coordinate list from point layer
         my_list1 = []
         my_list2 = []
         select_curs = arcpy.da.SearchCursor(Location + "\\\\" + "Z" + str(animal) + "pts", ["POINT_X","POINT_Y"], query)
         for row in select_curs:  #Cursor is an extract from point layer for an individual animal
            xcoord = float(row[0])
            ycoord = float(row[1])
            my_list1.append(xcoord)
            my_list2.append(ycoord)
         temp = list(zip(my_list1,my_list2))  #The zip pairs the x,y coordinates into one list


         #Check list result
         arcpy.AddMessage(temp)


         #Create Numpy array from list and check result
         my_array = np.array(temp)
         arcpy.AddMessage(my_array)


         #Gaussian Kernel Density routine
         kernel = stats.gaussian_kde(my_array, bw_method=searchRadius)


         #Next challenge is to figure out how to turn the kde kernel into a grid?

Dan

0 Kudos
DanGreene
New Contributor III

Hi All,

I worded this sentence incorrectly; "do you agree that ArcGIS is not capable of generating a home range polygon from telemetry points?"  I should have said; "do your agree that the ArcGIS 10.5.1 kernel ( quartic kernel ) should not be used for generating a bighorn sheep home range polygon from telemetry points since a true Gaussian Kernel is preferred".

And, Dan Patterson, it looks like scipy may be the solution to a true Gaussian kernel from ArcGIS Pro assuming that I can return the result as a raster registered to the real world.  My existing routine uses only arcpy and spatial analyst plus calculated Href for bandwidth (has other bandwidth options) and does not use either scipy or numpy.  It currently works with desktop versions 10.3.1 up to 10.6.  It works with earlier versions of ArcGIS PRO but has a problem with a missing OID in PRO 2.2.1.  I use several spatial analyst tools and some seem to work inconsistently with different versions of ArcGIS so I have found solutions that work for all of them (except now, PRO 2.2.1).

Dan

0 Kudos
DanPatterson_Retired
MVP Emeritus

creating the raster with coordinates is no problem

NumPyArrayToRaster—ArcPy Functions | ArcGIS Desktop 

check my blog through my profile... I have tons of numpy (scipy etc) posts

0 Kudos
DanPatterson_Retired
MVP Emeritus

Dan

sounds good

Post your solution back when you get done and mark this closed or change to a Discussion.

0 Kudos
DanGreene
New Contributor III

Hi Dan Patterson,

I am at a stopping point for now as I did not have much time to work on this as it is not a "have to" since I currently have a tool that calls an excellent and fully debugged R-script (R code written by the original developer of the model) to do the job.  But, it would be nice to be able do it in Arc to avoid an Arc plus an R install (although the R version with R script version install is not a big deal since multiple versions of R can be installed).  My first ArcGIS version that uses the PRO kde kernel does a pretty good job with the arcpy kernel (KDE) and creates a nice polygon (with the use of some additional tools), but it would be nice to be able to use the gaussian kernel.  This test routine does create a grid but it is not correct so it is only a start.  What I have learned is that the documentation for arcpy is excellent but the documentation for SCIPY is not so great (at least from my learning perspective).  So, this will have to sit for a while as I have some new work to get done thus I am not able to close this discussion with a solution.

DRAFT CODE SO FAR FOLLOWS:

def kerneldensityhrd(homerangedir, HomeRangeGDB, Points, SelBand, SeaRadUse, Cellsize, DebugThis, HrefMult):
   if SelBand == "Reference" or SelBand == "Reference2":
      # Determine bandwidth for Href (Reference)
      arcpy.AddXY_management(Points)
      xlist = []
      ylist = []
      href_curs = arcpy.da.SearchCursor(Points, ["POINT_X","POINT_Y"])
      for row in href_curs:
         pointx = row[0]
         pointy = row[1]       
         xlist.append(pointx)
         ylist.append(pointy)
      Href = Hreference(xlist,ylist,DebugThis)
      if DebugThis == "ON":
         arcpy.AddMessage("Href = " + str(Href))
      searchRadius = (Href + 2000) * HrefMult
   else: #ArcGIS and ArcGIS2
      searchRadius = SeaRadUse
   # Execute KernelDensity
   #Gaussian Kernel Density from point layer          
   KDEtest = "Yes"
   if KDEtest == "Yes":
      cellSize = int(Cellsize)
      #Create paired coordinate list from point layer
      my_list1 = []
      my_list2 = []
      select_curs = arcpy.da.SearchCursor(Points, ["POINT_X","POINT_Y"])
      for row in select_curs: 
         xcoord = float(row[0])
         ycoord = float(row[1])
         #xycoord = '[  ' + str(xcoord) + ',  ' + str(ycoord) + ']'
         my_list1.append(xcoord)
         my_list2.append(ycoord)
      #arcpy.AddMessage(my_list1)
      #arcpy.AddMessage(my_list2)
      ymin = min(my_list2)
      ymax = max(my_list2)
      xmin = min(my_list1)
      xmax = max(my_list1)
      #temp = list(zip(my_list1,my_list2))
      #my_array = np.array(temp)
      m1 = my_list1
      m2 = my_list2
      X, Y = np.mgrid[xmin:xmax:100j, ymin:ymax:100j]
      positions = np.vstack([X.ravel(), Y.ravel()])
      values = np.vstack([m1, m2])
      kernel = stats.gaussian_kde(values, bw_method=searchRadius)
      Z = np.reshape(kernel(positions).T, X.shape)
      outKernelDensity = arcpy.NumPyArrayToRaster(Z, arcpy.Point(xmin,ymin), cellSize)
      #GRAPH
      import matplotlib.pyplot as plt
      fig, ax = plt.subplots()
      ax.imshow(np.rot90(Z), cmap=plt.cm.gist_earth_r,
           extent=[xmin, xmax, ymin, ymax])
      ax.plot(m1, m2, 'k.', markersize=2)
      ax.set_xlim([xmin, xmax])
      ax.set_ylim([ymin, ymax])
      #plt.show()
      #exit(0)
   else:
DanPatterson_Retired
MVP Emeritus

Dan... I will have a look some time soon.  And yes, SciPy's documentation could use some work.

0 Kudos
DanGreene
New Contributor III

Hi Dan P.,

You could close this topic if you would like to since  I don't have time to dig into it anymore.  The ideal would be if ESRI supported the standard kernel in addition to the existing one.  Interest seems to have died down and calling an R-script does the job for now.

Thanks,

Dan

0 Kudos
DanielGreene
New Contributor III

Thanks Dan P.,  

I don't know if I will have time down the road to get back to this.  My thoughts are that my python routine using the existing ArcGIS kernel to identify a home range polygon works well and is close to the one generated via the desired kernel.  In reality it is only the scientists who object.  But in reality, there is no absolute correct home range polygon and it could just as well be delineated by a wildlife biologist.  Who is really to say what is the best polygon but I am not a kernel density expert.

Dan

DanPatterson_Retired
MVP Emeritus

So true...

0 Kudos