Skip navigation
All People > Dan_Patterson > Py... blog > 2018 > July
2018

The usual.... Another distance question.  This one is a little bit different.  Either the standard distance (a statistical measure in the Spatial Statistics toolset) or a distance matrix and its parameters were needed.

Why not do both without the extension or an advanced license. 

Totally supported since python and numpy and the tools to work with them are provided for you.

Note code savvy????  You can always purchase what you need.

 

Let's start with the scenario....

You can see the points that are within the polygons.  Objective! either determine the distance matrix of the points contained within each polygon AND/OR calculate the standard distance similarly.

 

: -----------------------------------------------------------------------------------------------------------------------------------------------------

In short... 

Here is the workflow.

  1. Make your imports.  Arraytools is stuff I have written (on GitHub in sortof organized form)
  2. Select your input featureclass which is created by 'Intersect' ing your points and the polygon.  That way you can bring over your attributes into the resultant point file.
  3. Make a structured array from the featureclass just bringing over the X, Y and an identifier field.
  4. Sort the result in step 3 by the identifier field... this just makes splitting it easier.
  5. Split the array into subarrays for processing.
  6. Lines 10 -19 are just for formatting the output.
  7. Line 21 and on, is the processing steps.
    1. for each subarray, get the X,Y coordinates (line 23)
    2. calculate the center of the point distribution
    3. calculate the variance for X and Y
    4. Use the above to determine the Standard Distance (1 std level, multiply by 2 or 3 for 2 std)
    5. Determine the interpoint distance matrix (line 28) returning an array with the diagonal set to zero (point to itself) and redundant (duplicate) calculations set to zero (line 29)
    6. From the nonzero values (there should be no duplicate points anyway, BTW), calculate the distance's statistical parameters.
    7. Fancy print the results

 

import arcpy
import arraytools as art

in_fc = r"C:\Path_to_Your\File_geodatabase.gdb\pnts_intersect_polygons"

a = arcpy.da.FeatureClassToNumPyArray(in_fc, ['SHAPE@X', 'SHAPE@Y', 'ID_poly'])
a_sort = np.sort(a, order='ID_poly')
a_split = art.split_array(a_sort, fld='ID_poly')

frmt = """
Group .... {}
center ... {}
standard distance ... {}
distance matrix...{}
distance results...  mean {}, std dev. {}  min  {}, max      {}
"""


# ---- let's role ----
for i in range(len(a_split)):
    a0 = a_split[i][['SHAPE@X', 'SHAPE@Y']]
    a0_xy = art.geom._view_(a0)
    cent = np.mean(a0_xy, axis=0)
    var_x = np.var(a0_xy[:, 0])
    var_y = np.var(a0_xy[:, 1])
    stand_dist = np.sqrt(var_x + var_y)
    dm = art.geom.e_dist(a0_xy, a0_xy)
    dm_result = np.tril(dm, -1)
    dm_vals = dm_result[np.nonzero(dm_result)]
    args = [i,
            cent,
            stand_dist,
            art.form_(dm_result, deci=1, prn=False),
            dm_vals.mean(),
            dm_vals.std(),
            dm_vals.min(),
            dm_vals.max()]
    print(frmt.format(*args))

 

Here is the output for the first polygon.

Group .... 0
center ... [ 304932.447  5029991.887]
standard distance ... 795.1178530213251
distance matrix...

Array... ndim: 2  shape: (24, 24)
. .     0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .   253.6    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .   179.9  214.1    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .   377.4  417.6  225.6    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .   644.3  513.3  466.6  379.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .   866.3  685.6  697.3  637.8  259.5    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .   819.0  696.3  639.5  514.0  183.1  219.6    0.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .  1289.9 1164.9 1110.1  960.7  654.6  527.5  473.0    0.0    0.0    0.0    0.0    0.0    0.0    0.0.....
. .  1431.1 1290.0 1251.8 1113.8  788.6  622.9  612.4  164.4    0.0    0.0    0.0    0.0    0.0    0.0.....
. .  1583.5 1385.9 1416.1 1340.3  965.7  718.9  836.0  567.6  446.2    0.0    0.0    0.0    0.0    0.0.....
. .  1504.4 1410.2 1326.3 1148.0  897.6  810.8  715.9  292.8  309.6  745.2    0.0    0.0    0.0    0.0.....
. .  1777.6 1566.1 1616.3 1556.9 1178.8  924.2 1061.2  801.3  669.2  237.4  953.7    0.0    0.0    0.0.....
. .  1587.4 1483.2 1408.4 1236.2  969.9  861.3  786.9  333.8  297.2  703.4  101.2  898.4    0.0    0.0.....
. .  1643.8 1489.1 1465.6 1336.7  999.5  807.5  829.1  391.9  228.1  368.6  429.3  540.1  361.5    0.0.....
. .  1825.6 1740.4 1649.0 1461.7 1228.7 1138.9 1047.5  613.0  571.6  932.6  332.2 1096.0  282.0  565.3.....
. .  2069.4 1859.9 1906.4 1838.6 1462.6 1211.4 1335.0 1018.9  866.6  499.1 1108.9  294.1 1033.9  680.2.....
. .  2218.6 2007.6 2056.0 1988.4 1612.5 1361.3 1484.1 1157.9 1002.1  648.5 1229.6  441.6 1149.2  804.7.....
. .  2004.9 1880.9 1825.1 1663.3 1371.9 1220.5 1190.2  717.4  597.6  791.3  533.6  889.5  435.4  454.5.....
. .  2202.6 2002.9 2034.8 1950.7 1580.4 1337.6 1439.5 1076.4  915.1  619.2 1116.6  455.6 1030.7  702.8.....
. .  2075.5 1916.9 1897.4 1766.0 1431.2 1232.4 1260.5  810.0  652.3  633.7  736.6  650.5  637.8  431.7.....
. .  2144.5 1974.1 1968.1 1848.3 1501.8 1288.7 1337.3  904.1  741.6  636.9  859.7  602.4  763.0  514.0.....
. .  2161.4 2055.3 1982.6 1806.5 1542.4 1413.2 1359.3  893.4  795.7 1021.5  658.6 1115.3  574.4  678.7.....
. .  2245.4 2089.5 2066.9 1931.0 1601.2 1405.5 1428.7  971.7  817.8  798.8  868.6  788.6  767.6  602.0.....
. .  2390.9 2267.3 2211.2 2046.4 1758.7 1603.1 1577.0 1104.2  981.0 1103.4  905.0 1140.0  811.6  813.9.....

I will put this in the Point Tools toolset at some point.  

If people need the code, email me at the link on my profile and I can direct you to the links on GitHub.

Make it happen

 

Making conda package installs more fun... 

 

 

 Install Spyder, Jupyter console and Jupyter notebook for ArcGIS Pro by default 

 

With ArcGIS PRO 2.2 there has been a slight change in the way packages can be installed. 

This will evolve but some pictures to get you going.

---------------------     ---------------------     ---------------------    ---------------------    

Updates....

---------------------     ---------------------     ---------------------    ---------------------    

Readings

 

Esri's Anaconda cloud repo... location of the install packages for Pro 2.2

Spyder .... a view of the IDE's capabilities

The Python Package Manager—ArcPy Get Started | ArcGIS Desktop 

 

Issues and workarounds

Mangled paths issue … package manager fails to read the _ in the _backup

comment on arcgis pro 2.2.0 and python package manager 

 

---------------------     ---------------------     ---------------------    ---------------------    

Current installation

Here is where I installed Pro 2.2 a while ago and here it still resides today.

Now some of you don't have the luxury of being able to install your software where you want, but most of this should work if you are given the slightest bit of freedom in how you want to perform your work.  You will be able to perform most of the following in your User profile though (albeit, creating ridiculously long and convoluted path lengths..

 

Onward.... find out where ArcGIS Pro is installed....

 

 

---------------------     ---------------------     ---------------------    ---------------------    

Make some shortcuts

I covered making shortcuts previously, but I will reiterate here.  I like shortcuts on the desktop and I like to dock them on the Task Bar.  First... create your shortcut

 

Conda access shortcut

 

Follow the path to the proenv.bat file located as shown in the following figure.

My installation path is C:\ArcGISPro\ You just need to change that portion in your searching and the rest will be the same.

When I located, the file, I created a shortcut and modified the contents of the shortcut as shown below.  I then copied that shortcut to my desktop.

I always keep a separate folder containing customizations like shortcuts so that I can share and/or modify as needed.

 

 

 

: ----------------------------------------------------------------------------

Installing Spyder and other packages

 

Yes... spyder is in the default package, but since the ArcGISpro-py3 environment is shut down, you have to do installs using conda..

 

If you don't have a shortcut created... just fire up the python prompt

 

 

Or fire up your new shortcut and follow along

 

 

Navigate to  C:\Your_Install_Folder\bin\Python\envs\arcgispro-py3\Scripts\spyder-script.py

and make a shortcut like...

 

Target

C:\ArcGISPro\bin\Python\envs\arcgispro-py3\pythonw.exe "C:\ArcGISPro\bin\Python\envs\arcgispro-py3\Scripts\spyder-script.py"

 

NOTE...

!!! if your default installation path has flotsam (like spaces, dashes and dots),

you may have to enclose both portions of the Target in double quotes.  !!!!!

 

 

If you want to make a development environment, you now need to clone it to install packages.

Here is the Manage Environments dialog, found on the Project page.

 

---------------------     ---------------------     ---------------------    ---------------------    

Setup Spyder as your script IDE

You get to reuse the Target shortcut used before.  

Just remember 

  • the pythonw.exe and spyder-script.py paths below need to be on one line.
  • if the python path is in a horrid space or flotsam filled path... enclose it in double quotes.
  • always enclose the spyder-script.py path in double quotes
  • Remember to replace C:\ArcGISPro with your installation path... everything else is the same
  • This is your target path and it goes into the geoprocessing editor selection as below
  • environment:  C:\ArcGISPro\bin\Python\envs\arcgispro-py

 

Put them together....

     env\pythonw.exe "env\Scripts\spyder-script.py"

Yields....

 

C:\ArcGISPro\bin\Python\envs\arcgispro-py3\pythonw.exe "C:\ArcGISPro\bin\Python\envs\arcgispro-py3\Scripts\spyder-script.py"

 

 

 

---------------------     ---------------------     ---------------------    ---------------------    

Alternative IDEs ...

Of course you can always install and use Pythonwin for that easy-to-use Python IDE

 

 

How about just a simple QtConsole for parallel IDEs

 

 

---------------------     ---------------------     ---------------------    ---------------------    

Jupyter Notebook... graphing, analysis and graphing...

Just create a folder structure to store your notebooks and it can be replicated within the jupyter setup making it easy to find your examples.

 

A graphing example... there are lots

 

---------------------     ---------------------     ---------------------    ---------------------    

To Come... Managing your packages...

 

If will fill these in over the next week or so... a few things have to be cleaned up since they may not be for everyone.  So I will delay for now

 

I just used conda to upgrade jupyter notebook and install jupyter lab.  To stop it from trying to downgrade a bunch of packages, just add the default channel and --no-pin.  

 

 

ArcGIS Pro's package manager

Currently locked down for editing the initial install... and there are issues 'cloning' the existing one to make changes in.  Mind you... you can do your package installations using conda after the initial install, skipping the package manager altogether.  BUT .... if things get totally fouled up, you may be left with a dysfunctional installation... so only proceed if you sortof know what you are doing

 

Anaconda Navigator

So very useful, since you can access other Anaconda packages through one interface... but currently not ArcGIS Pro, but many others including documentation and the like.

 

https://community.esri.com/ideas/14817-anaconda-navigator-to-arcgis-pro

 

 

So... Once Anaconda Navigator got installed, I decided to see whether I could clone an environment that Pro 2.2 could see ... which seems to be an issue.  It seems to have worked so far.  Have a look

 

 

Now to experiment....

 

IF you have comments and questions... email me through my profile link