Latest Contributions by JoshuaBixby

Kicked the tires on SciPy.  At first I got memory errors, but I was able to work around it.  Iterating over a 10,000 point feature class against a 50,000 point feature class, instead of comparing all 500,000,000 combinations at once, the SciPy method was ~15% faster than my original straight NumPy approach.  Assuming the memory errors are manageable, SciPy does offer a performance improvement in this case. It is good that Esri will be packaging and automatically installing SciPy with future ArcGIS releases. ... View more

Have you read PostgreSQL data types supported in ArcGIS?  It contains lots of good information about PostGIS and ArcGIS, including requirements for using PostGIS data types in ArcGIS, with or without SDE involved. ... View more

stat_point is primarily deployed on point sets with millions or tens of millions of comparisons, so an extra few seconds of overhead here or there really is negligible compared to the overall compute times.  That said, an apples-to-apples comparison would be interesting.  I am also interested in improving performance, and SciPy is rumored to be installed automatically with ArcGIS 10.3.1. ... View more

It isn't really an apples to apples test, is it?  Where did you test?  ArcMap interactive Python window?  Running Background processing? The arcpy.Describe call is needed to get spatial references so that you can ensure data in different projections is exported to NumPy with the same spatial reference. I can install SciPy sometime soon and do a more apples-to-apples test. ... View more

I stuck with NumPy because SciPy isn't shipped with ArcGIS Desktop, yet, and SciPy isn't part of our standard data center Python deployment either.  From a performance perspective, it would be interesting to compare the two approaches. ... View more

If you were interested in the farthest instead of nearest, then you would have to use Point Distance or Generate Near Table in your workflow, unless you wanted to role your own NumPy-based function.  When working with data sets in the tens of thousands, which much from a practical stand point, I have found the Point Distance tool to be very slow and sometimes Generate Near Table fails with unspecified errors. For situations where I want farthest points or quicker outputs similar to Point Distance, I have roled my own NumPy-based functions.  For example: def stat_point(in_features, other_features, stat='MINIMUM'):     import arcpy     import numpy        stats = {         'MINIMUM': {'FID': 'MIN_FID',                     'DIST': 'MIN_DIST',                     'INDEX': lambda x: 0},         'MAXIMUM': {'FID': 'MAX_FID',                     'DIST': 'MAX_DIST',                     'INDEX': lambda x: x - 1},         'MEDIAN_HIGH': {'FID': 'MEDH_FID',                         'DIST': 'MEDH_DIST',                         'INDEX': lambda x: x / 2},         'MEDIAN_LOW': {'FID': 'MEDL_FID',                     'DIST': 'MEDL_DIST',                     'INDEX': lambda x: x / 2 - 1 if x % 2 == 0 else x / 2 }     }        desc = arcpy.Describe(in_features)     SR = desc.spatialReference        desc = arcpy.Describe(other_features)     OID_name_other = desc.OIDFieldName     shape_name_other = desc.ShapeFieldName     narr_other = arcpy.da.FeatureClassToNumPyArray(         other_features,         [OID_name_other, desc.ShapeFieldName],         spatial_reference = SR     )     xy_other = narr_other[shape_name_other]     idx = stats[stat]['INDEX'](numpy.shape(xy_other)[0])        arcpy.AddField_management(in_features, stats[stat]['FID'], 'LONG')     arcpy.AddField_management(in_features, stats[stat]['DIST'], 'DOUBLE')        with arcpy.da.UpdateCursor(         in_features,         ["SHAPE@XY", stats[stat]['FID'], stats[stat]['DIST']]     ) as cur:         for xy, FID, DIST in cur:             xy_in = numpy.array(xy)             d0 = numpy.subtract.outer(xy_in[0], xy_other[:,0])             d1 = numpy.subtract.outer(xy_in[1], xy_other[:,1])             dist = numpy.hypot(d0, d1)             i = dist.argsort()[idx]             FID = narr_other[OID_name_other]             DIST = dist             cur.updateRow([xy, FID, DIST]) When used with "MINIMUM", the stat_point function mimics the Near tool.  The Near tool is more performant than using stat_point to find the minimum distance point, but stat_point is orders of magnitude faster at finding non-nearest points compared to Point Distance or Generate Near Table.  Of course, stat_point is doing planar measurements so that is something to be aware of.  I threw MEDIAN parameters in as an example of how simple it is to extend this methodology beyond minimum or maxiumum points. The NumPy heavy lifting, lines 43-45, comes from Alex Martelli in response to a Stack Overflow post:  Euclidean distance between points in two different Numpy arrays, not within... ... View more

Have you read Connect to PostgreSQL from ArcGIS?  If so, what step isn't working and what is the specific error message?  ArcGIS Desktop does not install the DBMS clients, the user needs to install it separately.  Can you connect to the PostgreSQL database outside of ArcGIS Desktop on the same machine? UPDATE:  The following is probably a better link to start with than my above one:  Database connections in ArcGIS for Desktop.  Specific error messages are helpful. ... View more

Turns out, existing bug: Esri Support: BUG-000083370 : Select by Location does not select all features that meet the criteria. Workround - Import the extent of the Source layer into the Target layer To workaround this issue, you have a couple of options. The first is to create a new feature above the square that is not being selected out side of the source layer. This will force the extent of the feature class out and allow the square feature to be selected. Once the extent has been extended you can go ahead and delete the other polygon. The other option is to temporarily load the large square (xxxx_ext1) into the xxxx_feature feature class to pull out the extent. This will guarantee that the extent is correct in every direction. ... View more

OK, I get it, you want to find the nearest hazard site to each construction site, and then rank/order all the construction sites in descending order of that value.  So, the construction site at the top has the largest minimum distance to a hazard site. ... View more

Are you after nearest or farthest points?  You say "furthest" a lot but then you also say "closest" at one point.  ... View more

I think part of what is confusing people is semantics, particularly the use of the word "furthest."  Assuming no two construction sites are exactly the same distance from a waste facility, you will only have one construction site that is furthest away.  If by furthest you mean far, then a definition of what is far and what is close would be helpful.  Or, are you looking to simply determine the distance from each construction site to each waste facility? ... View more

You can accomplish what you want, and a whole lot more, using Pivot Tables.  Pivot Tables are quite powerful and the learning curve isn't too steep. ... View more

If you are only working with Points, another approach is to use the new NumPy-based features that were introduced with the ArcPy data access (arcpy.da) module. import os import numpy as np in_features = #Point data set to have exported as individual data sets out_path = #Path to geodatabases or folder for individual data sets out_prefix = 'pnt_'  #Prefix for individual data sets, can be empty '' desc = arcpy.Describe(in_features) SR = desc.spatialReference OID_name = desc.OIDFieldName shape_name = desc.ShapeFieldName narr = arcpy.da.FeatureClassToNumPyArray(in_features, "*") ndtype = narr.dtype for row in narr:     n = np.array(row, ndtype)     OID = n[OID_name]     arcpy.da.NumPyArrayToFeatureClass(n,                                       os.path.join(out_path, out_prefix + str(OID)),                                       shape_name,                                       SR) The above script will work on shape files and feature classes because I am using an arcpy.Describe object to retrieve the name of the unique identifier and shape fields. ... View more

First, I don't think MakeFeatureLayer_management is what you want unless you are using it as an intermediate step before exporting the feature layer. Second, it helps to paste specific error messages. Third, what about putting str(i) in place of i in the layer name you are concatenating. ... View more

I hear ya.  Usually there are workarounds, but not always.  The frustration with relying on workarounds is that they tend to break even more frequently than ArcGIS between versions, which in effect puts you on a gerbil wheel.  Workarounds are a necessity at times, but they are also wasted time when there is work to be done. For your sake and mine, along with many others, let's hope each 10.3.x release adds more geodatabases administration functions/tools. ... View more