Latest Contributions by JoshuaBixby

If you are willing to install comtypes, then the SDE connection properties can be extracted without actually connecting to the database.  I ginned up the following function, just pass it the SDE connection file and it returns a dictionary with all of the connection properties. def GetConnectionPropertiesFromFile(in_workspace):     from comtypes.client import CreateObject, GetModule     import os         comDirectory = os.path.join(         os.path.join(arcpy.GetInstallInfo()['InstallDir']), 'com'     )     esriDataSourcesGDB = GetModule(os.path.join(comDirectory, 'esriDataSourcesGDB.olb'))     esriGeoDatabase = GetModule(os.path.join(comDirectory, 'esriGeodatabase.olb'))         pWSF = CreateObject(esriDataSourcesGDB.SdeWorkspaceFactory,                         interface=esriGeoDatabase.IWorkspaceFactory)     pPropSet = pWSF.ReadConnectionPropertiesFromFile(in_workspace)     names, values = pPropSet.GetAllProperties()         connectionProperties = {}     for i in range(len(names)):         connectionProperties[names] = values             return connectionProperties ... View more

Since Application Servers are dead at ArcGIS 10.3, I don't think Esri is going to invest anything in making it easier to identify them.  The "Database Platform" field you reference is relatively new-ish, starting at ArcGIS 10.1.  Although the GUI shows this new field, I don't believe there were any changes with the workspace connection properties data structure on the backend.  In the end, returning a Database Platform value would have to be from a function that processes the connection properties and not directly from the connection properties themselves. You raise an interesting, and annoying, part of processing SDE connection files.  I primarily work with operating system authentication, so the issue of having the username/password prompt doesn't usually impact me.  That said, it is frustrating that one has to fully authenticate for ArcPy to see any of the connection properties.  When working with the GUI, a user can look at several connection properties without having to actually connect to the database.  It would be nice to see similar behavior in ArcPy if no username or password are given. ... View more

What about passing an array variable (Cursor.arrayvar) instead of a cursor? ... View more

32-bit.  It would be interesting whether 64-bit SciPy would make a difference.  And yes, in chunks.  My original NumPy code worked in chunks anyways to make the memory footprint lower at the expense of slightly slower code. ... View more

Kicked the tires on SciPy.  At first I got memory errors, but I was able to work around it.  Iterating over a 10,000 point feature class against a 50,000 point feature class, instead of comparing all 500,000,000 combinations at once, the SciPy method was ~15% faster than my original straight NumPy approach.  Assuming the memory errors are manageable, SciPy does offer a performance improvement in this case. It is good that Esri will be packaging and automatically installing SciPy with future ArcGIS releases. ... View more

Have you read PostgreSQL data types supported in ArcGIS?  It contains lots of good information about PostGIS and ArcGIS, including requirements for using PostGIS data types in ArcGIS, with or without SDE involved. ... View more

stat_point is primarily deployed on point sets with millions or tens of millions of comparisons, so an extra few seconds of overhead here or there really is negligible compared to the overall compute times.  That said, an apples-to-apples comparison would be interesting.  I am also interested in improving performance, and SciPy is rumored to be installed automatically with ArcGIS 10.3.1. ... View more

It isn't really an apples to apples test, is it?  Where did you test?  ArcMap interactive Python window?  Running Background processing? The arcpy.Describe call is needed to get spatial references so that you can ensure data in different projections is exported to NumPy with the same spatial reference. I can install SciPy sometime soon and do a more apples-to-apples test. ... View more

I stuck with NumPy because SciPy isn't shipped with ArcGIS Desktop, yet, and SciPy isn't part of our standard data center Python deployment either.  From a performance perspective, it would be interesting to compare the two approaches. ... View more

If you were interested in the farthest instead of nearest, then you would have to use Point Distance or Generate Near Table in your workflow, unless you wanted to role your own NumPy-based function.  When working with data sets in the tens of thousands, which much from a practical stand point, I have found the Point Distance tool to be very slow and sometimes Generate Near Table fails with unspecified errors. For situations where I want farthest points or quicker outputs similar to Point Distance, I have roled my own NumPy-based functions.  For example: def stat_point(in_features, other_features, stat='MINIMUM'):     import arcpy     import numpy        stats = {         'MINIMUM': {'FID': 'MIN_FID',                     'DIST': 'MIN_DIST',                     'INDEX': lambda x: 0},         'MAXIMUM': {'FID': 'MAX_FID',                     'DIST': 'MAX_DIST',                     'INDEX': lambda x: x - 1},         'MEDIAN_HIGH': {'FID': 'MEDH_FID',                         'DIST': 'MEDH_DIST',                         'INDEX': lambda x: x / 2},         'MEDIAN_LOW': {'FID': 'MEDL_FID',                     'DIST': 'MEDL_DIST',                     'INDEX': lambda x: x / 2 - 1 if x % 2 == 0 else x / 2 }     }        desc = arcpy.Describe(in_features)     SR = desc.spatialReference        desc = arcpy.Describe(other_features)     OID_name_other = desc.OIDFieldName     shape_name_other = desc.ShapeFieldName     narr_other = arcpy.da.FeatureClassToNumPyArray(         other_features,         [OID_name_other, desc.ShapeFieldName],         spatial_reference = SR     )     xy_other = narr_other[shape_name_other]     idx = stats[stat]['INDEX'](numpy.shape(xy_other)[0])        arcpy.AddField_management(in_features, stats[stat]['FID'], 'LONG')     arcpy.AddField_management(in_features, stats[stat]['DIST'], 'DOUBLE')        with arcpy.da.UpdateCursor(         in_features,         ["SHAPE@XY", stats[stat]['FID'], stats[stat]['DIST']]     ) as cur:         for xy, FID, DIST in cur:             xy_in = numpy.array(xy)             d0 = numpy.subtract.outer(xy_in[0], xy_other[:,0])             d1 = numpy.subtract.outer(xy_in[1], xy_other[:,1])             dist = numpy.hypot(d0, d1)             i = dist.argsort()[idx]             FID = narr_other[OID_name_other]             DIST = dist             cur.updateRow([xy, FID, DIST]) When used with "MINIMUM", the stat_point function mimics the Near tool.  The Near tool is more performant than using stat_point to find the minimum distance point, but stat_point is orders of magnitude faster at finding non-nearest points compared to Point Distance or Generate Near Table.  Of course, stat_point is doing planar measurements so that is something to be aware of.  I threw MEDIAN parameters in as an example of how simple it is to extend this methodology beyond minimum or maxiumum points. The NumPy heavy lifting, lines 43-45, comes from Alex Martelli in response to a Stack Overflow post:  Euclidean distance between points in two different Numpy arrays, not within... ... View more

Have you read Connect to PostgreSQL from ArcGIS?  If so, what step isn't working and what is the specific error message?  ArcGIS Desktop does not install the DBMS clients, the user needs to install it separately.  Can you connect to the PostgreSQL database outside of ArcGIS Desktop on the same machine? UPDATE:  The following is probably a better link to start with than my above one:  Database connections in ArcGIS for Desktop.  Specific error messages are helpful. ... View more

Turns out, existing bug: Esri Support: BUG-000083370 : Select by Location does not select all features that meet the criteria. Workround - Import the extent of the Source layer into the Target layer To workaround this issue, you have a couple of options. The first is to create a new feature above the square that is not being selected out side of the source layer. This will force the extent of the feature class out and allow the square feature to be selected. Once the extent has been extended you can go ahead and delete the other polygon. The other option is to temporarily load the large square (xxxx_ext1) into the xxxx_feature feature class to pull out the extent. This will guarantee that the extent is correct in every direction. ... View more

OK, I get it, you want to find the nearest hazard site to each construction site, and then rank/order all the construction sites in descending order of that value.  So, the construction site at the top has the largest minimum distance to a hazard site. ... View more

Are you after nearest or farthest points?  You say "furthest" a lot but then you also say "closest" at one point.  ... View more