Josh, thanks very much for this helpful reply! Serves me right for not consulting the latest documentation. Out of curiosity, how were you able to peek into the code? The GitHub repo seems to contain only documentation and examples. Speaking of which, I found the documentation to be a little vague on this. to_featureclass() mentions the sanitize_columns parameter, but there's no explicit connection within the documentation itself between that parameter and the sanitize_column_names() method. In general, I would say that a method that exports data should not simultaneously mutate those data, either in the calling object or in the output data. That just feels like a weird design pattern. I think I'd rather see sedf.sanitize_column_names() be one method, and sedf.to_featureclass() be another, and if you want to sanitize your column names for output to feature class, the approach should be to just call the two methods separately.
... View more
I recently updated to ArcGIS Pro 2.7.1. This new version of ArcGIS Pro comes with a new version of the arcgis package (version 1.8.3). And this new version of the arcgis package, regrettably, comes with new bugs for Spatially Enabled DataFrames - even beyond the myriad bugs I've previously encountered and solved or worked around. Just to be clear, I'm talking about Spatially Enabled DataFrames (SEDFs), not Spatial DataFrames (SDFs). https://developers.arcgis.com/python/guide/introduction-to-the-spatially-enabled-dataframe/ I love, love love love, the idea of Spatially Enabled DataFrames - an easy way to get data from an Esri feature class into a pandas DataFrame and vice versa. But in practice, these have been the most cursed, unreliable, exasperating structures I've worked with for a long time. First there was the snafu where sedf.spatial.to_featureclass() silently mutated the calling object, overwriting whatever index the SEDF had had with a simple integer index. Of course this breaks the ability to join data onto that SEDF based on the original index. Nowhere was this behavior documented or justified. Still, there were workarounds (generally, just call sedf.reset_index() to capture the index in a column), but talk about ungraceful and un-Pythonic. Once upon a time, sedf.spatial.to_featureclass() could write out to an FC in the memory workspace - but no more, as of ArcGIS Pro version 2.5 or 2.6 (I forget which). Suddenly those straightforward calls were throwing errors that I could never fully resolve, so I had to rewrite a bunch of scripts to write out to FCs in the scratch GDB, which is slower and less elegant. The latest twist as of arcgis package version 1.8.3 is that sedf.spatial.to_featureclass() mutates the data itself, casting all column names to LOWERCASE - in the output FC and in the calling object itself! Why? Why, why, why? This ruins both the calling SEDF and the output FC. In my case this led to a brand-new, mysterious crash that took me the better part of a full day to diagnose and fix. The latest fix, by the way, was abandoning Spatially Enabled DataFrames entirely and falling back to some nice, simple, GP-tool only code from David Wasserman and Roland Viger. And that leads me back to the title of my post: I'm done with Spatially Enabled DataFrames. I have run headlong into one too many mysterious, subtle bugs that hold me up for many hours on end. I love the concept but the execution is just not there - and, as far as I can tell, it's getting worse, not better. Have other folks had similar issues with Spatially Enabled DataFrames? Have you found any good fixes? Is there any way I can help improve SEDFs so they deliver on their amazing promise?
... View more