Latest Contributions by JoshuaBixby

The complexity, size, or both of the dataset is driving whether or how tiling is being used.  If you have access to ArcGIS 10.1 or higher, you could try using 64-bit Background Geoprocessing.  The 64-bit Background Geoprocessing tools allow for much more RAM to be utilized, which could change or eliminate tiling and get you the results you are after. ... View more

Tile processing of large datasets was introduced in ArcGIS 9.2 (well, it could be done in 9.1, but it was more manual and not integrated or automatic).  Even without looking at the Help, the progress window for the Union tool shows there is tiling occurring: Maybe there were not enough qualifiers on your statement about the Union tool and new vertices because the Union tool has to add new vertices in many cases: If your statement was meant for self unions, then I agree that new vertices seem spurious. What steps are you using to find the extra vertices? I am running ArcGIS 10.2.2 with all but the last couple patches, and I did find extra vertices.  I only closely looked at a handful of the extra vertices, but they were all duplicate vertices.  If it turns out that all of the extra vertices are duplicates, then there is no difference in the geometries of the polygons, well the perimeter and such. The reason I asked what steps you took to find extra vertices is that I used the Feature Vertices to Points tool.  Without doing more digging, I can't say whether the issue resides in the Union tool or Feature Vertices to Points tool. Beyond the tools, have you checked how clean the polygons are?  Looking at a few of the duplicate vertices, they seem to be near areas where the polygons seem to be self intersecting, or really close. ... View more

Do you have access to the SDE schema tables?  Have you looked to see if there are possibly entries for the view even though it doesn't exist?  Have you tried creating the view with a slightly different name?  I know in the past, if I had an mv_view that I didn't clean up completely, I would sometimes get this error message when trying to recreate the view. ... View more

Vince Angelo‌, I remember that blog post.  It was quite interesting and good material to be referenced and looked at in relation to this discussion.  I very much look forward to seeing some of the ideas discussed in that blog post come to fruition, understanding they are side projects of course. The reason I linked to that thread in the SQL Server Spatial Forum was that I thought Jakub K's recursive-quartering tessellation scheme was fairly elegant, and quite simple to implement through a couple table-valued functions.  From what I have seen, the geoprocessing/spatial functions in question do seem sensitive to vertex count, so reducing vertex count definitely helps. For most datasets, maybe tessellation is overkill and gridding will suffice.  For one of the datasets I was working with, that wasn't the case.  Looking at a gridding scheme and recursive-quartering tessellation scheme that both generate similar numbers of partitions for one of the complex polygons in my dataset, one can see how the tessellation balanced out the number of vertices better. Balancing out the vertices better allowed for the next part of the geoprocessing/spatial functions to execute quicker.  Using just the DBMS, no RAM caching, the recursive-quartering tessellation scheme got me in the range of 0.5 ms/feature while the gridding scheme was closer to 1.5 ms/feature.  I admit I didn't spend much time on optimizing a gridding scheme, so there is likely room for improvement there, but I am not sure if a more efficient gridding scheme alone will do much better than the current overall performance of using the tessellation scheme. I am pretty content with 0.5 ms/feature, and Jakub K's tessellation scheme got me there, so I thought I would share.  I know there are techniques that can get even better performance, but for now that extra investment isn't necessary for my situation. ... View more

I bit more information about the database would be helpful.  For example, what version of SDE and what DBMS and version?  Also, are we talking about two feature classes in the feature dataset?  A feature class and a table?  What command and syntax are you using to try to create it from the command line? ... View more

Not allowing nulls is one way of forcing users to populate a field, but it sets the bar pretty low in terms of data quality expectations.  If users are forced to enter something because nulls aren't allowed, and there are no restrictions on what can be entered, people will get creative on what they enter.  In these situations, not allowing nulls is worse than allowing them because at least when nulls are allowed and a field is populated, the odds are better than not someone didn't just make up a value. If you want to make users populate a field and you also want to increase the odds of having decent data, then it might make sense to look into Attribute Domains and Subtypes.  If you want to allow non-null, free-form text than some level of programming will likely be needed to achieve what you are after. ... View more

Are you wanting to change the default value for a field?  Does the Assign Default To Field (Data Management) tool not work for you? Regarding your last question about finding a way to check whether the valued entered is blank or not, are you talking about doing this in real-time as a user inputs text or going back through the records to see which ones are blank and not blank? ... View more

SQL Server Express is good for what it is, a free and limited DBMS, but I don't think it will take much to start hitting resource limits in SQL Server Express once you take the 3-connection limitation away.  Depending on your usage, you might want to consider upgrading to SQL Server Standard or Enterprise or even PostgreSQL if cost is an issue for upgrading SQL Server. ... View more

Thinking back to the statement about a point-in-polygon query taking 17 hours to run, my guess is that you have some complex or dense municipal boundaries.  SQL Server, even Enterprise Edition, doesn't seem to handle point-in-polygon queries well when there are complex or dense polygons.  Some say it is tied to MS's implementation of quad-tree spatial indexes or a limitation of quad-tree indexes for spatial data.  I am not deep enough into databases to really have an informed opinion, but I do have to tweak spatial indexes much more in SQL Server than in PostgreSQL/PostGIS. For the complex/dense point-in-polygon situation, I think there is something more with how SQL Server implements some of its spatial methods.  If you either don't or can't go down the geoprocessing route, I encourage you to read through this MSDN forum thread on SQL 2K8R2: Any performance hints for bulk loading data from spatial/GIS db into a data warehouse?  Although the title doesn't make it seem relevant to this post, it is.  The author of the thread has an approach and some code for using tessellation or on-the-fly partitioning to break up complex polygons and dramatically reduce run times.  I have successfully used the technique and seen reductions of 90%+. ... View more

This one was on Esri.  The beta licenses expired, but the pre-release licenses were never automagically loaded when they were supposed to, which wasn't a problem until the beta licenses did expire.  New licenses were loaded, and we are back in business. Take home message, don't hesitate to contact your Esri Customer Service Representative if what you see in AGOL for Pro licenses doesn't make sense or seems like it should work but doesn't. ... View more

I agree with James Crandall‌ about SELECT * . Are you using pyodbc?  If so, fetchall() returns an empty list, even if no rows are returned, so I don't get why rows wouldn't be defined.  If a successful query, even one with no results, returns a list that could instantiate rows; maybe the queries are generating an error and fetchall is returning nothing, so rows doesn't exist. ... View more

We are on a large BPA/ELA which hasn't expired, or at least our current extension hasn't.  I have reached out to our Esri Customer Service Representative to help sort out what is going on.  A couple of our AGOL administrators said they never received an e-mail, so we have some work to sort out those mailing lists too. Once the dust settles, I will give an update in case others run into similar situations in the future. ... View more

Unfortunately, this is one of my gripes with the AGOL-based licensing scheme, i.e., AGOL administrators in larger organizations aren't necessarily in the same part of the organization as the GIS users because of the way ELA/contracts are established.  Maybe our admins got an e-mail, but would they even know what it meant if they actually read it. Appreciate the feedback, at least I now know who I have to go track down. ... View more