topic Re: SDE and Geoprocessing in Geoprocessing Questions

SDE and Geoprocessing

ChrisDonohue__GISP — Tue, 17 Feb 2015 18:55:53 GMT

A beginner ArcSDE question for you guys. I'm in ArcGIS 10.2.1 and working on a Version and I'd like to run some geoprocessing tools on the Version. Specifically, for starters I'd like to Erase out some areas so I can later append in replacements. However, Erase creates a new output feature class and the Version cannot be the output.

So my question is this, how does one work on a Version with geoprocessing tools that create a new output (which most geoprocesing tools do)? This ESRI help article seems to suggest that geoprocessing be done outside SDE:

Geoprocessing considerations for ArcSDE data

ArcGIS Help (10.2, 10.2.1, and 10.2.2)

If geoprocessing is done outside SDE, then what is the suggested workflow then to take the result and incorporate it back into the Version? For example, how do I get the Version to be Erased? Also, I didn't see it, but is there a list of what processes can be done directly to a Version? I discovered that the Append Geoprocessing tool works, but would like to know what other ones also can work.

Thanks in advance,

Chris Donohue, GISP

Re: SDE and Geoprocessing

JakeSkinner — Tue, 17 Feb 2015 19:33:28 GMT

Hi Chris,

When a new output is created, you will just have to register the feature class as versioned, and then you can work with the feature class in the desired version.

If you are looking to delete features from your feature class, I would recommend using the Delete Rows tool. This will not output a new a feature class. The Erase tool will modify the geometry of existing features based on another geometry (not sure if this is what you want or not).

Re: SDE and Geoprocessing

ChrisDonohue__GISP — Tue, 17 Feb 2015 19:38:48 GMT

Thanks for the reply, Jake. If I register the new feature class as Versioned, do I need to delete the old Version?

Also, I want to run the Erase for situations where only part of the polygon needs to be modified. I'll look into the Delete Rows suggestion for where full polygons need to be removed.

Chris Donohue, GISP

Re: SDE and Geoprocessing

JakeSkinner — Tue, 17 Feb 2015 19:58:46 GMT

No, you will not need to delete the old version. When you register a feature class as versioned, it is accessible in every version that the geodatabase contains. You can edit the feature class in any version, that is, if the user has permissions to that version. The version can either be Public, Private, or Protected.

Re: SDE and Geoprocessing

ChrisDonohue__GISP — Tue, 17 Feb 2015 21:45:59 GMT

Suppose I want to run several geoprocessing operations on my Version to come up with a result. If I pull the source feature class from SDE and run several geoprocessing operations with intermediate results in a File Geodatabase, and come up with a result, how would one get the final result that is in the FGDB back into SDE? Can I convert the final result that is in the File Geodatabase to a new Version? Would there be a "Data Loading" process for this? Or is it a "Delete all rows in the existing Version and Copy Features the new result back in" type-process? Or is there a more elegant tool/process?

I hope I didn't butcher the terminology too much here - I'm still trying to wrap my head around how SDE works after being predominantly a File Geodatabase addict for several years.

Chris Donohue, GISP

Re: SDE and Geoprocessing

VinceAngelo — Wed, 18 Feb 2015 02:03:20 GMT

It would probably help to review Versioning 101, because versioning isn't anything like what it seems you are trying to envision. Each database instance has ONE versioning state tree. "Versions" are just named states in the tree. If you take data out of the tree, you need a coherent plan to get it back in (which is what replication provides). If all you want is a cascade of changed data, then maybe you really don't want to use versioning, since the overhead of deleting all the records in a table as a regular practice is going to drag your system performance through a wringer.

- V

Re: SDE and Geoprocessing

ChrisDonohue__GISP — Wed, 18 Feb 2015 15:38:58 GMT

Thanks Vince, I'll take a look at that. What I'm trying to do is very easy in a File Geodatabase world; however, the organization I am currently with ultimately needs the final data put back into SDE, so I'm trying to get a handle on the SDE basics in order to determine the best way forward.

From what you mention, if I go the "cascade of changed data" route, how would I get the data back into SDE? Is that through a "Data Loading" process? (Sorry, I'm still not up on the exact SDE terminology for this).

Also, can anyone recommend training opportunities to learn more about editing and geoprocessing in SDE? I've been watching the ESRI training presentation "Versioned Editing Workflows for the Multiuser Geodatabase", which seems like a good introduction, but I wondering what other training opportunities people have found to be useful.

Chris Donohue, GISP

Re: SDE and Geoprocessing

VinceAngelo — Wed, 18 Feb 2015 16:33:14 GMT

The most important thing to know about SDE is that it no longer exists. The SDE product was renamed ArcSDE at 8.0, and removed from the product inventory at 9.2. Since then, only the connection protocol and the owner of geodatabase metadata has survived. "SDE" knows nothing about domains. "SDE" doesn't support CAD objects. "SDE" doesn't support replication or archiving. Even the vestigal ArcSDE capabilities like the command-line tools have been removed from ArcGIS 10.3. "SDE" has no significant role in enterprise geodatabases.

In order to come up to speed with enterprise geodatabases, I recommend classes in geodatabases. You need to be using enterprise geodatabase terminology, which fortunately for you, is the same as file geodatabase terminology (feature classes, feature datasets, domains,...).

I'm really not convinced you need to use versioned feature classes for your processing tasks. The optimistic locking algorithm at the heart of versioning is designed for small changes to a subset of rows. Large-scale bulk processing is better suited to the publishing of new feature classes, much the same as with file geodatabases.

- V

Re: SDE and Geoprocessing

ChrisDonohue__GISP — Wed, 18 Feb 2015 16:50:12 GMT

Thanks, Vince, for the info. I've done some quick editing to feature classes by Versioning, but those only involved a handful of changes. The current feature class I need to edit has dozens of edits, most of which can speedily be resolved with geoprocessing tools. After that I have another feature class that looks to have hundreds of edits - same deal. So I guess the ultimate question I have is how do I take a final updated result, which is in a File Geodatabase, and replace the existing parent feature class in DEFAULT? What specific tool/method/process accomplishes that? Based on what you and others have noted, it sounds like working against a Version is not the most effective way to go since there are so many changes.

Chris Donohue, GISP

Re: SDE and Geoprocessing

VinceAngelo — Wed, 18 Feb 2015 17:38:29 GMT

As with most aspects of ArcGIS, there are many ways to accomplish your goal:

You can replace (truncate and reload) the table contents
You can use a primary key to update the affected rows by edited feature
You can use check-out/check in replication

Having lots of changes isn't a problem with versioning, it's keeping lots of changes that's the problem. Once the edits are reconciled back into the base table they no longer use space. Yet if your edits are comprehensive enough, then truncate/reload becomes a simpler solution. If you don't have a viable primary key, then replication would provide one (in the form of a GlobalID), leveraging versioning in the process.

- V