Select to view content in your preferred language

Geodatabase replication and edits by third party apps outside of ArcGIS/ArcObjects?

957
3
11-17-2010 04:35 AM
AZendel
Frequent Contributor
Because VBA support is about to end, we are having to update a few apps that ran in VBA.  They currently run against personal geodatabases that are beginning to get bogged down with too many records.  The apps are very table intensive with only a few spatial operations that are needed (geocoding, append, overlay).  The application, which is only run once a month to update the database with new data sent to us from third parties, uses a large number of SQL statements to maintain and update the data via ADO.  Then a few geoprocessing models are called when needed.  This approach works very well and we'd like to continue forward with it.

This VBA --> VB.net migration presents an opportunity to move the data into a Personal ArcSDE database on SQL Express.  The SQL Express feature class will NOT need to be versioned because I am the only person who edits the data.  After the monthly script has been run to import and process newly acquired data, it is pushed to an Oracle SDE database.  Some of the existing data gets updated during this process (a certificate of occupancy is issued on an existing permit).  Currently, we start an edit session in ArcMap, delete ALL of the features in the Oracle SDE database, then use the append tool to append ALL of the features from the personal geodatabase to the Oracle feature class (the schema are identical).  So this seems like a good opportunity to incorporate geodatabase replication into our process so we don't have to delete and then append all of the records each month.  But what happens when we extensively edit the data in the SQL Express geodatabase using ADO/SQL and no ArcObjects or geoprocessing models?  Will the replication system be aware that some records/features have been added and/or updated?  Will these changes be pushed to the Oracle geodatabase?

Thanks,

Alex
0 Kudos
3 Replies
JonHall
Frequent Contributor
Alex-
I assume you intend to create a 1-way replica from the Desktop SDE db in SQL Express ("parent") to the Enterprise SDE db in Oracle ("child")

I'm no expert, but have used replication quite a bit for 3 years. It sounds like you have a good reason to use SDE replication for distributing GIS updates to the Oracle db.

I believe ArcSDE replica synchronization uses StateIDs and GlobalIDs to compare the parent & child databases.

Adds: If a parent record's GlobalID does not exist in the child table, then synchronization adds that new record in the child.

Deletes: If the child record's GlobalID doesn't exist in the parent, then synchronization deletes that missing record from the child.

Changes: I'm not exactly sure how the parent & child records with matching GlobalIDs are compared for differences in StateIDs

You will have to add the GUIDs to all featureclasses/featuredatasets/tables in both databases that participate in the replica, to support SDE replication.  The Desktop SDE database will be "versioned", because the replica is a type of version.  These system replica versions are filtered out, so they do not appear in the Version Manager alongside the user-created Versions, but they appear in the SDE_Versions table, if you (have permissions to) view it with DBMS tools or an OLE DBO connection in ArcCatalog.

The replica version can tie up a StateID or two, which can begin to affect performance on the parent or child, so you should develop a workflow for compressing immediately after synchronization, and minimizing the StateIDs.  This won't be much of an issue on the Oracle SDE side, if that is not edited after you sync and compress it.  The sync will unreference the replica version, essentially automatically posting-reconciling and deleting it, but I think the StateID of that version remains until you Compress. 


You mention "many records", so beware that synchronization can be slow, particularly if the network connection is slow, especially when you are updating very large numbers of records. Desktop SDE in SQL Express is limited to 1GB of RAM, which can be like putting a governor on yor race car. You should make a test replica with copies of the databases on each server, and test the time it takes to sync an entire set of records from SQL Express to Oracle, to get an understanding of the performance.

You will have to determine how to initially create the replica, so that both parent and child start out identical.  (BTW, use the ArcMap distributed gdb toolbar, create replica tool, and in the advanced options, be sure to change from the default option, to using the full extent of the data, and change all tables from schema-only to all records)

Since you describe deleting all features from the Oracle SDE db before appending all new records, I would do that first, and starting with empty tables/featureclasses, create the 1-way replica with the Desktop SDE db as the "parent", and the Oracle Enterprise SDE db as the "child". The first time you synchronize, it will re-populate the Oracle Enterprise

You asked the right question, though I'm not sure of the answer:

If you "edit the data in the SQL Express geodatabase using ADO/SQL and no ArcObjects or geoprocessing models", and bypass ArcSDE's ability to manage StateIDs of changed records, you may find that synchronization does not recognize attribute differences between the parent and child...

That's my 2 cents worth of advice, I hope I didn't discourage you. Good luck!
0 Kudos
IanStack
New Contributor
I realize this is an old thread, but was wondering if you ever tested or resolved your question:  "But what happens when we extensively edit the data in the SQL Express geodatabase using ADO/SQL and no ArcObjects or geoprocessing models? Will the replication system be aware that some records/features have been added and/or updated? Will these changes be pushed to the Oracle geodatabase?"

My gut says that changes will not be recognized as suggested by the other responder.

Thanks,

Ian
0 Kudos
HeatherMcCracken
Esri Contributor
The edits must be made in a "geodatabase" aware editor in order for these changes to be recognized by replication - so in the ArcMap editor, using ArcObjects, gp tools etc.  Direct SQL statements do not fall into this category, and edits made this way will not be included in synchronization.

thanks,
Heather
0 Kudos