topic Re: Geodata Service with 2-way Replication in ArcGIS Enterprise Questions

Geodata Service with 2-way Replication

mpboyle — Wed, 18 Feb 2015 14:35:17 GMT

I'm trying to setup a geodata service and use it as a means for 2-way geodatabase replication. I don't have any problem creating the geodata service from a versioned SDE geodatabase. However, I haven't been able to create a replica geodatabase and I'm not entirely sure why.

My goal is to publish a geodata service from a SDE gdb which other entities can use to create a 2-way replica and we can sync our edits back and forth...so as to have a common dataset for editing. Unless I'm mistaken, this should be doable...?

I have the following questions about the workflow and was wondering if anyone has experience using a similar approach and could explain their methods?

1) When publishing the geodata service, do you use a versioned connection or a connection to the DEFAULT version? Are there any permissions needed by the user the service is published from other than being able to edit the dataset?

2) What tool in ArcToolbox do you use to create the replica (Create Replica or Create Replica From Server)? Or is there an alternate workflow for creating the replica through ArcMap?

3) Are there any necessary permissions within SDE that I'm overlooking?

Environment:

1) SDE = 10.1

2) ArcGIS Server = 10.3

3) RDBMS = SQL Server 2008 R2

Thanks in advance!

Re: Geodata Service with 2-way Replication

JakeSkinner — Wed, 18 Feb 2015 19:39:56 GMT

Hi Matthew,

This is doable. To answer your questions:

1. This is up to you. If you want the edits replicated to/from a version, you can publish the geodata service when you are connected to the version (through the geodatabase connection properties). No, there aren't any other permissions the user will need other than edit.

2. I usually publish the geodata service, and then create the replica using the Distributed Geodatabase toolbar. I find the GUI a little easier to use. You will want to specify the geodata service for the geodatabase.

3. You shouldn't need any special permissions.

What are the errors you are receiving when you try and create the replica?

Re: Geodata Service with 2-way Replication

mpboyle — Wed, 18 Feb 2015 19:56:46 GMT

Jake Skinner‌

1) Creating the geodata service is pretty straight forward.

2) This is the step I'm a little confused about. When you say you usually publish the geodata service and then create the replica using the Distributed Geodatabase toolbar, I assume this means you are in ArcMap? How can you access the geodata service in ArcMap?...it's not an object that can be added to ArcMap.

Re: Geodata Service with 2-way Replication

JakeSkinner — Wed, 18 Feb 2015 20:12:41 GMT

You will want to browse to the Geodata Service under your GIS Servers. Ex:

Re: Geodata Service with 2-way Replication

mpboyle — Wed, 18 Feb 2015 20:32:09 GMT

In this scenario you're replicating from sde.DEFAULT...I assume this is the 'parent'?

When you specify the geodata service as the gdb to replicate to, does the geodata service connect to the child gdb?

Can you tell this is something I'm not familiar with?

Re: Geodata Service with 2-way Replication

JakeSkinner — Thu, 19 Feb 2015 13:18:59 GMT

In this scenario, you can consider the database I'm replicating from as the 'parent'. I published the database I want to replicate data to as a Geodata Service.

For example, I have database A and database B. I want to replicate data from A to B in a two way replica. I would publish B as a geodata service, and then create the two way replica by adding the data from A to ArcMap and run through the Create Replica tool on the Distributed Geodatabase using the Geodata Service as the database to replicate to.

Once the replica is complete, I can edit the feature class in either database A or B and sync the changes between the two using the Geodata Service.

Re: Geodata Service with 2-way Replication

mpboyle — Thu, 19 Feb 2015 15:43:00 GMT

Thank Jake...this makes sense and I've been able to successfully set up a replica using the geodata service pointed at the 'child' gdb (B).

Would I need a second geodata service for the 'child' (B) to push edits / sync back to the 'parent' (A)?

Re: Geodata Service with 2-way Replication

JakeSkinner — Thu, 19 Feb 2015 16:02:30 GMT

If I'm following you correctly, you have data replicated from (A) to (B). You can synchronize in both directions between these geodatabases. You will only need a second geodata service if you want to replicate data from (B) to (A).

Re: Geodata Service with 2-way Replication

mpboyle — Thu, 19 Feb 2015 16:08:46 GMT

Got it...you can use the Synchronize Changes tool to sync in both directions. I'm assuming this would be useful if trying to schedule syncing on a regular basis through a model or script.

I really appreciate your help with this Jake Skinner‌ !

Re: Geodata Service with 2-way Replication

WilliamWang — Mon, 09 Mar 2015 23:45:44 GMT

Just found your posting and it seems you have no problem with your 2 way replication using 1 replica and 1 Geodata service (GDS). I am curious to know how has the sync been going thus far?

Have been working on the same issue here with our outside entity and we found that we were unable to sync simply using 1 GDS. Any insight is helpful on this matter.

Also, are you replicating as your earlier description going to/from a versioned of your default GDB to your outside entity's versioned of their default GDB? If so, are you Default GDB set to protected on both ends?

Thank you.

Re: Geodata Service with 2-way Replication

mpboyle — Tue, 10 Mar 2015 19:13:35 GMT

William,

What I have setup is more of a test environment. What I have are two SDE geodatabases and two geodata services (one for each SDE gdb). In my test environment, one SDE gdb is meant to represent the 'county' gdb and the other is meant to represent the 'city' gdb.

I have scripts setup that will sync changes, analyze datasets, compress, rebuild indexes, and analyze datasets once again on each SDE gdb.

So...for example, if I'm synchronizing changes to my 'county' gdb, then the I use a database connection file to the 'county' gdb as gdb1 and a ArcGIS Server connection file to the 'city' geodata service as gdb2. What this would do is sync changes in both directions between my 'county' gdb and 'city' gdb using the geodata service as my remote connection to the 'city' gdb. On the flip-side when I sync changes and run the gdb admin tools on the 'city' gdb, you basically just flip-flop gdb1 and gdb2...where gdb1 would be the database connection file to the 'city' gdb and gdb2 would be the ArcGIS Server connection to the 'county' geodata service.

I run these scripts on a daily basis and while we haven't been testing thoroughly, they haven't failed, and when I have tested by entering data into the 'county' gdb and then syncing back to the 'city' geodata service, it has been successful...likewise if I edit data in the 'city' gdb and then sync back to the 'county' geodata service.

Both the geodata services are within a secured folder on our test ArcGIS Server site. Both SDE gdbs are versioned.

Re: Geodata Service with 2-way Replication

WilliamWang — Tue, 10 Mar 2015 20:26:20 GMT

Matthew,

Thank you for your detailed explanation.

We, too, have been testing with lots of frustrations. We have not been successful in running sync without having another Geodata Service (GDS). Meaning, 1 replica with 2 GDS. We weren't able to get sync to work both direction until the "child" GDB users publish a 1 GDS for the "parent" GDB users to absorb. An esri support analyst was able to duplicate the problem and agreed with our fining and it was noted as a bug. However, base on your description, you are running just fine. BTW, your county to/from city model is very much similar to ours.

What method did you use to bring your data over to the "child" GDB? We've been using Create Replica from Server GP tool rather than using the Distributed GDB toolbar in ArcMap.

Do you publish a map service along side with your GDS? We don't since we create our service from ArcCatalog.

As to my versioned question earlier, we've begun testing it recently and we found the if my Default GDB is set to protected and my versioned of the Default GDB is set to public, my outside entity would not be able to sync unless my Default GDB is reset to public.

Your experience and outcome were what we originally anticipated but never realized in our environment.

Re: Geodata Service with 2-way Replication

mpboyle — Tue, 10 Mar 2015 20:42:21 GMT

William,

It's been awhile since I created the replica...but if I remember correctly, what I did was use the 'Create Replica' (not Create Replica from Server) tool in the Distributed Geodatabase toolbox. Input parameters would have been along the lines of:

a. Replica Datasets = 'county' gdb SDE database connection

b. Replica Type = TWO_WAY_REPLICA

c. Geodatabase to replicate to = 'city' geodata service

d. Replica Name = whatever you want

e. Advanced Settings = all defaults

So...the 'county' gdb acted as the parent and pushed data to the 'city' gdb using the 'city' geodata service as the connection.

I did not publish a map service since I did it all through ArcCatalog...and for what we're testing, I don't think it would be necessary or helpful.

I'm not sure what you mean by 'Default GDB is set to protected vs. public'.

I originally struggled with the concept of a geodata service and how to replicate the data using a geodata service. I believe all SDE gdbs need to be in place as well as all geodata services. You should be able to then use the 'Create Replica' tool in the Distributed Geodatabase toolbox using the 'child' geodata service as geodatabase to replicate data to parameter...that way you don't have to create a XML or some other static snapshot of your data and load into the 'child' / replica gdb.

Re: Geodata Service with 2-way Replication

WilliamWang — Tue, 10 Mar 2015 21:10:18 GMT

Matthew,

Completely agree with you on xml or FGDB matters.

We are testing our replication over the net between 2 networks hence we have to use Create Replica from Server GP tool or else the outside party will need to provide the "parent" DB to me to restore in my SQL then we have to do the final hand shake using Register Existing Dataset.

Thanks again for your info.

Re: Geodata Service with 2-way Replication

JakeSkinner — Wed, 11 Mar 2015 10:48:23 GMT

I worked with the State of Maryland where counties are replicating their street centerlines and address point data to the State. Attached are PDFs for steps for the State to perform to create a geodata service, and another PDF for the county with steps how to replicate and synchronize their data. For this scenario, it is just a one-way replication from the county (Parent) to the state (Child). Not sure if this is helpful or not.

Re: Geodata Service with 2-way Replication

WilliamWang — Wed, 11 Mar 2015 17:10:25 GMT

Jake,

Your document helps. Likewise with Matthew's notes on Create Replica GP tool.

Interestingly, when using Create Replica tool yesterday, received a general replicating failed error message 00582 but when examining the "child" GDB, the replicating data appears. Any input on this?

Was able to reproduce the error and outcome twice.

Re: Geodata Service with 2-way Replication

JakeSkinner — Wed, 11 Mar 2015 18:12:40 GMT

I've seen this when trying to replicate a large amount of data using a geodata service. One thing to try is to increase the 'The maximum time a client can use the a service'. You can do this by right-clicking on the geodata service > Service Properties > Pooling.

If this does not work, it's recommended to create an XML file and then manually import this file to create the replica.

ArcGIS Help (10.2, 10.2.1, and 10.2.2)

If the network is available but not reliable, you may still want to consider using disconnected replication. A replica creation process running over a slow network may be time consuming and unreliable. With disconnected replication, you can export to a file and continue working without having to wait for the information to be sent over the network. In this case, however, you will want to create backups of these files in case they are lost before being imported on the target.

Re: Geodata Service with 2-way Replication

WilliamWang — Wed, 11 Mar 2015 20:06:03 GMT

Jake,

Your suggestion on increasing the pool time worked for my internal test - within the same server.

Prior to increasing the max time number, regardless of using Create Replica Wizard or GP tool, the replication would failed. In fact, via the Wizard, I did not get any error number except an error window that said replication failed, a syntax error.

Once I increase the max time from default of 600 to 1800, both methods worked. Will try on different and larger dataset that we are having trouble with.

Will you say the definition of "large" dataset that you mentioned earlier is fairly arbitrary? It seems, for city of our size, our data is small in comparison to other major cities or companies.

Also, we are testing and running full model replication, and thus far we are theorizing that a GDB cannot be replicated if it has a FDS that has both Geometric Network and M-N relationship. Any experience on this?

Lastly, on your suggestion of using xml. Is there a check list or some sort to find out if the network isn't at fault?

Thank you.

Re: Geodata Service with 2-way Replication

TuanVu1 — Thu, 16 Jul 2015 10:52:08 GMT

Dear William Wang,

But I try to do against you that mean I do twoway replica from geodata service to Enterprise database but I got fault.

But I do one way replica from Geodata service to Enterprise Database worked!

Please help me.

Thank you so much in advance !

Vu Huu Tuan

Re: Geodata Service with 2-way Replication

APANetworks — Fri, 08 Jan 2016 05:19:34 GMT

What sort of licencing is required overall for this to work?