Replication Tools in Pro

[mpboyle]

I attended the Esri UC this past week and was a little surprised to learn that Distributed Geodatabase tools are not supported in Pro and there is currently no plan to implement them.

I understand Pro already has tools for downloading a map and working offline, and that seems fine for a replacement for check-in / check-out replicas.  However, we rely pretty heavily on the one-way and two-way replica scenarios within our organization.

As an example, we have multiple editing geodatabases typically divided out by departments.  As part of our daily workflow, we use the synchronize changes tool to push edits from our production editing geodatabases to two replicated geodatabases, one that is an internal read-only geodatabase where non-editors can read other departmental data and a second that is used exclusively for web services.

We are very happy with this workflow for a number of reasons:

1. The number of users that can connect to a production editing database is relatively small and controlled
2. Production database size stays relatively small
3. High isolation of data ... for schema changes or other processes that require an exclusive schema lock, we only have to notify a small number of users.  We can then make the necessary changes and NOT affect any web services or users that are not production editors.

In another example, we have a two-way replica established with a City government in order to replicate 911 datasets between our organizations.  This workflow is important to both organizations as it greatly reduces the time it takes to compile datasets for dispatch updates and provides both organizations with current information in on-premise relational databases that are used for other business functions.

I'd like to hear from Esri folks what is being done to provide the same type of workflow with Pro tools?  I don't see any other type of workflow that can be used with Pro to accomplish the same type of workflow.  Copying datasets is NOT a realistic solution when replication is a much better solution.  Neither is truncating or appending data.  In both of these scenarios your moving entire datasets where replication will only move edits.

Again, I want to emphasize this is using one-way or two-way replication and NOT check-in / check-out.  If there are other tools within Pro or at the Enterprise GIS platform that provide the same workflow, I'd be very interested in learning more about them.

Otherwise, I have to think I'm not the only person that's really disappointed to learn that replication / distributed geodatabase tools are not being implemented with Pro.

[ThomasColson]

https://community.esri.com/ideas/14400 

[Nacogdoches_CountyGIS_Office]

While it is extremely disappointing, it looks like the current plan is ESRI is going to get rid of replication: 2018 Esri User Conference Q&A | UC Resources   

...In future releases ArcGIS Pro users will be able to perform two way syncing between datasources shared via feature services; this will provide greater equivalency in ArcGIS Pro for replication workflows....

Our office uses replication extensively as well, and I do not necessarily see publishing every feature class in our databases as a separate feature service which each presumably have to be individually synchronized to be a reasonable replacement, especially considering we use one-way replication to a file geodatabase, for similar purposes as you describe, which does not seem like it would be possible with the workflow the linked to ESRI information describes. 

[ThomasColson]

That is very disappointing to hear. I'm curious as to ESRI's reasoning for not supporting disconnected database editing. Just as not everyone has the luxury of installing and maintaining Portal, there are an equal number of shops that are not capable of installing and maintaining ArcGIS Server, and for those that can, as the previous comment points out, it is not feasible or realistic to imply that all of ESRI's customers have the capacity to push everything into feature services when you are limited to the number of GIS servers you can deploy and maintain. I hear that SDE-like capabilities getting baked into server are on the horizon, but returning to the original discussion point, what if you can't install server. Or 50 of them, as I'd need in order  to host every single enterprise feature class I maintain that needs to be available for offline editing?

[Anonymous User]

Wait... what? I bet they'll eventually support it. Just like your thread on publishing services from Pro.  Heavy hitters will weigh in I bet. That's key for places that are all on-premise with multiple geographic locations that are either isolated for security or lack fast enough connection for SDE (ArcStuff still uses way too much network than it should need particularly when editing hence reason for this functionality alone, i.e. editing data between military bases or other concepts which is why we did it... and those are people with the big pockets..) 

[ThomasColson]

Yeah, disconnected SDE editing allows for one security posture. Now you have to throw a web server on top of that to get the same functionality, which introduces yet another layer of security to be managed. 

[Nacogdoches_CountyGIS_Office]

Plus, now instead of a 2 way replication taking:

2 instances of an ESRI licensed database (ex: MS SQL Server on two different servers)

and arcpy running somewhere to synchronize

The way I read ESRI's info is that under the new model, for similar functionality, it would now take:

2 instances of an ESRI licensed database,

2 instances of ArcGIS Server, 

2 instances of Web Adapter for ArcGIS Server,

and since ESRI is clearly pushing to eventually requiring federated portal w/ hosting server:

it also would require 2 instances of ArcGIS Portal,

2 instances of Web Adapter for ArcGIS Portal

and potentially arcpy still running somewhere

I'm not understanding why it is necessary to add 8 additional intermediary components, and the additional associated hardware/processing requirements for running those components, just to achieve the same functionality.

[MichaelMixon]

We are dealing with similar issues in our organization. We run replication with 65 counties nightly in order to move edits from SQL EXPRESS db's to our enterprise environment. The only solution offered by ESRI so far has been to install Pro and maintain ArcMap on all of the county machines in order to maintain our replication capabilities. Unfortunately we are going to delay moving to Pro until a solution is provided by ESRI.

[DavidWatkins]

Thank you for all the great feedback on this thread.  We’d like to try to clarify a few things regarding our replication workflows moving forward with ArcGIS Pro. Our general direction is moving from the client/server model to a web GIS services model because we believe that there are inherent advantages in a services architecture. However, we understand that there are a wide variety of workflows and requirements. Our goal is to support those workflows without requiring a solution that is overly complex or requires a large investment on your part.  We are still working through our designs with an expectation of having something in a mid-term release of ArcGIS Pro toward the end of 2019.  Please continue to utilize the ArcMap replication tools until a next generation solution is available, and continue to post to this thread to describe your core workflows and concerns as we work on support for replication workflows in ArcGIS Pro.

Note: we are working on a blog post that will update what was in the 2018 User Conference FAQ. We will post back to this thread as soon as the blog post is available.

[Anonymous User]

David, I would put forth that since it is not yet possible to group layers in web maps, particularly sublayers of dynamic map services (now called map image services I guess) Hosted data is thus not usable until that is added. web hosted GIS is a non-starter for more than a quick little app. I hope we see that soon. It is becoming more of an issue each day and I probably just need to get LocalLayer widget up and running. And of course, replication allows for working in secure environments, remote locations with slow access, and providing an easy manner of making backups, and scripting workflows, all dependent on replication.