topic Re: Replication Tools in Pro in ArcGIS Pro Questions

Replication Tools in Pro

mpboyle — Mon, 16 Jul 2018 14:37:40 GMT

I attended the Esri UC this past week and was a little surprised to learn that Distributed Geodatabase tools are not supported in Pro and there is currently no plan to implement them.

I understand Pro already has tools for downloading a map and working offline, and that seems fine for a replacement for check-in / check-out replicas. However, we rely pretty heavily on the one-way and two-way replica scenarios within our organization.

As an example, we have multiple editing geodatabases typically divided out by departments. As part of our daily workflow, we use the synchronize changes tool to push edits from our production editing geodatabases to two replicated geodatabases, one that is an internal read-only geodatabase where non-editors can read other departmental data and a second that is used exclusively for web services.

We are very happy with this workflow for a number of reasons:

The number of users that can connect to a production editing database is relatively small and controlled
Production database size stays relatively small
High isolation of data ... for schema changes or other processes that require an exclusive schema lock, we only have to notify a small number of users. We can then make the necessary changes and NOT affect any web services or users that are not production editors.

In another example, we have a two-way replica established with a City government in order to replicate 911 datasets between our organizations. This workflow is important to both organizations as it greatly reduces the time it takes to compile datasets for dispatch updates and provides both organizations with current information in on-premise relational databases that are used for other business functions.

I'd like to hear from Esri folks what is being done to provide the same type of workflow with Pro tools? I don't see any other type of workflow that can be used with Pro to accomplish the same type of workflow. Copying datasets is NOT a realistic solution when replication is a much better solution. Neither is truncating or appending data. In both of these scenarios your moving entire datasets where replication will only move edits.

Again, I want to emphasize this is using one-way or two-way replication and NOT check-in / check-out. If there are other tools within Pro or at the Enterprise GIS platform that provide the same workflow, I'd be very interested in learning more about them.

Otherwise, I have to think I'm not the only person that's really disappointed to learn that replication / distributed geodatabase tools are not being implemented with Pro.

Re: Replication Tools in Pro

ThomasColson — Tue, 22 Jun 2021 08:29:30 GMT

https://community.esri.com/ideas/14400

Re: Replication Tools in Pro

Nacogdoches_CountyGIS_Office — Fri, 14 Sep 2018 16:47:14 GMT

While it is extremely disappointing, it looks like the current plan is ESRI is going to get rid of replication: 2018 Esri User Conference Q&A | UC Resources

...In future releases ArcGIS Pro users will be able to perform two way syncing between datasources shared via feature services; this will provide greater equivalency in ArcGIS Pro for replication workflows....

Our office uses replication extensively as well, and I do not necessarily see publishing every feature class in our databases as a separate feature service which each presumably have to be individually synchronized to be a reasonable replacement, especially considering we use one-way replication to a file geodatabase, for similar purposes as you describe, which does not seem like it would be possible with the workflow the linked to ESRI information describes.

Re: Replication Tools in Pro

ThomasColson — Sat, 15 Sep 2018 16:28:25 GMT

That is very disappointing to hear. I'm curious as to ESRI's reasoning for not supporting disconnected database editing. Just as not everyone has the luxury of installing and maintaining Portal, there are an equal number of shops that are not capable of installing and maintaining ArcGIS Server, and for those that can, as the previous comment points out, it is not feasible or realistic to imply that all of ESRI's customers have the capacity to push everything into feature services when you are limited to the number of GIS servers you can deploy and maintain. I hear that SDE-like capabilities getting baked into server are on the horizon, but returning to the original discussion point, what if you can't install server. Or 50 of them, as I'd need in order to host every single enterprise feature class I maintain that needs to be available for offline editing?

Re: Replication Tools in Pro

Anonymous User — Tue, 18 Sep 2018 15:19:41 GMT

Wait... what? I bet they'll eventually support it. Just like your thread on publishing services from Pro. Heavy hitters will weigh in I bet. That's key for places that are all on-premise with multiple geographic locations that are either isolated for security or lack fast enough connection for SDE (ArcStuff still uses way too much network than it should need particularly when editing hence reason for this functionality alone, i.e. editing data between military bases or other concepts which is why we did it... and those are people with the big pockets..)

Re: Replication Tools in Pro

ThomasColson — Tue, 18 Sep 2018 15:40:49 GMT

Yeah, disconnected SDE editing allows for one security posture. Now you have to throw a web server on top of that to get the same functionality, which introduces yet another layer of security to be managed.

Re: Replication Tools in Pro

Nacogdoches_CountyGIS_Office — Tue, 18 Sep 2018 17:03:23 GMT

Plus, now instead of a 2 way replication taking:

2 instances of an ESRI licensed database (ex: MS SQL Server on two different servers)

and arcpy running somewhere to synchronize

The way I read ESRI's info is that under the new model, for similar functionality, it would now take:

2 instances of an ESRI licensed database,

2 instances of ArcGIS Server,

2 instances of Web Adapter for ArcGIS Server,

and since ESRI is clearly pushing to eventually requiring federated portal w/ hosting server:

it also would require 2 instances of ArcGIS Portal,

2 instances of Web Adapter for ArcGIS Portal

and potentially arcpy still running somewhere

I'm not understanding why it is necessary to add 8 additional intermediary components, and the additional associated hardware/processing requirements for running those components, just to achieve the same functionality.

Re: Replication Tools in Pro

MichaelMixon — Fri, 21 Sep 2018 21:14:27 GMT

We are dealing with similar issues in our organization. We run replication with 65 counties nightly in order to move edits from SQL EXPRESS db's to our enterprise environment. The only solution offered by ESRI so far has been to install Pro and maintain ArcMap on all of the county machines in order to maintain our replication capabilities. Unfortunately we are going to delay moving to Pro until a solution is provided by ESRI.

Re: Replication Tools in Pro

DavidWatkins — Tue, 25 Sep 2018 23:40:01 GMT

Thank you for all the great feedback on this thread. We’d like to try to clarify a few things regarding our replication workflows moving forward with ArcGIS Pro. Our general direction is moving from the client/server model to a web GIS services model because we believe that there are inherent advantages in a services architecture. However, we understand that there are a wide variety of workflows and requirements. Our goal is to support those workflows without requiring a solution that is overly complex or requires a large investment on your part. We are still working through our designs with an expectation of having something in a mid-term release of ArcGIS Pro toward the end of 2019. Please continue to utilize the ArcMap replication tools until a next generation solution is available, and continue to post to this thread to describe your core workflows and concerns as we work on support for replication workflows in ArcGIS Pro.

Note: we are working on a blog post that will update what was in the 2018 User Conference FAQ. We will post back to this thread as soon as the blog post is available.

Re: Replication Tools in Pro

Anonymous User — Thu, 27 Sep 2018 16:57:00 GMT

David, I would put forth that since it is not yet possible to group layers in web maps, particularly sublayers of dynamic map services (now called map image services I guess) Hosted data is thus not usable until that is added. web hosted GIS is a non-starter for more than a quick little app. I hope we see that soon. It is becoming more of an issue each day and I probably just need to get LocalLayer widget up and running. And of course, replication allows for working in secure environments, remote locations with slow access, and providing an easy manner of making backups, and scripting workflows, all dependent on replication.

Re: Replication Tools in Pro

KoryKramer — Tue, 22 Jun 2021 08:29:33 GMT

Yup, that is a big request that is coming https://community.esri.com/ideas/9987

Re: Replication Tools in Pro

Anonymous User — Thu, 27 Sep 2018 19:32:56 GMT

Kory Kramer‌

Re: Replication Tools in Pro

ThomasColson — Thu, 27 Sep 2018 22:59:15 GMT

The number one big need for replication for us is:

The remote office that will never, ever, have an internet connection good enough to download even the smallest of feature services for offline editing, not ever 10 points;
The mega-giant project with thousands of points and photo attachments that even with a 64 Core arcgis server and a 100 MB fiber connection, cannot download the whole AOI as an offline editable map where editing will occur over many 40-hour weeks;

In both of those cases, the data steward uses the Arc Cat tools to create a distributed FGDB for editing, and usually thumbdrives it to the editor. This is a pretty common workflow. I'm currently an "editor" in scenario 2, for a public safety application. No way a dataset that size will "not" fail when attempting to "download map". Tried for years since ArcGIS server came out.

Putting an ArcGIS server at either end of that workflow is not feasible for replication, either, if I'm interpreting the previous answer correctly. We cannot afford to stand up ArcGIS servers all over the place when all the customer needs is a new replica FGDB every week.

Re: Replication Tools in Pro

DavidWatkins — Thu, 04 Oct 2018 15:56:36 GMT

Again, thank you for all of your feedback. We just put up a blog post related to this topic. We are working on a design that we believe will meet your requirements. Please take a look at the post when you get a chance and continue to let us know about the different ways you are using geodatabase replication in your organizations by commenting here or in the blog post thread.

Re: Replication Tools in Pro

DarylHochhalter — Mon, 28 Oct 2019 13:45:07 GMT

There is no good reason for ESRI to limit the users ability to use replication which is really all they would be doing by following the course they seem to have chosen. Basically, it would only be available through online services and now called sync. From my perspective, the concept makes no sense.

However, since we now need to provide reasons to keep functionality some of us have used for many years, that works quite well, I will provide this scenario/workflow. We have some data we make available to the public and some data we collect from the public, the data requires different permissions and is exposed in different ways to the public. it is an extra layer of security to publish the services from replicas of our enterprise database. In fact it works quite well to publish services from replica FGDB's for maps that will only be viewed by user.