topic Re: Synchronize data and geodatabase in ArcGIS Pro Questions

Synchronize data and geodatabase

deleted-user-jQabWmRafJc9 — Fri, 19 Oct 2018 12:44:38 GMT

We are building a custom mobile application using the ArcGIS Runtime SDK for iOS.

We have converted our SQL Server database into an enterprise geodatabase and published the feature classes and tables as a feature service to ArcGIS Online for the mobile app to consume and write data into it.

Questions:

What is the recommended method to synchronize the data generated by the mobile app from the feature service back to the geodatabase?
We connect to the SQL Server instance housing our geodatabase in ArcGIS Pro and manage tables and feature classes there. Why is it EXTREMELY slow? The schema is not complex; there are many tables and the data volume is not large.

Re: Synchronize data and geodatabase

JonathanFarmer_oldaccount — Fri, 19 Oct 2018 15:46:00 GMT

Hi Peter,

When you published to AGOL, did you choose to actually host the data in AGOL or did you publish the data to ArcGIS Server and then use the feature service URL to work with the data in AGOL? If you did publish to ArcGIS Server, did you reference the data in the geodatabase or host it in Data Store?
The performance issues you are seeing in Pro could be related to something on the client side or it could be something on the database side. But we are aware of an issue in Pro where the Catalog Pane (the window on the right side of the Pro interface) is quicker to use than the Catalog View (the large window in the middle of the interface). Are you using the Catalog Pane or the Catalog View? If it isn't that simple, we may need some more information on exactly what is slow on your end.

Jonathan

Re: Synchronize data and geodatabase

deleted-user-jQabWmRafJc9 — Sat, 20 Oct 2018 01:45:05 GMT

Hi Jonathan,

Is it possible to schedule a call with you to discuss further? I’d like to show you the workflow we have adopted.

Sent from my iPhone

Re: Synchronize data and geodatabase

ThomasColson — Sat, 20 Oct 2018 13:24:58 GMT

https://community.esri.com/groups/survey123/blog/2016/12/14/migrating-data-from-the-survey-feature-service-to-an-enterprise-geodatabase will sync (one way) your AGOL feature service data to an on-prem SDE instance, via a scheduled task, quite quickly. We are using it to sync collector and S123 app data to our on-prem SQL.
Where is your SDE/SQL on the network, and where is your workstation you are using to manage it? Same subnet, Server in a remote data center? Amazon hosted instance?
Verson of SQL? Version of SDE? What type of storage? Geometry? Geography? How are your spatial indexes performing? Any joins or relates on those many tables? If so, how are the join fields indexed?
How is your SQL set up? Everything on one physical spinning disk? Log and Temp files on a separate high-speed SAN or SSD? Full daily back daily to truncate those log files?

I've found, before calling TS with a SDE issue, to work through and solve all of the DB vendor performance issues first. On that note, please do call TS. Many of us are suffering horrible performance issues with SDE <-> Pro and more people experiencing the same problem means it likely will get fixed.

Re: Synchronize data and geodatabase

deleted-user-jQabWmRafJc9 — Mon, 22 Oct 2018 01:01:59 GMT

Hi Thomas,

Thanks for the suggestion on the synchronization method.

Regarding the SQL Server, it’s an on-premises instance ( SQL Server 2017) in our data center. We can connect to and operate our database in this instance fairly quickly via SQL Server Management Studio.

The workflow we adopted is basically as follows:

1. Build a database for our app on the SQL Server instance

2. Use ArcGIS for Desktop to run the Enable Enterprise Geodatabase tool to enable the database as an enterprise geodatabase

3. Set up relationship classes and indexes among the feature classes and tables

4. Drag the feature classes and tables over to a map within ArcMap or ArcGIS Pro

5. Publish them as a web layer (feature service) to the ArcGIS Online portal

Re: Synchronize data and geodatabase

ThomasColson — Mon, 22 Oct 2018 01:23:17 GMT

100% convinced the issue is with your relationship classes and indexes. How do they perform in SSMS when doing joins via select? Do them all, full join, left join, etc...look at query execution plan on each and see what's getting hit the most with cost. Again, rule out any DB issues first. Any performance difference between Pro and Map when "Dragging" Over? Do some zoom, pan, change symbology, bulk edits using a join. Need more details on "Publish them as a web layer (feature service) to the ArcGIS Online portal", are you publishing to a hosting server where SQL is a registered DB? A federated Server? A standalone GIS Server? How is auth setup? IWA? SSL? In your OP you mention AGOL....have you migrated this as a test environment to PTL? There are numerous "pain points" for a performance hit in enterprise. Again, TS, not Geonet, is going to ultimately solve this problem, but they're going to ask the same, and many more, detailed questions about your environment. Think instruct a robot to make a jelly sandwich. Specifically, how are you defining the indexes, and are they fragmenting already?

Re: Synchronize data and geodatabase

deleted-user-jQabWmRafJc9 — Mon, 22 Oct 2018 12:41:36 GMT

Thanks Thomas,

ArcGIS is still relatively new to our app dev team. We’ll optimize the database starting with indexes and relationship classes as you suggested.

Everything seems pretty smooth in SSMS.

Both ArcGIS Pro and ArcMap are slow ( e.g. 5 sec to right-click a table to display the context menu and 30 sec to get the properties ; no noticeable differences are observed between the two.

Sent from my iPhone

Re: Synchronize data and geodatabase

ThomasColson — Mon, 22 Oct 2018 14:26:28 GMT

What if you install Map or Pro on the same Server that SQL is on? Same response time?

Tech Support, 1 888 377 4575 Option 2, Option 1, Customer #.

They can walk you through doing a SQL trace and SDE Intercept, which will pinpoint the problem.

Re: Synchronize data and geodatabase

JonathanFarmer_oldaccount — Mon, 22 Oct 2018 16:13:31 GMT

Hi Peter,

I agree with Tom that Support should troubleshoot this performance issue. If you haven't already, I'd recommend calling the number Tom provided and letting them create a case for you. We can run some tools on this end to figure out what's going on.

In regards to question 1 though, the recommended way to sync the data back and forth, if you could let me know your response to my question earlier that would help determine the best workflow there. Or if you'd like to send me an email directly, that's fine too. jfarmer@esri.com

Thanks,

Jonathan

Re: Synchronize data and geodatabase

deleted-user-jQabWmRafJc9 — Tue, 23 Oct 2018 05:24:19 GMT

Thanks Jonathan,

I've sent an email to you with a diagram to illustrate our current workflow in terms of data synchronization.

Regards

Peter

Re: Synchronize data and geodatabase

deleted-user-jQabWmRafJc9 — Tue, 23 Oct 2018 05:29:53 GMT

Thanks Thomas,

I don't have the permission to install ArcMap or ArcGIS Pro on the same server where the SQL Server is. But we'll see if there will be any performance difference after we have optimized the database before we decide to call the Technical Support.

Re: Synchronize data and geodatabase

deleted-user-h_c0FKMnA3-a — Tue, 30 Oct 2018 11:01:20 GMT

Almost the same question:
Published a hosted feature service layer to AGOL for use with the Collector app.
Made some edits (Status has been changed) and added new points

How do we get the edits back in to the geodatabase? Append doesn't work (duplicates) but what does?
Maybe I'm looking over the obvious, but I'm stuck at this moment

Using ArcGIS Pro 2.2.# Standard, a fgdb, AGOL and the Collector App

Kind regards,
Martin

Re: Synchronize data and geodatabase

JonathanFarmer_oldaccount — Tue, 30 Oct 2018 14:40:16 GMT

Hi Martin,

This article deals specifically with attachments but the first few steps are still applicable to your question. This would be a good way to get the edits made in Collector back into the original file geodatabase.

How To: Transfer attachments created in Collector for ArcGIS from a hosted feature service to the original feature class…

Re: Synchronize data and geodatabase

deleted-user-h_c0FKMnA3-a — Wed, 31 Oct 2018 06:57:50 GMT

Thank you Jonathan for your quick answer!
Going to have a look at it today.

Kind regards,
Martin