Latest Contributions by ReedHunter

Please update the docs for the registerSyncEnabledGeodatabase to mention that timeouts for that command can be fixed by adjusting the timeoutInterval property of AGSRequestConfiguration.  The default is 60 seconds, and much time was lost determining whether the timeout was coming from the OS, the server, or the Runtime SDK.  Increasing the timeoutInterval property allows for a more forgiving client when a field crew is on a slow network, or once the organization's feature service becomes big enough to start slowing down.  Any mention in the docs of controlling registerSyncEnabledGeodatabase through AGSRequestConfiguration would have saved us a lot of time. The registerSyncEnabledGeodatabase command is a client-side wrapper around a call to the ArcGIS Server "createReplica" command, one of the functions of a sync-enabled feature service. I confirmed through REST endpoint tests after watching web traffic through a Fiddler proxy that the createReplica service itself was well past 70 seconds response time, and that the response content was successful and valid.   This can affect any organization that uses the preplanned disconnected editing workflow once their geodatabase, feature service, and number of users reaches high enough maturity to slow down registration past the default timeout of 60 seconds.  For reference, my organization has ~1000 field data collection devices, just over 100 feature classes, an enterprise geodatabase that has been around for about a year, about 500,000 features, and just over 5000 replicas. We support nightly automatic creation of the latest preplanned offline geodatabase for download, and we typically see about one to two thousand syncs per week.  When we first deployed, offline geodatabase registration never reached 60 seconds, though it would be an issue from some field offices with slow network connections.  Current speed of the createReplica command for our production service is about 80 seconds.   Thank you -Reed   ... View more

After sharing hosted copied featureclass data from 1. (guest) portal <= (distributed collaboration) => to 2.(host) Agol site <= (Partnered Collaboration) => and then  to 3. (partner) Agol site, syncs back from 2 to 1 fail as soon as a partner at 3 makes an edit to the hosted featureclass.   If nobody has thoughts on what might resolve this, I'll file a bug with Esri tech support within the week.   Steps:   Create a distributed collaboration between your Portal server and Agol.  The workspace should support "send and receive" for its data. Create a partnered collaboration between your Agol site and another organization's Agol site.  Make certain that the group members may update items. Create a hosted featureclass in your portal's distributed collaboration workspace, sharing as a copy and not just as a reference.  In our situation we created the featureclass logged in with admin privileges, and owner of the featureclass was the same as the owner/creator of the distributed collaboration as well as the partnered collaboration. Confirm that the featureclass you added in portal is showing up in your Agol site's distributed collaboration workspace. Make edits within both the Agol and portal sites, and then initiate a sync from the portal side's workspace.  Confirm that edits from both sides were synced. Now share the same hosted featureclass from your Agol site to the "group" for your partnered collaboration.  Make sure that the sharing of the featureclass is set to "owner and shared with groups". Make another edit to the featureclass within your portal site, and confirm from a member of the partnered organization that they can see the edit(s) you just made.  In our case we made an insert, and update, and a delete to first a point feature class, and the 2nd time around to a polygon featureclass. Have a member of the partner organization make some edit to the hosted featureclass from within their Agol site.  We did this using the built in web mapping application's editing tools.   Expected Result: The result I expected was for the edits from the partner organization to show up a few minutes later in my portal copy of the feature class.   Result: The edits from the partner Agol organization were visible when viewing the featureclass within ArcGIS Online, but no edits sync back to the portal copy of the data. Further attempts to make edits to the Portal copy of the data are successful, but do not sync up to Agol. Further attempts to edit the data in Agol are successful, but do not sync down to Portal.  In other words the break is at the distributed collaboration sync relationship.   This break is at the featureclass level, not at the entire collaboration.  I know this because I've run this test twice with the same collaborations, and the 2nd hosted featureclass synced beautifully across the distributed collaboration, showing up across the partnered collaboration after the 1st hosted featureclass was failing to sync. When accessing the sync logs at https://webadaptorhost.domain.com/webadaptorname/portaladmin/logs/queryfilter/, the following error is reported:         {             "type": "WARNING",             "message": " Cannot create replica for item '7d67e6c4c7054b4eb46027eebb9c80e5' for peer portal 'IYrj3otxNjPsrTRD' as maximum number ('6') of replicas have been created and not yet synchronized. To share the data for this feature layer, please un-share and re-share the item with the group.",             "time": 1629487329136,             "source": "Sharing",             "machine": "<For security I removed our host server name>",             "user": "",             "code": 219999,             "elapsed": "",             "process": "5856",             "thread": "1",             "methodName": "",             "requestID": ""         }     ] } When I follow the following instructions on geonet to pull up more info on the portal logs while in debug mode, portal doesn't detect any errors: Login to portal's admin site to be ready to switch portal's logging from Warning to Debug mode at https://webadaptorhost.domain.com/webadaptorname/portaladmin/ -> logs -> settings -> edit Connect another web page to https://webadaptorhost.domain.com/webadaptorname/sharing/rest.  Click on Home > Portals > Self > Scroll to the bottom on the page > Collaboration.  Click on the relevant collaboration id > Collaboration Workspaces > Click on the relevant collaboration workspace id > Sync Status  Have a 3rd web page connect to your Portal site's collaboration workspace site to be ready to manually sync the workspace. Switch your portal to Debug mode, then manually trigger the workspace sync, monitor the Sync Status for a few minutes, and then switch your portal back to warning mode when you're done.    Following those instructions at 11:20 on Oct 29, I saw these results.: Status IdSync Execution TypeStatusStartedEndedModifiedStatus Summary 94fab80dc82640a8b4e68c8ad2ef0926scheduledcompletedOct 29, 2021, 11:22:33 AMOct 29, 2021, 11:24:20 AMOct 29, 2021, 11:24:20 AMCompleted sync successfully. 436748d0ad914011aa90853070ccc6c5scheduledcompletedOct 29, 2021, 11:20:16 AMOct 29, 2021, 11:22:07 AMOct 29, 2021, 11:22:07 AMCompleted sync successfully. 14b2f3915626424a97e35e97a04f10b1scheduledcompletedOct 29, 2021, 10:20:16 AMOct 29, 2021, 10:22:13 AMOct 29, 2021, 10:22:13 AMCompleted sync successfully. 7316d07dd5ed439d8abd3b33f865ed70scheduledcompletedOct 29, 2021, 9:20:16 AMOct 29, 2021, 9:22:06 AMOct 29, 2021, 9:22:06 AMCompleted sync successfully. e263112e51e54f5c82de1d950473c300scheduledcompletedOct 29, 2021, 8:20:16 AMOct 29, 2021, 8:22:09 AMOct 29, 2021, 8:22:09 AMCompleted sync successfully. f75ac0ef3f984e688b547f3d38453376scheduledcompletedOct 29, 2021, 7:20:16 AMOct 29, 2021, 7:22:09 AMOct 29, 2021, 7:22:09 AMCompleted sync successfully. d80f5c057a6c4f11968d60a2a725a2a3scheduledcompletedOct 29, 2021, 6:20:16 AMOct 29, 2021, 6:22:15 AMOct 29, 2021, 6:22:15 AMCompleted sync successfully. 9c83a8453b6e434cbd90af47a2fe925dscheduledcompletedOct 29, 2021, 5:20:16 AMOct 29, 2021, 5:22:06 AMOct 29, 2021, 5:22:06 AMCompleted sync successfully. c677b80873e04551a7bcc62326ee8b7fscheduledcompletedOct 29, 2021, 4:20:16 AMOct 29, 2021, 4:22:14 AMOct 29, 2021, 4:22:14 AMCompleted sync successfully. 28909ae4418a43fcabc3d7b3bcecc2f6scheduledcompletedOct 29, 2021, 3:20:16 AMOct 29, 2021, 3:22:07 AMOct 29, 2021, 3:22:07 AMCompleted sync successfully. ce068189cf3a41138fd373e776555af4scheduledin_progressSep 11, 2021, 9:20:16 AMN/ASep 11, 2021, 9:20:16 AMCompleted initial validation. Syncing items to participants. 94077618cf654853801bfc966beefa98scheduledin_progressAug 30, 2021, 5:20:16 AMN/AAug 30, 2021, 5:20:16 AMCompleted initial validation. Syncing items to participants. 8cb980156ab749c4ba5fdf4b844bbcc9 realtime completed Jul 20, 2021, 4:31:39 PM Jul 20, 2021, 4:32:05 PM Jul 20, 2021, 4:32:05 PM Completed sync successfully.   The messages say that the syncs are successful, but the edits from the partnered Agol organization on our Agol site are still missing from the copy on Portal.   ... View more

Symptoms Users of our offline, sync enabled mobile geodatabases were reporting timeouts when trying to register their offline files.     We have around a thousand ArcGIS Runtime SDK for iOS apps supporting disconnected editing.  Nightly automation pre-generates and zips an offline .geodatabase file with the latest state of the geodatabase, and our users will download this file after initial deployment or the rare occasion of recovering from some error state.  After the zipped file is downloaded and expanded, the client app registers it with the feature service for sync support.  This "preplanned workflow" creates a replica in the geodatabase.  Unfortunately having multiple people download and reuse the same file had a side effect that if anyone unregistered their replica generated from that file, anyone else who also downloaded that file could no longer sync.  For this reason we had to never unregister offline files on the fly, and accept an accumulation of replica ids over time.    Over time We created new map services and retired old ones on top of the same geodatabase.  Unknown to us, the time it takes to register a map service gradually lengthened as the replica counts mounted.  Slow networks also cause these map registration timeouts, so we were slow to learn that our users were experiencing this ever more often on fast networks as well.  Several registration attempts often had to be made even on fast networks before registration could succeed.    Diagnosis It turns out we had too many old replicas in our enterprise geodatabase, slowing down the registration of new replicas enough that we experienced blocking timeouts.     Solution Once the replica count was reduced from around 13,000 to around 4,000 the timeouts almost completely stopped.  I uncovered the sql to discover old replica Ids from the enterprise geodatabase's GDB_ITEMS table, and then learned how to identify which were from old, retired feature services versus current production services. Using sql to extract replica IDs just from retired feature services, these IDs were then fed into the ArcGIS Pro unregister replicas command.  Out of caution of possible performance and functional side effects, the script contains a count restraint of how many replicas to process before exiting.   Software Used The python script was run on a Windows computer with ArcGIS Pro installed 2.6.2. 64bit python 3.6.10 was used for development and execution of the script.  The geodatabase was on Sql Server.     The Sql:   Note that all queries against the GDB_ITEMS table are read only.  All updates to replicas work through Esri's application tier.  Be aware that unregistering a replica causes it to be removed from the GDB_ITEMS table.     To find the count of all replicas in GDB_ITEMS regardless of map service of origin, this sql was used:   SELECT COUNT(*)  FROM [TheDatabaseName].[dbo].[GDB_ITEMS]  WHERE [Type] = '5B966567-FB87-4DDE-938B-B4B37423539D'    To acquire the count for a specific map service, include this clause: AND [DatasetInfo1] = 'http://thegisserverhostname/arcgis/rest/services/theMapService/FeatureServer'   Why query for replica counts?  Getting the replica counts down solves the problem, and the script below includes a limit on how many replicas to unregister in a given batch.     If you need to confirm for yourself what the guid '5B966567-FB87-4DDE-938B-B4B37423539D' means and where it came from, you will find it in the GDB_ITEMTYPES table.  Filtering based on this guid allows your queries to ignore all the other geodatabase item types like featureclasses. For a list of available feature services for which replicas have been created, run the following sql against the enterprise geodatabase.  Be careful to only pick feature services from these results that are no longer in use, otherwise you will prevent someone from being able to check-in their edits or to acquire recent edits of others through a sync.   SELECT DISTINCT [DatasetInfo1] from [TheDatabaseName].[dbo].[GDB_ITEMS] WHERE [Type] = '5B966567-FB87-4DDE-938B-B4B37423539D'      Setting up the script to run.   Update the ini file:   Once the url and the count to unregister are acquired, update serviceUrl and replicaCount properties of UnregisterGdbReplicas.ini.   For ease of switching between testing and production, the ini file is separated according to QA vs Prod. A Qa and/or Production .SDE file connection file is required in order to establish access to the geodatabase.      Setting up Python   In order to make this script work, you need a python environment from ArcGIS Pro 2.6.2 or better to have the unregister replicas command available.  You will need to open ArcGIS Pro, and then access "Settings" -> "Python" to first clone the default python environment if you haven't already in order to have write access to the python env.  After that you can add the configparser, sqlalchemy, and pyodbc libraries.   You will also likely need to add the path to that python library's folder to the user's path environment variable. When executing the script, you can either run it in debug inside of visual studio code or from a command prompt.  To run in Visual Studio Code you need to first enter the interpreter path of the cloned python environment's python.exe file.   When executing from the command prompt, it worked better to change directory first to the location of the script, and then execute the command: <path to python env>\python.exe .\UnregisterGdbReplicas.py   In order to confirm that the script is running successfully, it can help to set a breakpoint before and  after the unregister replica call, and then execute sql on the GDB_ITEMS table before and after to look  at the row count for that replica's url.  It was also useful to have code at the end of main that would output the list of replicaIds that had been processed.     Some Disclaimers   The code below is provided as is for reference purposes.  If you use this, expect to have to rewrite  at least part of it for your environment and business needs.  Even in our own environment data could be lost or sync access for field crews could be lost were this script run against the wrong urls.    This code is being shared partly for future reference if this task needs to be performed again in a year or two,  and partly as a proof of concept for anyone else who has a similar problem.  Notice that all these instructions read more like reminder notes how to ramp up quickly next time this is needed.  If a bug is found and a help request posted on this script, it's unlikely to even be noticed until the map registration timeout problem resurfaces, and this script has to be dusted off again.     The Code:   Pasting code into this ui resulted in all python indentation being lost, so I instead posted this to GitHub at   https://github.com/ReedHunter-Wsdot/GdbUnregisterReplicas   ... View more