Collector: Offline Syncing Fails to Retain Photo Attachments

[PaigeLester1]

 I am using Collector version 10.4.2 with iOS devices including both iPhone and iPad. My editable feature service is hosted on ArcGIS Online, along with a few other non-editable layers present in the web map. My editable feature service was a point feature class shared from a file geodatabase using ArcMap 10.3.1, and includes subtypes and various domains. We are also using our own tile basemaps.

The task was to send staff out to survey different geographic areas where they would collect 27 – 28 points each, and take four (4) photos at each point. This operation was conducted disconnected by four different groups simultaneously. Once back in the office, connected to Wi-Fi, the offline syncs successfully uploaded the points, but the attachments failed. Over two days, using different Wi-Fi connections, and even instructing them to collect less information at a time between syncs still proved to fail to retain the photo attachments.

Is there a limit to the file size for the offline sync upload for photo attachments with the Collector app? What is the limit? Once the sync fails, where have my photos gone? Are they retrievable at that point (able to still be uploaded and attached to their point data), or do I have one chance at success or they are lost forever?

 A workaround that I attempted was to instruct staff to save each photo to the device after each submission, which is time-consuming and averts from using the Collector app as designed. However, the photo metadata is stripped with this method, so this option is useless.

Either Collector is going to be a reliable data collection method or it’s going to be a toy with no use or real value. Which is it, ESRI?

Please advise.

[MichaelDavis3]

Yes there is a limit at which a large sync will fail to completely upload.  The specific limit seems to be dependent on a combination of upload size and network speed.  This behavior has been well documented on these forums.

Our frustration with the unreliability of data sync led us to move away from collector in favor of custom Runtime apps, and later move away from geodatabase attachments entirely in favor of photoInfo table that links to a photo files uploaded to a custom service endpoint.  We have run projects with over 50,000 collected photos this way with no issues, but I can't imagine trying to do this with Collector and an AGOL service.  So much energy would be wasted managing the collection (breaking things up into "Areas", stoping to upload after X number of photos collected, etc...) it just wouldn't be cost effective.

If you do want to continue with Collector I'd strongly recommend moving to a feature service hosted on an on-premise ArcGIS Server and published on Portal.  With this configuration it is possible to retrieve the uploaded database files in the case of a failed upload and convert them to File Geodatabsae to manually add it to your master DB.  With AGOL there is no way to access these files.

Good luck!

[Anonymous User]

Depending on whether the device thinks the sync went through or not, there may still be an offline files on the device from which you can manually pull edits. 

Here are the steps to manually download your data from Collector:

IOS

-- Connect iPad to computer

Save geodatabase file to computer:
- In iTunes, select device
- Click “Apps”
- Scroll to “File Sharing”
- Select Collector App
- Click “Save to”
- Save to desired location on computer
*More detailed instructions here: http://support.esri.com/en/knowledgebase/techarticles/detail/44588

In ArcMap 10.3 or higher:
- Search to find "Copy Runtime geodatabase to geodatabase" tool - this tool was specifically designed for this purpose
- Use .geodatabase file from previous step as input
- Select desired output location on computer

- Open Feature Class in ArcMap to confirm that attachments are present 

Android
- Using a file explorer software (such as Windows Explorer), browse to the file directory of Collector for ArcGIS on the Android device, and navigate to the folder containing the disconnected edits.
-In this folder, locate the .geodatabase file and copy it to your local machine.
- In ArcMap 10.3 or higher:
   - Search to find "Copy Runtime geodatabase to geodatabase" tool - this tool was specifically designed for this purpose
   - Use .geodatabase file from previous step as input
   - Select desired output location on computer

- Open geodatabase feature class in ArcMap to confirm that edits made offline are present.

[MichaelDavis3]

It's important to note that the file uploaded to ArcGIS Server (or AGOL) is a diff database containing only the adds/edits/deletes made on the mobile device.  The geodatabase you pull from Collector will have the entire data set, and once converted it will be up to you to sort through and figure out what is new vs. edited vs. deleted.  Depending on the type of work you are doing this can be a very time consuming process.  Having access to the uploaded diff database has been a lifesaver to us on many jobs when sync issues occur.

[KenMarshall]

I'm having the same issue. What bad about my situation is that i can't even preform this side extraction cause I work for the federal government and our firewalls and security is so strong that I can do it. I have to use my home computer to give this a try. But still doesn't help me going into the future in photo and point data gathering.

This new AGOL cloud system is really hurt us on the government side of things and we put a GIS tech team together to work around issues and tell you what were all ready to through in the towel on ESRI and we are probably one of there biggest clients. We can not store our data in the cloud to government rules. What were they thinking there no QA/QC on data anymore it crazy.

I'll give this method a try but not going to data collect this way until they fix it.

Can't wait to retire and be done with ESRI and that is not soon enough. Possible career change after 20 years in GIS is starting to look good.

[MichaelDavis3]

I logged an enhancement request with ESRI through our support dashboard last spring to add support for the new-ish iOS file browser.  Once enabled you would be able to browse the local database files in Collector and email/iMessage/dropbox/sharepoint/etc... them off the device thereby getting around USB/firewall restrictions.  

Last I checked it was slated for consideration but no idea if when it might happen.  It's a shame too because it's literally as simple as adding one line to a .plist file in the Xcode project and it would make so many things easier on users.