Workflow(s) for merging AGOL hosted data edits into Desktop FGDB

6796
5
Jump to solution
05-17-2016 06:10 AM
JeffreyWarren
New Contributor III

Good morning all,

We currently host all of our feature services on AGOL.  This will be the first time that I have pulled the data off of AGOL and back in to ArcGIS Desktop.  I'm looking for workflow ideas to get the data off of the fgdb (created from AGOL) and merged back in to my original dataset.  The field crew has gone out and updated a lot of attributes and I want to bring those over.  I don't want to do anything with the geometry as I have moved several.  All I want is the attribution as well as attachments/pictures they assigned.

Anyone have any workflow ideas they are willing to share?  I'd like to eventually automate the process if at all possible.

The only way I can think of off the top of my head is to create a join (using Facility ID) and do a field calc., but didn't know if anyone had any other ideas.  Our data is in the LGIM schema, but for now I'm focusing on Waste Water.  But I have other utilities waiting once I get this one finished.

Thanks in advance,

Jeff

1 Solution

Accepted Solutions
EmilieBrill-Duisberg
New Contributor II

Hi Jeff,

This is a challenge that Enterprise geodatabases and ArcGIS for Server services are well-suited for.  However, when working with file geodatabases and hosted feature services, you can get your newly collected attributes from your Collector data into your old feature class using joins and field calculations.  Transferring the attachments is a bit trickier, but follows the same basic logic.  You can use the following workflow to port your Collector attachments over to your previous geometry without bringing along the geometry created in the Collector app.  This workflow could be made into a ModelBuilder model or Python script for automation.

1.  Export your ArcGIS Online feature service, edited in Collector, to a new file geodatabase and download it to your local machine.

0.png

-1.png

2.  The file geodatabase contains a relationship class which connects the features to the attachments using a GlobalID and a REL_GLOBALID field:

step2.png

3.  Perform an attribute join to append the attribute fields from the downloaded feature class to your original feature class.  This join is based on a unique ID that exists in both tables and is necessary because we need to get the GlobalID field created by ArcGIS Online when the photos were collected into your original feature class as shown below.  It is a good practice to export to a new feature class after the join has been created to prevent any unexpected behavior caused by an active join.

step3.png

4.  Next, perform an attribute join to append the attribute fields from the downloaded attachment table to the output of Step 3 above.  This join is based on the Global ID and REL_GLOBALID fields:

step4.png


5. It's a good idea once again to make the join permanent by copying the features to a new feature class.  Now, we have a single feature class containing your original geometry and containing the Attachment IDs and the Attachment Names from the Collector data downloaded from ArcGIS Online.  This will be critical for matching up the correct photos and features.

step5.png


6. Create a file directory on your computer and batch download all your attachments using the instructions given at http://support.esri.com/technical-article/000011912.

7.  Add a new text field to the output of Step 5 for the filepaths of your attachments.

8. Populate the field with your attachment filepaths using Field Calculator.  In VB, the code would be:

"<path to the folder with your photos>\ATT" & [ATTACHMENTID] & "_" & [ATT_NAME]

step8.png


9. You should end up with a complete filepath which matches those of the photos you batch downloaded in Step 6:

step9.png


10.  Enable attachments on this feature class using the Enable Attachments tool.  Then use the Add Attachments tool to attach the photo/video attachments:

step10.png


This will match your attachments to the correct feature and create a new relationship class between the final output of the above workflow and an attachment table containing your media.

Best wishes,

Emilie

Esri Support

View solution in original post

5 Replies
EmilieBrill-Duisberg
New Contributor II

Hi Jeff,

This is a challenge that Enterprise geodatabases and ArcGIS for Server services are well-suited for.  However, when working with file geodatabases and hosted feature services, you can get your newly collected attributes from your Collector data into your old feature class using joins and field calculations.  Transferring the attachments is a bit trickier, but follows the same basic logic.  You can use the following workflow to port your Collector attachments over to your previous geometry without bringing along the geometry created in the Collector app.  This workflow could be made into a ModelBuilder model or Python script for automation.

1.  Export your ArcGIS Online feature service, edited in Collector, to a new file geodatabase and download it to your local machine.

0.png

-1.png

2.  The file geodatabase contains a relationship class which connects the features to the attachments using a GlobalID and a REL_GLOBALID field:

step2.png

3.  Perform an attribute join to append the attribute fields from the downloaded feature class to your original feature class.  This join is based on a unique ID that exists in both tables and is necessary because we need to get the GlobalID field created by ArcGIS Online when the photos were collected into your original feature class as shown below.  It is a good practice to export to a new feature class after the join has been created to prevent any unexpected behavior caused by an active join.

step3.png

4.  Next, perform an attribute join to append the attribute fields from the downloaded attachment table to the output of Step 3 above.  This join is based on the Global ID and REL_GLOBALID fields:

step4.png


5. It's a good idea once again to make the join permanent by copying the features to a new feature class.  Now, we have a single feature class containing your original geometry and containing the Attachment IDs and the Attachment Names from the Collector data downloaded from ArcGIS Online.  This will be critical for matching up the correct photos and features.

step5.png


6. Create a file directory on your computer and batch download all your attachments using the instructions given at http://support.esri.com/technical-article/000011912.

7.  Add a new text field to the output of Step 5 for the filepaths of your attachments.

8. Populate the field with your attachment filepaths using Field Calculator.  In VB, the code would be:

"<path to the folder with your photos>\ATT" & [ATTACHMENTID] & "_" & [ATT_NAME]

step8.png


9. You should end up with a complete filepath which matches those of the photos you batch downloaded in Step 6:

step9.png


10.  Enable attachments on this feature class using the Enable Attachments tool.  Then use the Add Attachments tool to attach the photo/video attachments:

step10.png


This will match your attachments to the correct feature and create a new relationship class between the final output of the above workflow and an attachment table containing your media.

Best wishes,

Emilie

Esri Support

View solution in original post

AdrianWelsh
MVP Notable Contributor

Emilie,

This is a great step-by-step workflow that you have here. One thing though, I cannot see most of your images. This is what I see:

Is there a way to get those images to appear?

0 Kudos
EmilieBrill-Duisberg
New Contributor II

Hi, Adrian! 

Sorry about that.  The images should work now.

-Emilie

CodyBarrett1
New Contributor II

Hi Emilie-

Does this take into account multiple photos per one feature(1 to Many)?

0 Kudos
EmilieBrill-Duisberg
New Contributor II

Hi Cody,

Yes, this can work even with a 1:M cardinality of features to attachments.  It is actually possible to perform a 1:M join in ArcGIS (you would do this in Step 4 of the above workflow); this will create duplicate geometry for any feature that has more than one attachment:

  1. Make sure both tables/feature classes are in a file geodatabase.
  2. Create the attribute join as you normally would: right click on the larger table, choose join. Be sure to choose the option to Keep all records.
  3. You will get join validation errors; this is expected.
  4. Right-click on the feature class to which you just applied a join, and export it to a new file geodatabase feature class. After the export open up the table and note that there are now many more records -- instead of only the first matching record being displayed, each matching record has its own line in the table.
  5. Remove the attribute join you created.

Run through the steps to create the folder path to your photos in the output of the 1:M join, and then use this as the Match Table when you run the Add Attachments tool.  The Input Dataset will be the original feature class to which you're trying to add the attachments.  This will match multiple attachments to a single feature as specified in the Input Join Field and Match Join Field, and re-create the 1:M relationship class of the downloaded file geodatabase.