Related Data in ArcGIS
Over the past couple of months, I have being working quite a bit with related data in ArcGIS Online and associated apps. In this blog, I am going to discuss some of my experiences to help others get a better understanding of the capabilities available to them and how they can be setup.
What is Related Data?
The answer is in its name! Related data is data that has an association or relationship with another piece of data. I think the easiest way to explain related data is with a real case example:
Lets say you were collecting housing data in a particular area. You want to collect data about the house itself, for example its address, type, condition, date built, etc. You also want to collect data about the residents in each house. For each resident, you want to collect their name, date of birth, occupation, etc. Some houses will obviously have multiple residents. If you were to store all this data in the one table, you would have to have an infinite list of fields for the residents (resident one name, resident one date of birth, resident one occupation, resident two name, resident two date of birth, resident two occupation, etc.). This is obviously inefficient and impractical. In order to deal with this, you setup two separate tables, one for houses, and one for residents. You setup a unique identifier for each row in the houses table, and then for each resident we store a reference to this meaning that you can look at house, and then check what houses are related based on this value. In database lingo, this structure has a 'cardinality' of one to many, whereby the house has an association or relationship with multiple residents. an illustration of this can be found below:
There are two other main types of relationships; one to one and many to many. A one to one relationship refers to cases whereby one record is associated with another record. For example a husband may be associated with his wife. A many to many relationship refers to cases whereby many records are associated with many other records. Students and subjects is the typical example that is used. A student may study multiple subjects at school, while a subject may be studied by multiple students.
Finally relationships can also be nested. You may want to store related data about related data! For example, you may want to gather information about the cars that residents own. In that case, your data would contain information about Houses which may contain many Residents who may own many Cars. You can find out more about Relationships and ArcGIS here.
So how can you use these kind of data structures in ArcGIS? You first design the data structure in your desktop software such as ArcGIS Pro/ArcMap/Survey123 Connect for ArcGIS. Once you have the data structure setup, you can start to edit the data. You can edit through ArcGIS Pro/ArcMap, or your can publish to ArcGIS Online or ArcGIS Enterprise and edit data through the various apps that you configure in ArcGIS Online/Portal for ArcGIS. I have put together some videos which demonstrate how you setup the data structure - this is typically the part which people can struggle with but hopefully the videos help you get a solid start. Once you have the data structure setup and published to ArcGIS Online/ArcGIS Enterprise, you configure the various apps as per usual.
Designing your Data Schema
Related data is supported when using ArcGIS Online hosted feature layers or feature services from ArcGIS for Server 10.3.1 or later. Typically these are designed via ArcGIS for Desktop (ArcGIS Pro or ArcMap), but there is now an option to create them using Survey123 Connect for ArcGIS (once you only need to collect point features). When using related data, it is strongly recommended that you use GlobalIDs when defining relationships. User-maintained relationships are not recommended. Check out the videos below for the workflows that I typically follow. In each video, the same data schema is created so you can compare their setup.
ArcGIS Pro (link to video here in case it doesn't display properly)
ArcMap (link to video here in case it doesn't display properly)
Survey123 for ArcGIS (link to video here in case it doesn't display properly)
Survey123 for ArcGIS can only design data schemas for point features
Mobile Data Editing
Survey123 for ArcGIS (Documentation here)
Add
Update (Attachments not currently supported)
Delete
If you just need to add new point features with related data, Survey123 for ArcGIS is probably your best bet. It has a very intuitive interface and was specifically designed to be used offline. It supports one-to-many feature-to-table relationships. See v2.4 Release Notes (Section: Support for editing repeats) for full notes on editing related records support.
In order to collect related data using Survey123 for ArcGIS, your data schema must have been published directly to ArcGIS Online/Portal for ArcGIS using Survey123 Connect for ArcGIS (unless you want to check its advanced features)
Feature services with related data created in Survey123 Connect for ArcGIS do not currently work in Collector for ArcGIS (Resolved as of Survey123 Connect for ArcGIS v1.10.43)
Collector for ArcGIS (Documentation here)
Add
Update
Delete
Collector supports the following relationships as documented here:
- Feature-to-table relationships (one-to-one and one-to-many)
- Feature-to-feature relationships (one-to-one and one-to-many)
- Table-to-table relationships (when the first table is accessed through a feature)
Once you have your feature service published as documented in the Designing your Data Schema section above, all you have to do is add the feature layer to a webmap and share the webmap with a group. Once you have done so, your map should appear when you login to Collector. You can alter how the attributes can be viewed/edited by configuring the webmap popup for the layer and table.
Further information about related tables in Collector for ArcGIS can be found in this blog post
Data Editing on the Web
A common misconception is that ArcGIS Online does not allow you create apps which can edit related data. OK, until the September 2016 release you couldn't edit related data in Web AppBuilder. But there have been configurable templates that allow you to add and update related data available for some time. Here are some of the options that are currently available to you:
Web AppBuilder for ArcGIS Edit Widget
Add
Update
Delete
This is definitely a popular update from the September 2016 release. This widget gives you the ability to edit related records within Web AppBuilder!
ArcGIS Online Attribute Table (Added 20th December 2016)
Add
Update
Delete
With the December 2016 ArcGIS Online release, the ability to edit attributes directly through the Attribute Table became possible. This is something that was suggested via the ArcGIS Idea's site some time ago. Related data can also be viewed and updated through this interface, so it is quite a useful feature.
Feature Layer Item Details Page
Add
Update
Delete
With the June 2016 ArcGIS Online release, the ability to view and edit related data was added to the item details page. This functionality is quite a nice way of viewing the raw data including its relationships and attachments (pictures, documents, etc.).
Add
Update
Delete
This template is available when you create what's known as a configurable group template. The app looks at all the webmaps shared with the group, and bases the application on these. The key is to check the box beside 'Display related records using the popup configuration'.
Here is a link to a sample app created with this template
Add
Update
Delete
Again, this template is available when you create what's known as a configurable group template. The app looks at all the webmaps shared with the group, and bases the application on these. The key is to check the box beside 'Display related records using the popup configuration'.
Here is a link to a sample app created with this templateAdd
Update
Delete
This configurable webmap template differs from the previous two apps, in that it is based on a single webmap, not a group.
Here is a link to a sample app created with this templateData Editing on the Desktop
ArcGIS Pro
Should you have ArcGIS Pro, you also have the ability to edit feature services live in ArcGIS Online. This can be done by searching through your portal for a webmap, and opening that webmap. Once this is done you can use the Edit tab to add modify and delete features as necessary! See this documentation for further details.
Edits cannot be undone if modifying ArcGIS Online feature layers
ArcMap
With ArcMap, the workflow isn't quite as tailored as with ArcGIS Pro. You first need to make a local copy of the service, and then sync your local edits with the server. The edits aren't done in real time like with ArcGIS Pro. You can check out this documentation for further details.
Conclusion
There is quite a bit covered in this blog post, but hopefully it will help you get a better understanding of how related data can be used in ArcGIS Online. Please feel free to comment below if I have missed out on any capabilities or made mistakes along the way. I will do my best to keep this up to date as new releases come out.