Approval step for online edits

[ChrisMcLean2]

The online editing (feature access) web service is a great tool for users that do not have a knowledge of the ArcGIS software, but are required to update features on a regular basis. The question is how does one maintain the spatial data quality of the edits, before rolling the data out to a wider audience? 

The editor tracking functionality is great to see who create or modified elements, however there is no step for the approval of those edits. The edits are immediately available for users with view access to make decisions. In order to maintain the integrity of the data, it would be great if an approval step could be integrated with the layer. Where the owner of the data is required to approve the changes before the rest of the organization is able to view the changes.  

A potential solution for this would be to only update the Web Map Service (WMS) using the Feature Service when the changes to the Feature Service have been approved by the data owner. At the moment the WMS is updated as soon as the Feature Service. It should be possible to disconnect these to service and only update manually when approved.

[ThomasEdghill]

Thank you for this idea!

Addressing this idea will be different depending on the type of Feature Service being leveraged. For example, if your Feature Service is based on data coming from a registered Enterprise Geodatabase, it is possible to utilize versioning to create a version for online edits, and share a Feature Service which is referencing this version. This allows for some level of data validation at the database level, where the edits sent to the aforementioned version can undergo QA/QC before being synced back to a parent version.

While the versioning option could be seen as a workaround to this, it does not address Feature Services which use nonversioned archiving; nor does it address Feature Services which are shared to ArcGIS Online or Portal for ArcGIS via a Hosting Server.

While we investigate this idea further, I'd like to understand your workflows here - are you typically using a Feature Service based in an Enterprise Geodatabase? If so, have you explored the option of using versioned data to see if it may meet your needs, or are there other limitations which you are facing?

[ChrisMcLean2]

Hi Thomas,

Thank you for the reply. 

Yes, we are using a Feature service hosted in an Enterprise Geodatabase. The Portal is federated with the server, and the same server acts as the Hosting Server. The editing of features is online, so I'm not sure if the versioning is built for this. 

The overview:

Everything takes place within the Portal Web App environment. Our users are not able to use ArcGIS Desktop to view or approve data. The reason for this model is essentially to mitigate the change management and training required. We do not have a GIS department - there are two of us implementing the Enterprise GIS solution to 3000 employees. We have integrated all the current source systems that they use (mostly CAD and Excel) to the Server database and from there push and pull data. The main purpose for the GIS solution is to allow all the users to have access to view the information. With that comes issues, whereas previously maps were printed and signed as an approval process, now currently the information is available for all users over the web (Portal) to see and make business decisions. The viewing users need to be confident that the data they see is the most up-to-date and has been approved by the responsible person. 

The business requirements:

1) The editor of the data (either in the Source system or Portal online) may not be the approver of the data. 

               - Current solution: AD groups to control access to editor and approver web maps, some SQL

2) The approver requires functionality to see the previous version and the changes made quickly

              - Current solution: Editor tracking, time slider and comparison screen, colour coding

3) The approver must be notified and a system in place to track edits

             - Current solution: SQL back-end, SharePoint front-end

After this Idea post, I worked a little more on the problem and came up with the following work around:

The current solution uses the Editor Tracking functionality to track the edits, there is a field called "Approval" that the editor does not have access to change. The default for this field is "Pending", so any new edits are automatically assigned as "Pending". The approver only have access to the "Pending" field and can not update geometry or change other fields, he/she can change the "Pending" to "Approved" (this is a domain) - either one feature at a time or as a batch (web app tool). Then to display the feature I used some colour changes to make everything that is "Pending" grey for the approver and all the other viewers. Once this changes to "Approved" it adopts the correct colour and line styles. 

The issues:

If the feature is "Approved", and the editor makes changes is does not change the field back to "Pending". This is where some SQL comes in and checks to see if the last_editor field (editor tracking) is equal to the approver user name. If it is then those features are set to "Approved", if it is not then the feature was edited and the field is set to "Pending". At the same time the information is posted to SharePoint for the approver user to get a notification. 

The unfixable (at the moment):

I can not seem to figure out a way to publish a feature service that is symbolized using two fields. For example, the symbology field (what ever that might be) and the Approval field. This would make it a lot easier to manage this process. And I know that I can concatenate the fields together to make and Symbology-Approval field, but then the editor user have the option to immediately draw a feature that will be approved. 

Next steps:

Offline Editing on Web? Offline Editing - COP     GitHub - Esri/offline-editor-js: ArcGIS JavaScript library for handling offline editing and tiling. - but this is not something I'm familiar with implementing

Looking forward to your response,

Chris

[ThomasEdghill]

Hi Chris,

In reading through your overview and current way of tackling these workflows, it sounds like you may still be in a position where versioning could be implemented at the Enterprise Geodatabase level and may be beneficial. As long as the data is still stored in a registered Enterprise Geodatabase, and is not copied into a managed database (such as the ArcGIS Data Store), then the workflow linked in my comment may still be applicable.

Here is an additional resource which outlines the ways that versioned data can be implemented in workflows involving offline edits. The core principle of maintaining a version for field edits which can be posted and reconciled with a parent version following data validation is still applicable here, even if the maps won't be going offline.

If it works for you, I'll reach out via a private message so that we can discuss some of the finer details about your project there and hopefully help to understand your workflow further.

Aside from this, the workflow which you have detailed above is a clever solution, and the idea posting does raise some interesting use-cases where approval processes for online edits are needed. The current "unfixable" which you have referred to is an idea which currently has a corresponding posting on the Ideas site, so you are certainly not alone in this request:  

Publish service that is symbolized on multiple attributes‌

I recommend that you upvote this idea if you have not already done so, so that it gains some more traction on this site. 

Best,

Thomas

[ChrisMcLean2]

Update - and review:

Thanks Thomas, I got the versioning sorted out and it does work very well – there should be a topic on the help desk to be able to do this. It was quite different compared to collector type versioning setup.

The technical setup:

1. Versioning the database with all the bells and whistles – GUID, Editor Tracking

2. Created a child version of the DEFAULT – I called it “Editing”

3. Registered the Enterprise Geodatabase while connected to the “DEFAULT” version

4. Registered it again, but this time connected to the “Editing” version

5. Published a feature service of the DEFUALT version (with sync disabled)

6. Published another feature service while connected to the “Editing” version (also with sync disabled)

7. Published a Geoprocessing Service to reconcile the “Editing” version to the “DEFAULT” version

The results:

1. The editors can now go into the feature service published under the Editing version and make changes

2. The approvers see those changes immediately – and on their Web App have both the DEFAULT and the Editing versions with a swiper and slightly different colours to see what changed

3. In the approver web app I added the GP service to reconcile, so when they click on that they “approve” the changes

4. The viewers see the map service of the DEFAULT version – so they can’t edit and only see what has been approved

5. It works very well, but brings with it its own set of challenges

The challenges:

1. The entire Enterprise Geodatabase is reconciled 
   • But this can be fixed by adding multiple versions for each feature class and publishing GP services to onoly reconcile those versions - so just a lot of admin
2. You will be registering many Enterprise Geodatabase instances, one for each version
3. Additional load on the server to spool the extra services
4. The approver must wither approve all changes, or reject all. They are not able to select certain entities and only approve those.
5. Unknowns... I'm not sure how conflicts will be handled, or if strange things will start to happen.

The summary:

• Three different techniques identified
  1. Versioning
     • The method described above 
  2. Approval Field
     • Colour changes to show which feature entities are still Pending
  3. Query Layers based on the Approval field
     • Not discussed in this post, but possible if you don't want the normal viewer users to see the "Pending" changes at all before they are approved
• Each have their own pro's and con's, and by having all at your disposal it's possible to meet the many approval requirements.

Please add your comments and your ideas. Hopefully, this will be a built in function one day without the complexities of the technical setup. 

All the best and good luck,

Chris