Using sensitive data in web map/app.

[KenBouchard]

Hi,

We have a hosted layer which contains sensitive data (restricted areas) that can't be share publicly.  However, we need to let the public know if  a location is within a restricted area.  

In summary, showing all the restricted areas at once is impossible because of privacy issue. But it's possible to give an answer to someone if he clicks at a specific location.  (I know, this is crazy but this is what we have to deal with...)

My idea was to create a Map/ExB and make the layer invisible (no colour). That way it would not show the whole layer but would still be able to pull information from the layer.  To have this happens, I'll need to make the hosted layer public...

So, my questions are:

What does making a hosted layer public does exactly?  

Can I limit the usage of the layer to a specific map/app or will the layer be searchable and usable by everyone in ArcGIS Online?

Thanks,

Ken

[AdrianWelsh]

Ken,

This is an interesting scenario. I think you are on the right approach with what your idea is, though, like you mentioned, it isn't exactly ideal. I will attempt to answer your questions. Making a the layer public is exactly that, it is public and yes, it is 'searchable', but not always easily discoverable. I am not sure about limiting the use but I think what might help you is making a hosted feature layer view:

https://doc.arcgis.com/en/arcgis-online/manage-data/create-hosted-views.htm

This will allow you to make like a copy of the layer and only allow certain parts of it viewable to the public. This might get you a little further along the way, but it still isn't the most ideal scenario.

[RyanUthoff]

I think your solution is going to depend on how "sensitive" your data is. If you're going to make this layer "publicly" available, you need to be prepared for people to access the entire dataset. Yes, you can "hide" it by obscuration, but it doesn't eliminate the risk of someone being able to get their hands on it.

For example, it only takes me about 60 seconds to open an EB app, open dev options in my browser to retrieve the internet traffic and see the rest endpoint of the feature service, then add it to ArcGIS Pro or another web map in AGOL and then I'm able to freely access the layer however I like within the capabilities of how the feature service was published. This potentially includes being able to change the symbology, visibility, etc. so I can see the entire layer.

At the end of the day, the most important question that needs to be asked (imo) is how much risk are you and/or your organization willing to take with this "sensitive data"? Is security by obscuration good enough? Or is it vitally important no one have access to the entire dataset? If it's vitally important no one have access to the dataset, it shouldn't be publicly accessible to begin with.

As another person commented, hosted service views are a good way to limit access to data, although I'm not sure how it could be applied to your scenario for when someone clicks on a specific location.

I don't really have much more advice for your situation for AGOL. If you were using ArcGIS Enterprise, you might be able to create a custom SOI or EB widget to accomplish what you want (although that goes waaaay over my head). But with AGOL, you can't use custom widgets or SOIs, so your options might be limited.

[BobBooth1]

One thing that might work would be to have a widget to allow the user to add a point, and then on a machine running elsewhere, keep a script running to check the point feature service at regular intervals, compare it to a locally stored copy of the sensitive data, and update a Yes/No attribute on the point depending on whether the point falls in the sensitive area or not.

The attribute could control the point symbology.

[BobBooth1]

Someone could still "brute force" discover the sensitive areas by automating submission of lots of points, though.

[Grant-S-Carroll · Accepted solution]

Hi Ken,

It is possible to do this, and we have done something similar in the past where we needed to display data to the public, but that data was under licence so it could only be shared with the app and no other application should be allowed access to it.

What we did was keep the original layer as is, shared only to those who were allowed to access it, eg not public. Next we created a hosted feature layer view of that layer, that only had the required fields, in this case only an OBJECTID and the geometry. This was also only shared to the a group (not publicly).

When then took the URL for the hosted feature layer and then used the Add Item -> URL within ArcGIS Online to add the view back in to ArcGIS Online.

This then recognised that the layer was secured and asked for credentials. These were provided for a service type account that had access to the layer, these credentials were then saved with the layer. This forces ArcGIS Online to proxy the request.

[cid:85301051-e91a-43cc-8899-fe9170df605d]

This new proxied layer then has the ability to have the domains that can access the layer restricted, this is via the Limit Usage section of the newly added layer proxied layer. In our case, this restricted access only the application that we has built. This meant that even if users tried to sniff out the layer by viewing the network traffic, they were unable to access the rest end point, or open the layer in other tools such as ArcGIS Pro.

You should be able to grab the layer URL from the normal place within you application. This is a more complex workflow, and you can get to the point where even ArcGIS Online cannot access the URL, which is the case with our application. In this instance, we added in a the normal layer to the web map for development, then when we went live, we used AGO Assistant to modify the URL of the layer. As with anything like this I would recommend thoroughly testing the access constraints.

[TimWestern]

I like this idea.  I wonder if you could leverage a relationship that could be queried for the existence of a relationship of a location and something in the restricted layer with even less real data going out.

[KenBouchard · Accepted solution]

@RyanUthoff  The layer is sensitive and can't be accessed by anyone. Your scenario of someone "sniffing" the browser traffic to access the data is exactly what I'm worried about. Personally, I don't know how to do that, but there are a lot of people smarter than me who would be able to do it... That's the main issue, how can I test the security of a method when I know I don't have all that knowledge...

@BobBooth1  Yes, we know that with "brute force" someone could automate something to submit thousands of points and basically create their own map of the restricted areas. This is something my organization is willing to risk...
(We can't release the layer, but we wouldn't be held responsible if someone created their own layer. We are walking the fine line of Canadian privacy laws here. People have a right to know if a property is in a restricted area before they buy it, but we can't just bluntly point out all the properties that are restricted).

@Grant-S-Carroll  I found this technical paper after reading your post... 

https://support.esri.com/en-us/knowledge-base/how-to-limit-access-to-secured-hosted-services-or-map-...

Is this what you are referring to? Would this protect the layer so that no one could access it in any way, change the symbology, etc?

Thanks guys!  I feel I'm  on the right track here! 🙂

Ken

[Grant-S-Carroll]

Hi Ken,

Yes this is exactly what I'm talking about, you can limit access to the layer so that only the application you have shared it in to is able to access it, this means no ability to change anything in the layer or change any of the properties. As I said, this also prevents someone from taking the URL and loading that into any other client such as ArcGIS Pro or their own ArcGIS Online/Enterprise. It will even prevent access via the normal REST interface, eg just taking the URL and pasting it in to a browser.