topic Re: Problem adding a secured wms layer from Geoserver to AGOL in ArcGIS Online Questions

Problem adding a secured wms layer from Geoserver to AGOL

SuzanaBarreto — Thu, 20 Jun 2019 14:48:42 GMT

Hi everyone,

I am having some unexpected behaviour when attempting to access a secured layer from Geoserver, in AGOL. I have a GEOSERVER instance that responds to both HTTP and HTTPS requests (it redirects http to https), it has both secured and unsecured layers and has CORS configured.

Problems with addition of domain to trusted server list:

I have tried to add the Geoserver domain to the trusted sites list with some issues -

Adding the domain eo4c-geoserver.envsys.co.uk to the trusted list, results in the following error when trying to add a layer to a map 'The WMS service, https://eo4c-geoserver.envsys.co.uk/geoserver/eo4c_data_delivery/wms?version=1.1.0, cannot be added to the map. It is either not available or you have entered an invalid URL for the type of layer you want to reference.'
Adding the http protocol to the domain (http://eo4c-geoserver.envsys.co.uk) in the trusted servers list seems to work in the sense that I can access and the layer but I am not prompted for a username or password - as this is a secured layer I want to be prompted for credentials.
If I add the domain with the https protocol (https://eo4c-geoserver.envsys.co.uk) it fails to find the resource when attempting to add a layer - the error is once again the same as in point 1 above.

If I don't add the domain to the trusted servers list, I am able to add the layer without any problems and am not prompted for passwords - again undesirable behaviour.

Can anyone please explain why the https protocol appears to be a problem, or for that matter why the domain on its own as in point 1 above fails and why I am able to access the resource without being prompted for a username and password as in point 2 above, or why the behaviour is the same whether the domain is in the trusted list or not.

If this is a CORS config issue, does anyone know what the correct CORS config should be (see my CORS config in attached file).

I have tested that this resource is indeed secured using both Qgis and via get requests in the browser, in both cases I am prompted for credentials.

Any help would be greatly appreciated.

Re: Problem adding a secured wms layer from Geoserver to AGOL

ChrisWhitmore — Fri, 21 Jun 2019 15:26:27 GMT

Hi Suzana,

It doesn't look like CORS is configured correctly - when I add your GeoServer instance (After adding the domain to my org's trusted servers list), I'm not seeing any CORS headers in the response:

It looks like the web server you have set up as a reverse proxy / load balancer may be overriding the CORS configuration set up on your GeoServer instance (but just a guess based on your comments above and what the response headers are returning). There may be other issues downstream but this would seem to be the first obstacle.

Cheers,

Chris

Re: Problem adding a secured wms layer from Geoserver to AGOL

SuzanaBarreto — Mon, 24 Jun 2019 15:53:55 GMT

Hi Chris,

Thanks for the help so far, I have now configured the CORS configuration on the proxy server too and although I can see that the following headers when doing a curl request, I am still experiencing the same issues in regards to adding the domain to the trusted servers for my organisation, and then accessing the resource.

HTTP/1.1 200
Server: nginx/1.14.0 (Ubuntu)
Date: Mon, 24 Jun 2019 15:38:03 GMT
Content-Type: text/xml;charset=UTF-8
Content-Length: 560
Connection: keep-alive
vary: Origin
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Access-Control-Allow-Origin,Access-Control-Allow-Credentials
X-Frame-Options: SAMEORIGIN

Is there anything blatantly obvious that I am missing or do you have any ideas where I might look next?

thanks,

Suzana

Re: Problem adding a secured wms layer from Geoserver to AGOL

ChrisWhitmore — Wed, 26 Jun 2019 06:23:45 GMT

Hi Suzana, for sure..closer but there still seems to be some issues with the CORS headers. I'm not seeing any in the response headers coming back for the initial request to the WMS service (for the capabilities doc). The browser shows a console error that indicates `Access-Control-Allow-Origin` header is not being returned..same thing in the headers you posted above too it looks like. Even though defined as an allowed property in the Access-Control-Expose-Headers list, think it would still need to be returned as its own property.

This is the error message the browser's console shows when adding the wms:

cheers,

Chris

Re: Problem adding a secured wms layer from Geoserver to AGOL

SuzanaBarreto — Wed, 26 Jun 2019 12:46:32 GMT

Hi Chris,

Yes you were right, after having configured the nginx server and also checking that the response contained the 'Access-Control-Allow-Origin: https://envsys.maps.arcgis.com ' header I had further issues with duplicate values for Access-Control-Allow-Credentials: true. I now no longer have any header errors in my JS console but am still not being prompted for credentials. Also doing a capabilities request with version set to 1.1.0 gives me the same errors as above (no JS errors though)

cUrl and headers:

curl -H "Origin: https://envsys.maps.arcgis.com" --head 'https://eo4c-geoserver.envsys.co.uk/geoserver/eo4c_data_delivery/wms?service=WMS&version=1.1.0&request=GetCapabilities'
HTTP/1.1 200
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 26 Jun 2019 10:20:26 GMT
Content-Type: application/vnd.ogc.wms_xml
Content-Length: 168472
Connection: keep-alive
vary: Origin
Access-Control-Allow-Origin: https://envsys.maps.arcgis.com
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Access-Control-Allow-Origin,Access-Control-Allow-Credentials
X-Frame-Options: SAMEORIGIN
Cache-Control: max-age=0, must-revalidate
Content-Disposition: inline; filename=getcapabilities_1.3.0.xml.

preflight request headers:

> Host: eo4c-geoserver.envsys.co.uk
> User-Agent: curl/7.58.0
> Accept: */*
> Origin: https://envsys.maps.arcgis.com
> Access-Control-Request-Method: GET
> Access-Control-Request-Headers: X-Requested-With
>
< HTTP/1.1 204 No Content
< Server: nginx/1.14.0 (Ubuntu)
< Date: Wed, 26 Jun 2019 10:38:35 GMT
< Connection: keep-alive
< Access-Control-Allow-Origin: https://envsys.maps.arcgis.com
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Methods: GET, POST, OPTIONS
< Access-Control-Allow-Headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type
< Access-Control-Max-Age: 1728000
< Access-Control-Expose-Headers: Access-Control-Allow-Origin,Access-Control-Allow-Credentials
< Content-Type: text/plain charset=UTF-8
< Content-Length: 0

I am now pretty confident that the configuration is correct and I know that my layer is secured. I have tested this in firefox and chromium. I do notice that although I can access the layer, in firefox, I cannot view the layer, and in chromium there is a 401 returned on the XHR request - I also cannot view the layer. Is there perhaps some other header that AGOL expects that is missing?

XHR headers:

HTTP/1.1 200
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 26 Jun 2019 12:36:55 GMT
Content-Type: text/xml
Transfer-Encoding: chunked
Connection: keep-alive
vary: Origin
Access-Control-Allow-Origin: https://envsys.maps.arcgis.com
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: Access-Control-Allow-Origin,Access-Control-Allow-Credentials
X-Frame-Options: SAMEORIGIN
Cache-Control: max-age=0, must-revalidate
Content-Disposition: inline; filename=getcapabilities_1.3.0.xml
Content-Encoding: gzip
Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
Access-Control-Allow-Headers: Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type

EDIT**

I have been examining the JS console in firefox and notice that there appear to be 2 requests - the first is an XHR request, this request is made at the point when the get capabilities is requested (get layer is clicked), the second is a request, and I gather this is made when the add layer button is clicked. This request does not have an origin header, and neither does the response.

Re: Problem adding a secured wms layer from Geoserver to AGOL

ChrisWhitmore — Thu, 27 Jun 2019 05:04:27 GMT

hmm everything seems good from what I can tell. It looks like you only allow specific origins to connect? Would it be possible to add https://prodtesting.maps.arcgis.com as well, if so (when i check with this origin instead of yours, i'm not seeing any CORS headers returned)? That should help with debugging.

Thanks,

Chris

Re: Problem adding a secured wms layer from Geoserver to AGOL

SuzanaBarreto — Mon, 01 Jul 2019 08:57:11 GMT

Hi Chris,

Sorry for the delay, I have now added your domain to my server's allowed origins CORS config. Thanks for your time and help so far, it is much appreciated.

Re: Problem adding a secured wms layer from Geoserver to AGOL

ChrisWhitmore — Tue, 02 Jul 2019 21:45:59 GMT

Awesome thanks! There still seems to be something going on with the response. The CORS headers look correct but the browser is expecting a 401 http response rather than a 200. The 401 code indicates to the browser that authentication is required for the request.

Re: Problem adding a secured wms layer from Geoserver to AGOL

SuzanaBarreto — Wed, 03 Jul 2019 08:18:42 GMT

Hi Chris,

yes I understand that, but are you saying that it needs to return a 401 at the point where the get capabilities is first requested? The process appears to be add the URL, click get layers -> XHR request is made 200 returned, select the layer and click get layer -> GET request which returns a 401. I will attempt to make it return the 401 at first attempt.

Thanks again.

Re: Problem adding a secured wms layer from Geoserver to AGOL

SuzanaBarreto — Wed, 03 Jul 2019 08:54:19 GMT

Hi Chris,

Thanks for all your time and help, that resolved the issue. I had secured the layer, but needed to secure the WMS service too.

Re: Problem adding a secured wms layer from Geoserver to AGOL

ChrisWhitmore — Wed, 03 Jul 2019 19:51:12 GMT

Hi Suzana, great, good to hear! Glad to be of help. Let me know if you run into anything else.

Cheers,

Chris

Re: Problem adding a secured wms layer from Geoserver to AGOL

Terralytics — Sun, 18 Sep 2022 18:24:16 GMT

The disadvantage of the solution mentioned above is that end-users always need to login when they want to request/see the WMS. In many occasions this is no option. I found some work-around for this: if you use SQLviews in Geoserver you can add parameters to your request. For example using a guid/projectid. In this way you can make the data freely accessible (so read/write by any roles in geoserver), but you won't see anything until you add a viewparam to your request (for example &viewparams=PROJECTID:f252e0e3fe7534a47989beffe36a291d0e. You can add this as parameter to the request within the mapviewer. In this way the data is secure and still visible without having to login.

Re: Problem adding a secured wms layer from Geoserver to AGOL

PaulLohr — Mon, 11 Mar 2024 12:43:36 GMT

@Terralytics,

This sounds like it could be very helpful.

Do you know if the parameter can be saved with the WMS as an item in AGOL? If so, does this mean security can be handled entirely by AGOL?

Thanks for any help.

Re: Problem adding a secured wms layer from Geoserver to AGOL

Terralytics — Mon, 11 Mar 2024 15:11:47 GMT

@PaulLohr , I didn't check it right now, but I would expect that you can add the parameter as part of the standard URL. I would not call it 'security handling' but more 'not able to guess the url'. It works perfectly for my situation, but if the data is more sensitive I would not go for this solution.